The officially official Devuan Forum!

You are not logged in.

#1 2019-11-14 01:13:43

Altoid
Member
Registered: 2017-05-07
Posts: 278  

Problem with *.iso file

Hello:

I'm trying to put together a very small Devuan install to replace a TCCore installation which lacks the Nouveau drivers I need for my NVidia cards.

To do that, I downloaded the devuan_ascii_2.1_i386_netinst.iso file and once I checked the SHA256SUM, burned it with Xfburn and attempted to install.

The installation failed toward the end with a pop-up notice about not being able to install the software.

To me, it was rather obvious the DVD was at fault and with Xfburn having no integrity check, I booted up the DVD again to check the its integrity and yes, it turned out to be bad.
The isolinux file was compromised.

So I just burned another DVD, this time checking that the DVD drive was clean and burning the *.iso file at a lower speed.
This time I checked the integrity of the DVD before attempting the installation and it also also turned out to be bad but ...

It was bad at the same point: the isolinux file was compromised.

Fearing the worse (a new $ATA DVD burner) I decidec to dd the *.iso file on to a USB stick to install from there.

devuan:~$ sudo dd bs=4M if=devuan_ascii_2.1_i386_netinst.iso of=/dev/sdb

Just in case, this time I booted up and checked the DVD's integrity first.

Guess what?
It also failed the test and with the same issue: a compromised isolinux file.

---
Edit I:

The compromised /isolinux file is ./isolinux/isolinux.bin.

This is from the *.iso file dd'd to a USB stick.

According to md5sum.txt, this it should compute thus:

81d876d6234d3ca002390e7cb361bb61  ./isolinux/isolinux.bin

But File -> Properties -> Digests says it is 3b36f20bc14cf4ad0f046962c4414221.

The specific *.iso file is timestamped as Devuan GNU/Linux 2.1 (ascii) i386 NETINSTALL - 2019-10-21 23:05:54 UTC
---
Edit II:

If I directly mount the downloaded and verified *.iso file with Acetone ISO and check ./isolinux/isolinux.bin with Properties -> Digests, it also says it computes as 3b36f20bc14cf4ad0f046962c4414221 instead of what md5sum.txt states.

ie: 81d876d6234d3ca002390e7cb361bb61, so it would not seem to be an issue with the download, its dd'ing to the USB stick or burning to a DVD. 

Could it possibly be a compromised file within the *.iso?
---
I've never come across something like this before: it has always been a bad *.iso file or a bad burn due to the drive, the media or the software/speed.
But a bad dd?

Any help with this will be appreciated.
Thanks in advance,

A.

Last edited by Altoid (2019-11-14 16:46:26)

Offline

#2 2019-11-18 17:59:09

fsmithred
Administrator
Registered: 2016-11-25
Posts: 1,281  

Re: Problem with *.iso file

I'm looking into this. Below are some collected md5sums on isolinux.bin that I have in various places. I checked the sha256sums on the netinstall isos and they are correct. Both were downloaded today.  I don't know why i386 and amd64 made at or around the same day are different. The isolinux package is not arch-specific.

ascii-backports (2019-02-06  ~bpo9+2)
ea5013e737ba5ea49161a2b646312276  usr/lib/ISOLINUX/isolinux.bin

ascii (current 3:6.03+dfsg-14.1+deb9u1)
81d876d6234d3ca002390e7cb361bb61  /usr/lib/ISOLINUX/isolinux.bin

ascii (isolinux_3%3a6.03+dfsg-14.1_all.deb)
3e58f3f1a85d430556259f27f3b36d1a  usr/lib/ISOLINUX/isolinux.bin

i386 2.1 netinstall iso
3b36f20bc14cf4ad0f046962c4414221  mnt/isolinux/isolinux.bin

amd64 2.1 netinstall iso
bdad948d65c1dea713e1698d04a4e75d  mnt/isolinux/isolinux.bin

amd64 2.1 desktop-live iso (September 20)
bb8555ec5b6af5bebac5bfc08f88a543  mnt/isolinux/isolinux.bin

i386 2.1 desktop-live iso (September 20)
f03d6ecc57dad4524a0cab76b7afab41  mnt/isolinux/isolinux.bin

in live-sdk, possibly beowulf  (Oct 18)
caaa9f76d534f8c710c78cef00388ed1  /home/phred/my-live-sdk/live-sdk/extra/syslinux/isolinux.bin
caaa9f76d534f8c710c78cef00388ed1  /home/phred/my-live-sdk/live-sdk/tmp/devuan-i386-build/binary/isolinux/isolinux.bin

beowulf
5424e654f2bb81acfb7726631a3a0c1b  /usr/lib/ISOLINUX/isolinux.bin

refractasnapshot-base  (this one is old)
a45f9b17d309273e3efa588611141b43

jessie
35c69b13a8771b012d6e86b323827d78  /usr/lib/ISOLINUX/isolinux.bin

wheezy
6b8de496c485f2a82ca5f51eb59887fd  /usr/lib/syslinux/isolinux.bin

Offline

#3 2019-11-18 19:34:38

Altoid
Member
Registered: 2017-05-07
Posts: 278  

Re: Problem with *.iso file

Hello:

fsmithred wrote:

I'm looking into this.

Thank you.

fsmithred wrote:

... some collected md5sums on isolinux.bin that I have in various places.
... sha256sums on the netinstall isos and they are correct.

Yes, that's what seems (to me) odd.
I was expecting this to be just a bad download, but no.

Here's what I have.
I mounted the *.iso files with AcetoneISO and checked the respective isolinux.bin files.

---
devuan_ascii_2.1_amd64_netinst.iso - 21-Oct-2019 08:35 - 319.8 MB (319815680 bytes)
From https://ftp.nluug.nl/pub/os/Linux/distr … aller-iso/
---

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
bdad948d65c1dea713e1698d04a4e75d  isolinux.bin
groucho@devuan:~/virtual-drives/1/isolinux$

But the respective md5sum.txt says otherwise:

81d876d6234d3ca002390e7cb361bb61  ./isolinux/isolinux.bin

---
devuan_ascii_2.1_i386_netinst.iso - 22-Oct-2019 02:47 366.0 MB (365953024 bytes)
From: https://ftp.nluug.nl/pub/os/Linux/distr … aller-iso/
---

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
3b36f20bc14cf4ad0f046962c4414221  isolinux.bin
groucho@devuan:~/virtual-drives/1/isolinux$

But the respective md5sum.txt says otherwise:

81d876d6234d3ca002390e7cb361bb61  ./isolinux/isolinux.bin

Since the *.iso files are intact and the isolinux.bin files are not arch specific what I think we are seeing in the sample I am posting about is that there are (at least) two versions but none of them with the correct md5sum  ie:  81d876d6234d3ca002390e7cb361bb61  ./isolinux/isolinux.bin

I have not checked other sources as I expect they are mirrored.
Let me know if I can help out in any way.

Thanks in advance.

A.

Offline

#4 2019-11-18 20:18:18

fsmithred
Administrator
Registered: 2016-11-25
Posts: 1,281  

Re: Problem with *.iso file

cancel that. If you read it, forget that I said it.

Sill looking...

Offline

#5 2019-11-18 21:29:06

ralph.ronnquist
Administrator
From: Clifton Hill, Victoria, AUS
Registered: 2016-11-30
Posts: 381  

Re: Problem with *.iso file

Note that isolinux/isolinux.bin is the "El Torito boot image", and that mkisofs does something to the first 64 bytes on transfer from disk to CD/DVD image, i.e. when the .iso is created.

Thus, the published md5sum in md5sum.txt for isolinux/isolinux.bin should be ignored.

Probably someone could work out exactly what changes and how, and unless it includes some checksum of the content (esp the md5sum.txt file) and/or some time stamp, those changes could be applied to the source prior to checksumming it.

Offline

#6 2019-11-18 23:21:23

Altoid
Member
Registered: 2017-05-07
Posts: 278  

Re: Problem with *.iso file

Hello:

ralph.ronnquist wrote:

... that mkisofs does something to the first 64 bytes on transfer from disk to CD/DVD image, i.e. when the .iso is created.

I have seen this in the ascii 2.1 netinstall *.iso files only.
The ascii 2.0.0 netinstall *.iso files don't seem to have this problem.

ie: media verification using the tool available within the 2.0.0 netinstall media does not fail and the isolinux.bin file produces the correct md5sum.

Best,

A.

Offline

#7 2019-11-18 23:33:12

ralph.ronnquist
Administrator
From: Clifton Hill, Victoria, AUS
Registered: 2016-11-30
Posts: 381  

Re: Problem with *.iso file

It's a good observation. The ISO sets of 2.0.0 and 2.1 where produced in different ways, and obviosuly the 2.1 method is somewhat lacking in care. A re-release of the latter would indeed be a good thing.

thanks,

Ralph.

Offline

#8 2019-11-19 00:40:35

fsmithred
Administrator
Registered: 2016-11-25
Posts: 1,281  

Re: Problem with *.iso file

Thanks, Ralph. I was planning to ask you about this, but I got delayed checking md5sums on more isos.

Offline

#9 2019-11-19 00:51:45

Altoid
Member
Registered: 2017-05-07
Posts: 278  

Re: Problem with *.iso file

Hello:

ralph.ronnquist wrote:

... ISO sets of 2.0.0 and 2.1 where produced in different ways ...

OK.

But if the isolinux.bin files are not (fsmithred says, I wouldn't know) arch specific, why is the 2.1_amd64 version different from the 2.1_i386?
ie: I'm assuming that they are different because they produce different md5sums but maybe functionally they are the same.

To top it off, neither of the md5sums they produce are the same md5sum in the md5sum.txt file.
Makes no sense ...

ralph.ronnquist wrote:

A re-release of the latter would indeed be a good thing.

Sure ...
But I'd pull them from the servers asap to do some forensics on those two files to see just what happened.
Cannot be too careful.

ralph.ronnquist wrote:

thanks ...

No need.
You are the guys moving the Dev1 project along.

Cheers,

A.

Offline

#10 2019-11-19 08:22:06

ralph.ronnquist
Administrator
From: Clifton Hill, Victoria, AUS
Registered: 2016-11-30
Posts: 381  

Re: Problem with *.iso file

Yes, the 2.0.0 ISO builder(s) took the easy way out and excluded isolinux.bin from the md5sum.txt file.
That's of course one way to avoid a false negative smile

Offline

#11 2019-11-19 10:14:57

Altoid
Member
Registered: 2017-05-07
Posts: 278  

Re: Problem with *.iso file

Hello:

ralph.ronnquist wrote:

... took the easy way out and excluded isolinux.bin from the md5sum.txt file.

Maybe not the easy way out.
It probably slipped past them.

Now it has to get fixed, somehow.

Has it been the same with previous versions?

ralph.ronnquist wrote:

... one way to avoid a false negative

Indeed ...  8^* !

That with respect to the 2.0.0 *.isos.

But with respect to the 2.1 *.isos, if the 2.1_amd64 version is supposed to be the same (?) as the 2.1_i386 version, why are they different? (ie: produce different md5sums)
ie: I'm assuming that they are the *exact* same file because they are not arch specific, as fsmithred pointed out earlier.

This would imply (?) that whatever happened to the original source file was affected differently when going through the process of compiling the respective *.iso.
Am I making sense here?

The plot seems to thicken ... 

---
Edit:

This is the data I have wrt some ascii 2.0.0 isos I have in storage:

ascii_2.0.0_i386_netinst.iso

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
b6838c8e3c68b64b813cfab7ea0a200e  isolinux.bin
groucho@devuan:~/virtual-drives/1/isolinux$

devuan_ascii_2.0.0_i386_minimal-live.iso

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
f03d6ecc57dad4524a0cab76b7afab41  isolinux.bin
groucho@devuan:~/virtual-drives/2/isolinux$

devuan_ascii_2.0.0_i386_dvd-1.iso

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
4709734ad535226a10bef3ece43ed9d4  isolinux.bin
groucho@devuan:~/virtual-drives/1/isolinux$

devuan_ascii_2.0.0_i386_desktop-live.iso

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
4709734ad535226a10bef3ece43ed9d4  isolinux.bin
groucho@devuan:~/virtual-drives/1/isolinux$

Note that devuan_ascii_2.0.0_i386_dvd-1.iso and devuan_ascii_2.0.0_i386_desktop-live.iso
seem to share the same isolinux.bin file.

---

Best,

A.

Last edited by Altoid (2019-11-19 12:37:13)

Offline

#12 2019-11-19 13:03:35

fsmithred
Administrator
Registered: 2016-11-25
Posts: 1,281  

Re: Problem with *.iso file

The 2.1 minimal-live isos both have the same isolinux.bin. The amd64 and i386 are both built using the same xorriso/mkisofs command. The desktop-live i386 and amd64 use slightly different commands, because the amd64 supports uefi. I'm a little bit surprised that the isolinux.bin in the 2.1 minimal-lives are the same as the 2.0.0 you posted.

f03d6ecc57dad4524a0cab76b7afab41

I checked some Refracta isos and the i386 and amd64 are different (assuming for the same reason as the desktop-live) but the md5sums are consistent over time and only change occasionally. Possibly with new versions of isolinux or something else.

Getting back to the original post, I think the problem with installing software is not related to isolinux.bin. Sometimes the installer fails to get all the software and you need to repeat that step. This was more common with jessie isos, but I think it's happened with ascii, too.

Offline

#13 2019-11-19 15:44:32

Altoid
Member
Registered: 2017-05-07
Posts: 278  

Re: Problem with *.iso file

Hello:

fsmithred wrote:

... 2.1 minimal-live isos both have the same isolinux.bin.

OK.

fsmithred wrote:

... amd64 and i386 are both built using the same xorriso/mkisofs command.

OK.

fsmithred wrote:

... desktop-live i386 and amd64 use slightly different commands ...

OK.

It's all rather over my head/pay grade.
Eventually, I guess ...

fsmithred wrote:

... surprised that the isolinux.bin in the 2.1 minimal-lives are the same as the 2.0.0 you posted.

f03d6ecc57dad4524a0cab76b7afab41

I computed them in a teminal so it would be hard to make a typo and checked the values twice.

A coincidence?
 
This isolinux.bin only came up because of my checking the installation media with the install's verification routine.
The *.iso file was intact after download and was correctly written to media.

But as you pointed out, the issue did not come up in the 2.0.0 *.isos because isolinux.bin was not in the respective md5sum.txt file. 
It was not scanned, ergo no error was computed.   =-)

fsmithred wrote:

... problem with installing software is not related to isolinux.bin. Sometimes the installer fails ...

Agreed ...
As I mentioned in my email, due to a USB stick with not enough space.

I suppose then that all is well?

Thanks for taking the time to look into this.

Best,

A.

Offline

#14 2019-11-20 19:27:05

czeekaj
Member
Registered: 2019-06-12
Posts: 24  

Re: Problem with *.iso file

I have tried the minimal and live desktop recently because the ISO i have been snapshotting only started recently having some difficulty doing a full disk encryption.
Also I wanted to start with a minimal install and only include the packages I need to have a lighter iso.

However, downloading and verifying the check sum seemed to match for the iso.

When I dd it onto a USB. Try to boot to it, I get grub but than an error it wont boot the kernel. Missing magic number you need to load the kernel first'.

Not sure what went wrong. Usually that's all I need to do is dd it to a bootable USB and check the hash.

Offline

#15 2019-11-20 20:00:50

Altoid
Member
Registered: 2017-05-07
Posts: 278  

Re: Problem with *.iso file

Hello:

czeekaj wrote:

... to start with a minimal install ...

Same here.

czeekaj wrote:

... verifying the check sum seemed to match ...

It has to match exacly.
Seemed won't do. 

Just pulling your leg ...   =-D

czeekaj wrote:

Not sure what went wrong.

In my limited experience, the first check you do is on the *.iso file you downloaded.

Then you do a check on ther file burned on the CD/DVD or USB.

I found a way to do this here:
https://askubuntu.com/questions/547332/ … -boot-disk

URL wrote:

To check the integrity of a usb boot disk, first find the size of the iso image with

stat -c '%s' imagename.iso

This will output an image size which you can enter in place of <imagesize> in the command below.
The next command sends (through a pipe) all bytes corresponding to the size of the image to the md5sum command:

sudo head -c <imagesize> /dev/sdb1 | md5sum

You can compare this with the md5sum of your .iso file.

md5sum imagename.iso

If md5sums are different then there was an issue while copying the data.
If md5sums are the same, you have successfully checked data integrity on your usb disk!

The third and final step is to boot the installaiton *.iso and run a check on the installation media itself with the tool available in the menu.
Although, as we have seen in this thread, it can sometimes give a false positive.

Cheers,

A.

Offline

#16 2019-11-30 01:15:27

czeekaj
Member
Registered: 2019-06-12
Posts: 24  

Re: Problem with *.iso file

Yeah I noticed the hash was changing after running dd command.

I got the hash to match if I dd it into a partition rather than just the root of the drive. smile

Offline

#17 2019-11-30 15:04:04

FM81
Member
Registered: 2017-09-16
Posts: 15  

Re: Problem with *.iso file

Typically an ISO-image (with ISOLINUX.BIN) is created by the following:

mkisofs -o NAME_OF_ISOIMAGE -b isolinux/isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table ISO_ROOT_OF_FILES

The parameter "-boot-info-table" is essential, to get it working. But this parameter does the following: it modifies isolinux.bin itself during the build of the isofile. A previously created checksum will never match after that! That's why this special file should be always excluded from such checksum-tests.

Or in other words: It is not a bug, it is a feature ...

Best Regards, FM_81

Offline

#18 2019-11-30 16:49:03

chris2be8
Member
Registered: 2018-08-11
Posts: 68  

Re: Problem with *.iso file

ralph.ronnquist wrote:

Note that isolinux/isolinux.bin is the "El Torito boot image", and that mkisofs does something to the first 64 bytes on transfer from disk to CD/DVD image, i.e. when the .iso is created.

Why not use something like tail --bytes=+64 isolinux.bin | md5sum to skip the first 64 bytes. That should tell you if the bulk of the file is OK.

Chris

Last edited by chris2be8 (2019-11-30 16:49:31)

Offline

#19 2019-11-30 21:17:33

czeekaj
Member
Registered: 2019-06-12
Posts: 24  

Re: Problem with *.iso file

I apologize I still don't understand. I have always just used dd to get a bootable iso.

mkisofs -o NAME_OF_ISOIMAGE -b isolinux/isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table ISO_ROOT_OF_FILES

Do I replace NAME_OF_ISOIMAGE with the location of the iso?
also what do I replace ISO_ROOT_OF_FILES with?
I have working Iso, However,  I had used refracta snapshot to get it. It dd s just fine. (So I am in no rush to try a minimal and build a new one but I noticed I have over 1500 packages trying to debloat a bit smile.

Issue is, isn't -b bios boot only?(wow this conjunction xD) I would like to get a EFI booting Iso. would I use -e instead?
I am happy with bios boot on my thinkpad, however, sometime EFI is easier to work with as I'm planning to make a more minimal Iso. (so I may have 2 partitions temporarily for easier backups)  Maybe, eventually I'll remove the efi grub tools.

PS: I have not tried rufus on windows. last time I used that was probably couple years ago when I did my first bare metal install  not counting wheezy on a pi. Now I'd prefer to build the iso on linux. DD has worked fine up untill now. If mkisofs is the best way to do things I'd gladly switch and figure it out.

Last edited by czeekaj (2019-11-30 21:36:08)

Offline

#20 2019-11-30 23:35:40

fsmithred
Administrator
Registered: 2016-11-25
Posts: 1,281  

Re: Problem with *.iso file

I think FM81 was showing the mkisofs command as an example. You don't need to run that command - it's used to make the .iso file, not to put it on a usb.

You can make an iso that boots uefi or bios with refractasnapshot. See the config file. See the xorriso command in the script itself. There are a couple of options needed for uefi. If you have questions, we can talk about it in a separate thread.

NAME_OF_ISO would be the name of the .iso file you are creating. With just the file name and no path, it would be created in the current working directory. Add a path if you want it to land someplace else.

ISO_ROOT_OF_FILES is the top-level directory of the file tree you want to put inside the iso. Typically, it contains an isolinux directory and a live directory.

Offline

Board footer