Problem with *.iso file

Altoid · 2019-11-14 01:13:43

Hello:

I'm trying to put together a very small Devuan install to replace a TCCore installation which lacks the Nouveau drivers I need for my NVidia cards.

To do that, I downloaded the devuan_ascii_2.1_i386_netinst.iso file and once I checked the SHA256SUM, burned it with Xfburn and attempted to install.

The installation failed toward the end with a pop-up notice about not being able to install the software.

To me, it was rather obvious the DVD was at fault and with Xfburn having no integrity check, I booted up the DVD again to check the its integrity and yes, it turned out to be bad.
The isolinux file was compromised.

So I just burned another DVD, this time checking that the DVD drive was clean and burning the *.iso file at a lower speed.
This time I checked the integrity of the DVD before attempting the installation and it also also turned out to be bad but ...

It was bad at the same point: the isolinux file was compromised.

Fearing the worse (a new $ATA DVD burner) I decidec to dd the *.iso file on to a USB stick to install from there.

devuan:~$ sudo dd bs=4M if=devuan_ascii_2.1_i386_netinst.iso of=/dev/sdb

Just in case, this time I booted up and checked the DVD's integrity first.

Guess what?
It also failed the test and with the same issue: a compromised isolinux file.

---
Edit I:

The compromised /isolinux file is ./isolinux/isolinux.bin.

This is from the *.iso file dd'd to a USB stick.

According to md5sum.txt, this it should compute thus:

81d876d6234d3ca002390e7cb361bb61 ./isolinux/isolinux.bin

But File -> Properties -> Digests says it is 3b36f20bc14cf4ad0f046962c4414221.

The specific *.iso file is timestamped as Devuan GNU/Linux 2.1 (ascii) i386 NETINSTALL - 2019-10-21 23:05:54 UTC
---
Edit II:

If I directly mount the downloaded and verified *.iso file with Acetone ISO and check ./isolinux/isolinux.bin with Properties -> Digests, it also says it computes as 3b36f20bc14cf4ad0f046962c4414221 instead of what md5sum.txt states.

ie: 81d876d6234d3ca002390e7cb361bb61, so it would not seem to be an issue with the download, its dd'ing to the USB stick or burning to a DVD.

Could it possibly be a compromised file within the *.iso?
---
I've never come across something like this before: it has always been a bad *.iso file or a bad burn due to the drive, the media or the software/speed.
But a bad dd?

Any help with this will be appreciated.
Thanks in advance,

A.

Last edited by Altoid (2019-11-14 16:46:26)

fsmithred · 2019-11-18 17:59:09

I'm looking into this. Below are some collected md5sums on isolinux.bin that I have in various places. I checked the sha256sums on the netinstall isos and they are correct. Both were downloaded today. I don't know why i386 and amd64 made at or around the same day are different. The isolinux package is not arch-specific.

ascii-backports (2019-02-06  ~bpo9+2)
ea5013e737ba5ea49161a2b646312276  usr/lib/ISOLINUX/isolinux.bin

ascii (current 3:6.03+dfsg-14.1+deb9u1)
81d876d6234d3ca002390e7cb361bb61  /usr/lib/ISOLINUX/isolinux.bin

ascii (isolinux_3%3a6.03+dfsg-14.1_all.deb)
3e58f3f1a85d430556259f27f3b36d1a  usr/lib/ISOLINUX/isolinux.bin

i386 2.1 netinstall iso
3b36f20bc14cf4ad0f046962c4414221  mnt/isolinux/isolinux.bin

amd64 2.1 netinstall iso
bdad948d65c1dea713e1698d04a4e75d  mnt/isolinux/isolinux.bin

amd64 2.1 desktop-live iso (September 20)
bb8555ec5b6af5bebac5bfc08f88a543  mnt/isolinux/isolinux.bin

i386 2.1 desktop-live iso (September 20)
f03d6ecc57dad4524a0cab76b7afab41  mnt/isolinux/isolinux.bin

in live-sdk, possibly beowulf  (Oct 18)
caaa9f76d534f8c710c78cef00388ed1  /home/phred/my-live-sdk/live-sdk/extra/syslinux/isolinux.bin
caaa9f76d534f8c710c78cef00388ed1  /home/phred/my-live-sdk/live-sdk/tmp/devuan-i386-build/binary/isolinux/isolinux.bin

beowulf
5424e654f2bb81acfb7726631a3a0c1b  /usr/lib/ISOLINUX/isolinux.bin

refractasnapshot-base  (this one is old)
a45f9b17d309273e3efa588611141b43

jessie
35c69b13a8771b012d6e86b323827d78  /usr/lib/ISOLINUX/isolinux.bin

wheezy
6b8de496c485f2a82ca5f51eb59887fd  /usr/lib/syslinux/isolinux.bin

Altoid · 2019-11-18 19:34:38

Hello:

fsmithred wrote:

I'm looking into this.

Thank you.

fsmithred wrote:

... some collected md5sums on isolinux.bin that I have in various places.
... sha256sums on the netinstall isos and they are correct.

Yes, that's what seems (to me) odd.
I was expecting this to be just a bad download, but no.

Here's what I have.
I mounted the *.iso files with AcetoneISO and checked the respective isolinux.bin files.

---
devuan_ascii_2.1_amd64_netinst.iso - 21-Oct-2019 08:35 - 319.8 MB (319815680 bytes)
From https://ftp.nluug.nl/pub/os/Linux/distr … aller-iso/
---

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
bdad948d65c1dea713e1698d04a4e75d  isolinux.bin
groucho@devuan:~/virtual-drives/1/isolinux$

But the respective md5sum.txt says otherwise:

81d876d6234d3ca002390e7cb361bb61 ./isolinux/isolinux.bin

---
devuan_ascii_2.1_i386_netinst.iso - 22-Oct-2019 02:47 366.0 MB (365953024 bytes)
From: https://ftp.nluug.nl/pub/os/Linux/distr … aller-iso/
---

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
3b36f20bc14cf4ad0f046962c4414221  isolinux.bin
groucho@devuan:~/virtual-drives/1/isolinux$

But the respective md5sum.txt says otherwise:

81d876d6234d3ca002390e7cb361bb61 ./isolinux/isolinux.bin

Since the *.iso files are intact and the isolinux.bin files are not arch specific what I think we are seeing in the sample I am posting about is that there are (at least) two versions but none of them with the correct md5sum ie: 81d876d6234d3ca002390e7cb361bb61 ./isolinux/isolinux.bin

I have not checked other sources as I expect they are mirrored.
Let me know if I can help out in any way.

Thanks in advance.

A.

fsmithred · 2019-11-18 20:18:18

cancel that. If you read it, forget that I said it.

Sill looking...

ralph.ronnquist · 2019-11-18 21:29:06

Note that isolinux/isolinux.bin is the "El Torito boot image", and that mkisofs does something to the first 64 bytes on transfer from disk to CD/DVD image, i.e. when the .iso is created.

Thus, the published md5sum in md5sum.txt for isolinux/isolinux.bin should be ignored.

Probably someone could work out exactly what changes and how, and unless it includes some checksum of the content (esp the md5sum.txt file) and/or some time stamp, those changes could be applied to the source prior to checksumming it.

Altoid · 2019-11-18 23:21:23

Hello:

ralph.ronnquist wrote:

... that mkisofs does something to the first 64 bytes on transfer from disk to CD/DVD image, i.e. when the .iso is created.

I have seen this in the ascii 2.1 netinstall *.iso files only.
The ascii 2.0.0 netinstall *.iso files don't seem to have this problem.

ie: media verification using the tool available within the 2.0.0 netinstall media does not fail and the isolinux.bin file produces the correct md5sum.

Best,

A.

ralph.ronnquist · 2019-11-18 23:33:12

It's a good observation. The ISO sets of 2.0.0 and 2.1 where produced in different ways, and obviosuly the 2.1 method is somewhat lacking in care. A re-release of the latter would indeed be a good thing.

thanks,

Ralph.

fsmithred · 2019-11-19 00:40:35

Thanks, Ralph. I was planning to ask you about this, but I got delayed checking md5sums on more isos.

Altoid · 2019-11-19 00:51:45

Hello:

ralph.ronnquist wrote:

... ISO sets of 2.0.0 and 2.1 where produced in different ways ...

OK.

But if the isolinux.bin files are not (fsmithred says, I wouldn't know) arch specific, why is the 2.1_amd64 version different from the 2.1_i386?
ie: I'm assuming that they are different because they produce different md5sums but maybe functionally they are the same.

To top it off, neither of the md5sums they produce are the same md5sum in the md5sum.txt file.
Makes no sense ...

ralph.ronnquist wrote:

A re-release of the latter would indeed be a good thing.

Sure ...
But I'd pull them from the servers asap to do some forensics on those two files to see just what happened.
Cannot be too careful.

ralph.ronnquist wrote:

thanks ...

No need.
You are the guys moving the Dev1 project along.

Cheers,

A.

ralph.ronnquist · 2019-11-19 08:22:06

Yes, the 2.0.0 ISO builder(s) took the easy way out and excluded isolinux.bin from the md5sum.txt file.
That's of course one way to avoid a false negative

Altoid · 2019-11-19 10:14:57

Hello:

ralph.ronnquist wrote:

... took the easy way out and excluded isolinux.bin from the md5sum.txt file.

Maybe not the easy way out.
It probably slipped past them.

Now it has to get fixed, somehow.

Has it been the same with previous versions?

ralph.ronnquist wrote:

... one way to avoid a false negative

Indeed ... 8^* !

That with respect to the 2.0.0 *.isos.

But with respect to the 2.1 *.isos, if the 2.1_amd64 version is supposed to be the same (?) as the 2.1_i386 version, why are they different? (ie: produce different md5sums)
ie: I'm assuming that they are the *exact* same file because they are not arch specific, as fsmithred pointed out earlier.

This would imply (?) that whatever happened to the original source file was affected differently when going through the process of compiling the respective *.iso.
Am I making sense here?

The plot seems to thicken ...

---
Edit:

This is the data I have wrt some ascii 2.0.0 isos I have in storage:

ascii_2.0.0_i386_netinst.iso

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
b6838c8e3c68b64b813cfab7ea0a200e  isolinux.bin
groucho@devuan:~/virtual-drives/1/isolinux$

devuan_ascii_2.0.0_i386_minimal-live.iso

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
f03d6ecc57dad4524a0cab76b7afab41  isolinux.bin
groucho@devuan:~/virtual-drives/2/isolinux$

devuan_ascii_2.0.0_i386_dvd-1.iso

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
4709734ad535226a10bef3ece43ed9d4  isolinux.bin
groucho@devuan:~/virtual-drives/1/isolinux$

devuan_ascii_2.0.0_i386_desktop-live.iso

groucho@devuan:~/virtual-drives/1/isolinux$ md5sum isolinux.bin
4709734ad535226a10bef3ece43ed9d4  isolinux.bin
groucho@devuan:~/virtual-drives/1/isolinux$

Note that devuan_ascii_2.0.0_i386_dvd-1.iso and devuan_ascii_2.0.0_i386_desktop-live.iso
seem to share the same isolinux.bin file.

---

Best,

A.

Last edited by Altoid (2019-11-19 12:37:13)

fsmithred · 2019-11-19 13:03:35

The 2.1 minimal-live isos both have the same isolinux.bin. The amd64 and i386 are both built using the same xorriso/mkisofs command. The desktop-live i386 and amd64 use slightly different commands, because the amd64 supports uefi. I'm a little bit surprised that the isolinux.bin in the 2.1 minimal-lives are the same as the 2.0.0 you posted.

f03d6ecc57dad4524a0cab76b7afab41

I checked some Refracta isos and the i386 and amd64 are different (assuming for the same reason as the desktop-live) but the md5sums are consistent over time and only change occasionally. Possibly with new versions of isolinux or something else.

Getting back to the original post, I think the problem with installing software is not related to isolinux.bin. Sometimes the installer fails to get all the software and you need to repeat that step. This was more common with jessie isos, but I think it's happened with ascii, too.

Altoid · 2019-11-19 15:44:32

Hello:

fsmithred wrote:

... 2.1 minimal-live isos both have the same isolinux.bin.

OK.

fsmithred wrote:

... amd64 and i386 are both built using the same xorriso/mkisofs command.

OK.

fsmithred wrote:

... desktop-live i386 and amd64 use slightly different commands ...

OK.

It's all rather over my head/pay grade.
Eventually, I guess ...

fsmithred wrote:

... surprised that the isolinux.bin in the 2.1 minimal-lives are the same as the 2.0.0 you posted.
f03d6ecc57dad4524a0cab76b7afab41

I computed them in a teminal so it would be hard to make a typo and checked the values twice.

A coincidence?

This isolinux.bin only came up because of my checking the installation media with the install's verification routine.
The *.iso file was intact after download and was correctly written to media.

But as you pointed out, the issue did not come up in the 2.0.0 *.isos because isolinux.bin was not in the respective md5sum.txt file.
It was not scanned, ergo no error was computed. =-)

fsmithred wrote:

... problem with installing software is not related to isolinux.bin. Sometimes the installer fails ...

Agreed ...
As I mentioned in my email, due to a USB stick with not enough space.

I suppose then that all is well?

Thanks for taking the time to look into this.

Best,

A.

czeekaj · 2019-11-20 19:27:05

I have tried the minimal and live desktop recently because the ISO i have been snapshotting only started recently having some difficulty doing a full disk encryption.
Also I wanted to start with a minimal install and only include the packages I need to have a lighter iso.

However, downloading and verifying the check sum seemed to match for the iso.

When I dd it onto a USB. Try to boot to it, I get grub but than an error it wont boot the kernel. Missing magic number you need to load the kernel first'.

Not sure what went wrong. Usually that's all I need to do is dd it to a bootable USB and check the hash.

Altoid · 2019-11-20 20:00:50

Hello:

czeekaj wrote:

... to start with a minimal install ...

Same here.

czeekaj wrote:

... verifying the check sum seemed to match ...

It has to match exacly.
Seemed won't do.

Just pulling your leg ... =-D

czeekaj wrote:

Not sure what went wrong.

In my limited experience, the first check you do is on the *.iso file you downloaded.

Then you do a check on ther file burned on the CD/DVD or USB.

I found a way to do this here:
https://askubuntu.com/questions/547332/ … -boot-disk

URL wrote:

To check the integrity of a usb boot disk, first find the size of the iso image with
stat -c '%s' imagename.iso
This will output an image size which you can enter in place of <imagesize> in the command below.
The next command sends (through a pipe) all bytes corresponding to the size of the image to the md5sum command:
sudo head -c <imagesize> /dev/sdb1 | md5sum
You can compare this with the md5sum of your .iso file.
md5sum imagename.iso
If md5sums are different then there was an issue while copying the data.
If md5sums are the same, you have successfully checked data integrity on your usb disk!

The third and final step is to boot the installaiton *.iso and run a check on the installation media itself with the tool available in the menu.
Although, as we have seen in this thread, it can sometimes give a false positive.

Cheers,

A.

czeekaj · 2019-11-30 01:15:27

Yeah I noticed the hash was changing after running dd command.

I got the hash to match if I dd it into a partition rather than just the root of the drive.

FM81 · 2019-11-30 15:04:04

Typically an ISO-image (with ISOLINUX.BIN) is created by the following:

mkisofs -o NAME_OF_ISOIMAGE -b isolinux/isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table ISO_ROOT_OF_FILES

The parameter "-boot-info-table" is essential, to get it working. But this parameter does the following: it modifies isolinux.bin itself during the build of the isofile. A previously created checksum will never match after that! That's why this special file should be always excluded from such checksum-tests.

Or in other words: It is not a bug, it is a feature ...

Best Regards, FM_81

chris2be8 · 2019-11-30 16:49:03

ralph.ronnquist wrote:

Note that isolinux/isolinux.bin is the "El Torito boot image", and that mkisofs does something to the first 64 bytes on transfer from disk to CD/DVD image, i.e. when the .iso is created.

Why not use something like tail --bytes=+64 isolinux.bin | md5sum to skip the first 64 bytes. That should tell you if the bulk of the file is OK.

Chris

Last edited by chris2be8 (2019-11-30 16:49:31)

czeekaj · 2019-11-30 21:17:33

I apologize I still don't understand. I have always just used dd to get a bootable iso.

mkisofs -o NAME_OF_ISOIMAGE -b isolinux/isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table ISO_ROOT_OF_FILES

Do I replace NAME_OF_ISOIMAGE with the location of the iso?
also what do I replace ISO_ROOT_OF_FILES with?
I have working Iso, However, I had used refracta snapshot to get it. It dd s just fine. (So I am in no rush to try a minimal and build a new one but I noticed I have over 1500 packages trying to debloat a bit .

Issue is, isn't -b bios boot only?(wow this conjunction xD) I would like to get a EFI booting Iso. would I use -e instead?
I am happy with bios boot on my thinkpad, however, sometime EFI is easier to work with as I'm planning to make a more minimal Iso. (so I may have 2 partitions temporarily for easier backups) Maybe, eventually I'll remove the efi grub tools.

PS: I have not tried rufus on windows. last time I used that was probably couple years ago when I did my first bare metal install not counting wheezy on a pi. Now I'd prefer to build the iso on linux. DD has worked fine up untill now. If mkisofs is the best way to do things I'd gladly switch and figure it out.

Last edited by czeekaj (2019-11-30 21:36:08)

fsmithred · 2019-11-30 23:35:40

I think FM81 was showing the mkisofs command as an example. You don't need to run that command - it's used to make the .iso file, not to put it on a usb.

You can make an iso that boots uefi or bios with refractasnapshot. See the config file. See the xorriso command in the script itself. There are a couple of options needed for uefi. If you have questions, we can talk about it in a separate thread.

NAME_OF_ISO would be the name of the .iso file you are creating. With just the file name and no path, it would be created in the current working directory. Add a path if you want it to land someplace else.

ISO_ROOT_OF_FILES is the top-level directory of the file tree you want to put inside the iso. Typically, it contains an isolinux directory and a live directory.

berghsg · 2020-02-20 13:53:20

MY RESULTS:
md5sum devuan_ascii_2.1_amd64_dvd-1.iso AEFA3DC1461FD0A7792E27ABD171C27A (ORIGINAL ISO)
head -c 4675600384 /dev/sdf1 | md5sum 904a76b91eded9993b7b37f186227ff - (FLASH DRIVE)

I have managed to install after a second attempt devuan ascii, how ever my mda5sum / sha256sum check showed a discrepancy on the contents (from image) loaded to my flash drive, yet i managed to install ascii.
As ralph.ronnquist mentioned on 2019-11-18 23:29:06 that isolinux/isolinux.bin is left out because its md5sum count changes after installing on flash drive.
I am wondering if it applies the same to the rest of the files. ???
I ask this question since I had many problems with installing devuan jessie and getting it to work, until i did a sha256sum check on my downloaded file and found it corrupt. I re-downloaded the devuan_jessie_1.0.0_amd64_CD.iso file, sha256sum checked out, and re-installed. A number of the problems vanished including trying to get hplip to see my scanner (via hp-setup). So the same could be said for ascii.

How ever Altoid makes a very compelling argument on 2019-11-20 22:00:50 alluding to the fact that corruption can also occur in writing to flash drive. As can be seen by my results above my image seems a problem yet my installation went fine after second attempt.

fsmithred mentioned on 2019-11-19 15:03:35 a valid point: which i personally have experienced on both jessie and ascii and that is for whatever reason the installing of software and even boot loader (grub2) fails, on re-trying 'it came right'. Not a very stable installation though. I don't recall having this on ubuntu or linux mint, or puppy, knoppix etc! Not sure if it is due to large amounts of data needing to be downloaded to complete the installation, yet i expected devuan_ascii_2.1_amd64_dvd-1.iso to be a far more easier installation in this regard, since it did not require much download to complete the installation. Most files were in the iso. Yet it too was a problem.

IN CONCLUSION
So far my installed ascii does not show any signs of corruption. I installed a couple of programs that i use and they all went fine. I even installed my hp scanner the quickest way i found, by using hp-setup -i in command line.
I think it will be safe to say that if the flash drive being used is in question buy another one (good make), sha256sum check the downloaded iso with the sha256 values provided by devuan and write to flash as normal.
During installation: Make sure apt package manager is updated correctly, retry if having trouble. Retry any software failures and or bootloader install failures.
This is about the best advice i can give.
Devuan iso's have brilliant partitioning utility in their installer but the installation with setup has been the most difficult out of any linux so far. I entertain all the nonsense because i want to boot up with initd NOT systemd! I found devuan very fast on an old Intel Core 2 duo core E7500 PC, 4 Gb ram.

ralph.ronnquist · 2020-02-20 20:15:01

When written to a flash, I assume you'd written it to /dev/sdf and not /dev/sdf1 (which is the first partition) and maybe the mdfsum from there matches the iso file?

head -c 4675600384 /dev/sdf | md5sum

berghsg · 2020-03-05 21:02:15

Hi Ralph
Yes i did try checking /dev/sdf as well but don't get the correct matching hash numbers.
I will have to try the head command exercise another time. Since then I re-used the flash drive for something else. My priority was to successfully install Devuan. Phew! it took me weeks but eventually came right.

ghp · 2020-09-30 18:50:45

Just to mention I had the same problem with the beowulf amd64 netinstall iso.

kapqa · 2020-09-30 19:26:42

hello,

i installed several times in the last days devuan ascii and beowulf.

used "mkusb" to install to usb (look for "install to debian" version) and had no problems with netinstall64bit.
problem only with beowulf-live image 64 bit that gave errors during installation and after insatllation it booted the old linux (that was meant to be replaced) as if nothing was installed over it. i am not sure if i made an error but to me it seemed the live-installer was faulty.
maybe someone can confirm this?

The officially official Devuan Forum!

#1 2019-11-14 01:13:43

Problem with *.iso file

#2 2019-11-18 17:59:09

Re: Problem with *.iso file

#3 2019-11-18 19:34:38

Re: Problem with *.iso file

#4 2019-11-18 20:18:18

Re: Problem with *.iso file

#5 2019-11-18 21:29:06

Re: Problem with *.iso file

#6 2019-11-18 23:21:23

Re: Problem with *.iso file

#7 2019-11-18 23:33:12

Re: Problem with *.iso file

#8 2019-11-19 00:40:35

Re: Problem with *.iso file

#9 2019-11-19 00:51:45

Re: Problem with *.iso file

#10 2019-11-19 08:22:06

Re: Problem with *.iso file

#11 2019-11-19 10:14:57

Re: Problem with *.iso file

#12 2019-11-19 13:03:35

Re: Problem with *.iso file

#13 2019-11-19 15:44:32

Re: Problem with *.iso file

#14 2019-11-20 19:27:05

Re: Problem with *.iso file

#15 2019-11-20 20:00:50

Re: Problem with *.iso file

#16 2019-11-30 01:15:27

Re: Problem with *.iso file

#17 2019-11-30 15:04:04

Re: Problem with *.iso file

#18 2019-11-30 16:49:03

Re: Problem with *.iso file

#19 2019-11-30 21:17:33

Re: Problem with *.iso file

#20 2019-11-30 23:35:40

Re: Problem with *.iso file

#21 2020-02-20 13:53:20

Re: Problem with *.iso file

#22 2020-02-20 20:15:01

Re: Problem with *.iso file

#23 2020-03-05 21:02:15

Re: Problem with *.iso file

#24 2020-09-30 18:50:45

Re: Problem with *.iso file

#25 2020-09-30 19:26:42

Re: Problem with *.iso file

Board footer