[SOLVED] Backup entire installed Beowulf w/software + configs + /home?

dice · 2021-03-13 14:59:21

Hi fsmithred, does refractasnapshot depend on having "initrd (initial ram disk): generic (all available drivers)" as opposed to "only select drivers to suit this hardware", during expert install? I have a feeling this is where the initramfs is failing.

I must have confused machines as the machine i was using was done with expert install using only the option to install drivers suited that particular computer whereas i have no issues with another machine an asus laptop that were installed with all available drivers?

Ive even tested a qemu installation and did a refractasnapshot and dd'd that qemu created snapshot to a usb and it boots fine. So im fairly certain is a driver issue.

Last edited by dice (2021-03-13 15:01:31)

fsmithred · 2021-03-13 15:17:36

dice wrote:

Hi fsmithred, does refractasnapshot depend on having "initrd (initial ram disk): generic (all available drivers)" as opposed to "only select drivers to suit this hardware", during expert install? I have a feeling this is where the initramfs is failing.

Wow. Excellent question. I don't know the answer. I can't recall if this has ever come up in the 10 years I've been maintaining that script. I would expect a snapshot with limited drivers would only boot on the same or similar hardware.

That option is probably set in /etc/initramfs-tools/initramfs.conf. Compare the one with limited drivers to one with all available. If there's something obvious to change, change it, rebuild initramfs and make a new snapshot. update-initramfs -u

If you want to save time in the above procedure, make the new initrd.img and copy it to /home/work/iso/live and then run refractasnapshot, but choose the option to run xorriso only. You'll be done in one minute.

I don't think I've ever done an install with drivers limited to the current hardware. I always assume that my old boxen will die and I'll have to move the hard drive to another computer.

Edit: if you do the shortcut in this case, and you use that iso to install on another system, you will need to edit initramfs.conf in that system. That could be done in the live system before install. That way when the initrd gets rebuilt during the install it will be good.

Edit 2: change initramfs.conf in /home/work/myfs/ and run the re-squash option in refractasnapshot. That will avoid the problem mentioned in the first edit. It'll take longer, too.

dice · 2021-03-13 15:23:53

I will get back to you tomorrow on that, thanks.

ComputerBob · 2021-03-13 18:29:50

fsmithred wrote:

Perfect!!!
ComputerBob wrote:
(embarrassed) Well, I have no idea what I'm doing, but you're giving me great instructions on how to do it all anyway, so I'm happy to (try to) do it!
root@robinson:~# lsinitramfs /initrd.img | grep resume
conf/conf.d/resume
conf/conf.d/zz-resume-auto
scripts/local-premount/resume
usr/bin/resume
root@robinson:~# lsinitramfs /initrd.img | grep resume
Here's the fix.
1. Make a backup copy of /etc/initramfs-tools/conf.d/resume
If you don't ever hibernate your computer, this step is not really necessary.
2. As root, edit /etc/initramfs-tools/conf.d/resume so that it contains only the following:
RESUME=none
Save the file.
3. Run update-initramfs -u
4. Make a new snapshot.
5. (Optional) If you do use hibernate, you need to undo what you did. Copy the original resume file back to where it was and run 'update-initramfs -u' again. If you do that, you'll have to do this whole procedure every time you want to make a snapshot.

I just did steps 2 and 3:

root@robinson:~# update-initramfs -u
update-initramfs: Generating /boot/initrd.img-4.19.0-14-amd64
W: Possible missing firmware /lib/firmware/nvidia/gv100/acr/ucode_load.bin for module nouveau
live-boot: core filesystems devices utils udev blockdev dns.
root@robinson:~#

I hope that the results are what you hoped for/expected. My system uses nouveau, but Synaptic doesn't see any nouveau firmware, installed or not. Maybe it's actually nvidia firmware, that somehow works with nouveau?

I deleted the previous RefractaSnapshot iso and its SHAsum fle from /home/snapshot, to make more room on the / partition.

Now, I will try to make a new snapshot. (fingers crossed...)

fsmithred · 2021-03-13 19:11:01

Bob, that output looks normal. If you want to get rid of the message about missing firmware, you could install firmware-misc-nonfree if you have non-free repositories enabled. (non-free refers to speech, not beer)

The real test will be to see if the new snapshot boots.

ComputerBob · 2021-03-13 19:20:31

fsmithred wrote:

Bob, that output looks normal. If you want to get rid of the message about missing firmware, you could install firmware-misc-nonfree if you have non-free repositories enables. (non-free refers to speech, not beer)
The real test will to see if the new snapshot boots.

I just checked -- I DO already have firmware-misc-nonfree installed.

But, more importantly, the new snapshot DOES EXACTLY THE SAME THING as the old one!

It boots up with many, many commands flying by on the screen.
It gets to the line that says:

random: crng init done

and there it stops for a minute or two, then:

BOOT FAILED!

Unable to find a medium... (exactly as I described above, when I used the old snapshot.

fsmithred · 2021-03-13 19:36:51

Let's see what's in these two resume files. I don't understand why there are both of them. The first one is in older releases and then the name was changed to the second one in ascii or beowulf.

conf/conf.d/resume
conf/conf.d/zz-resume-auto

In order to get to those files, initrd.img needs to be extracted. Here are the commands to do that. Do this as unprivileged user in your home directory.

cp /initrd.img /home/computerbob/
mkdir extracted
cd extracted
(cpio -i ; zcat | cpio -i) < ../initrd.img

Then look in extracted/conf/conf.d/resume and zz-resume-auto
One or both will probably have the uuid of your swap partition. One or both might have RESUME=none. Or maybe something else I haven't thought of.

Don't delete the extracted stuff. It might be possible to fix and repack the initrd and plug it into the build process.

ComputerBob · 2021-03-13 20:18:40

fsmithred wrote:

Let's see what's in these two resume files. I don't understand why there are both of them. The first one is in older releases and then the name was changed to the second one in ascii or beowulf.
conf/conf.d/resume
conf/conf.d/zz-resume-auto
In order to get to those files, initrd.img needs to be extracted. Here are the commands to do that. Do this as unprivileged user in your home directory.
cp /initrd.img /home/computerbob/
mkdir extracted
cd extracted
(cpio -i ; zcat | cpio -i) < ../initrd.img
Then look in extracted/conf/conf.d/resume and zz-resume-auto
One or both will probably have the uuid of your swap partition. One or both might have RESUME=none. Or maybe something else I haven't thought of.
Don't delete the extracted stuff. It might be possible to fix and repack the initrd and plug it into the build process.

Thank you! I will try to do all of that now.

In the meantime, I wonder why I'm STILL seeing only symlinks in my SnapShot (when viewed from Thunar of my running HD installation). Is that normal. Could it be that the "BOOT FAILED" error message is telling the truth about not finding a bootable system, because all it sees are symlinks? Probably not, but I thought I'd ask, just in case.

Last edited by ComputerBob (2021-03-13 21:08:33)

fsmithred · 2021-03-13 20:28:29

I don't understand the symlinks at all. When you looked at the mounted usb with ls, it showed the right things.

You can mount the iso as root and look at it that way.

mount /home/snapshot/snapshot.iso /mnt
ls /mnt
ls /mnt/live
ls /mnt/isolinux
# and when you're done looking around in there
umount /mnt

You should see /live /isolinux /pkglist_whatever. Look inside /live and you should see initrd.img vmlinuz and filesystem.squashfs. Inside /isolinux should be live.cfg (the boot menu) and a bunch of other files that are the same ones in /usr/lib/refractasnapshot/iso/isolinux/.

Edit: you could do the same with the mounted usb and look inside the directories.

ls /media/computerbob/liveiso/live    # and so on.

ComputerBob · 2021-03-13 20:42:34

I'm still in the middle of following your latest instructions, but I'm writing to let you know that

conf/conf.d/resume said, "RESUME=none"

That's all it said. No UUIDs. The old snapshot had a RESUME=UUID= (long string) instead of "none".

and

conf/conf.d/zz-resume-auto was not found.

Now, I will try to do the third step, looking at the extracted initrd.img...

I don't know how to view it, I tried the command "nano initrd.img" but all I saw were garbage characters. I suspect that I was looking at the unextracted file, not the extracted one.

I JUST FOUND four initrd.img in various locations, three of them were dated today, but I couldn't see any readable characters in any of them.

Last edited by ComputerBob (2021-03-13 21:10:38)

ComputerBob · 2021-03-13 20:54:42

fsmithred wrote:

I don't understand the symlinks at all. When you looked at the mounted usb with ls, it showed the right things.
You can mount the iso as root and look at it that way.
mount /home/snapshot/snapshot.iso /mnt
ls /mnt
ls /mnt/live
ls /mnt/isolinux
# and when you're done looking around in there
umount /mnt
You should see /live /isolinux /pkglist_whatever. Look inside /live and you should see initrd.img vmlinuz and filesystem.squashfs. Inside /isolinux should be live.cfg (the boot menu) and a bunch of other files that are the same ones in /usr/lib/refractasnapshot/iso/isolinux/.
Edit: you could do the same with the mounted usb and look inside the directories.
ls /media/computerbob/liveiso/live    # and so on.

Yes, everything I saw was there and, and far as I could tell, it all looked like real files. I haven't checked the USB stick yet. I may or may be able to figure out how to do that. Is there ANY possibility that there are snapshot FILES on my HD, but they're somehow converted to symlinks on my USB stick when I'm copying them there? So my USB stick boots (linked to existing files on my HD), until it wants something that it can't find. I'm just shooting in the dark here, so just ignore me if my ideas are not helpful.

NEVER MIND - I figured out how to mount my USB stick and view its various contents, using terminal commands. Everything I've seen there looks like REAL files, not symlinks. Man, this stuff is a real challenge to me!

Last edited by ComputerBob (2021-03-13 21:22:11)

ComputerBob · 2021-03-13 21:15:54

My wife and I have to leave to go to a lengthy appointment in 20 minutes. Before I leave, or after I get back, I may or may not have time to do any more troubleshooting (or dumb typing, depending on how you see it) today.

fsmithred · 2021-03-15 02:01:33

@ComputerBob:
When you return to this, here's a simple check to see if your problem is the same as dice's. Adjust the file name if you have a different kernel.

ls -lh /boot/initrd.img-4.19.0-14-amd64

Do you recall if you did an expert install and chose to only have drivers targeted for your hardware in the initramfs? If you don't recall, that's fine. The output of the command above will tell us. It'll either be big or small. (I want the number, please.)

ComputerBob · 2021-03-15 04:12:24

fsmithred wrote:

@ComputerBob:
When you return to this, here's a simple check to see if your problem is the same as dice's. Adjust the file name if you have a different kernel.
ls -lh /boot/initrd.img-4.19.0-14-amd64
Do you recall if you did an expert install and chose to only have drivers targeted for your hardware in the initramfs? If you don't recall, that's fine. The output of the command above will tell us. It'll either be big or small. (I want the number, please.)

Even before I run that command, in Thunar, I can see one /grub/ folder and eight files inside the /boot/ folder, but I'm guessing that the ls command itself might show you those anyway:

computerbob@robinson:~$ ls -lh /boot/initrd.img-4.19.0-14-amd64
-rw-r--r-- 1 root root 24M Mar 13 13:12 /boot/initrd.img-4.19.0-14-amd64
computerbob@robinson:~$

Since I really don't know what I'm doing, here's a different version of the same ls command, to show the entire contents of that folder:

computerbob@robinson:~$ ls -lh /boot
total 44M
-rw-r--r-- 1 root root 202K Jan 30 04:35 config-4.19.0-14-amd64
-rw-r--r-- 1 root root 183K Dec 17 07:51 config-4.9.0-14-amd64
drwxr-xr-x 5 root root 4.0K Mar 10 10:30 grub
-rw-r--r-- 1 root root  24M Mar 13 13:12 initrd.img-4.19.0-14-amd64
-rw-r--r-- 1 root root 4.7M Dec 18 13:22 initrd.img-4.9.0-14-amd64
-rw-r--r-- 1 root root 3.3M Jan 30 04:35 System.map-4.19.0-14-amd64
-rw-r--r-- 1 root root 3.1M Dec 17 07:51 System.map-4.9.0-14-amd64
-rw-r--r-- 1 root root 5.1M Jan 30 04:35 vmlinuz-4.19.0-14-amd64
-rw-r--r-- 1 root root 4.1M Dec 17 07:51 vmlinuz-4.9.0-14-amd64
computerbob@robinson:~$

For the first time in many years, I used the first (simpler) install option, instead of the more-detailed expert install option, installing Beowulf onto partitions that I had already created in GParted. I don't remember if that gave me the option to install only specific drivers or all drivers. However, if it DID give me that option, I know that -- in the past - - I've always chosen to install ALL drivers.

Last edited by ComputerBob (2021-03-15 04:17:56)

fsmithred · 2021-03-15 13:22:54

24M is big, and you confirmed that you didn't select for targeted drivers only. So that means your problem is not the same as dice's.

dice · 2021-03-15 14:05:38

I suppose initrd.img will vary from system to system but my initrd.img is 30M, the system that had limited drivers was around 19M.

Is the file /etc/initramfs-tools/conf.d/driver-policy present ?

What does Computer Bobs /etc/initramfs-tools/initramfs.conf look like ?

ComputerBob · 2021-03-15 14:15:03

Just in case this information is helpful, too:

computerbob@robinson:~$ uname -a
Linux robinson 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64 GNU/Linux
computerbob@robinson:~$

ComputerBob · 2021-03-15 14:20:17

dice wrote:

I suppose initrd.img will vary from system to system but my initrd.img is 30M, the system that had limited drivers was around 19M.
Is the file /etc/initramfs-tools/conf.d/driver-policy present ?

Yes, its contents are as follows:

# Driver inclusion policy selected during installation
# Note: this setting overrides the value set in the file
# /etc/initramfs-tools/initramfs.conf
MODULES=dep

dice wrote:

What does Computer Bobs /etc/initramfs-tools/initramfs.conf look like ?

#
# initramfs.conf
# Configuration file for mkinitramfs(8). See initramfs.conf(5).
#
# Note that configuration options from this file can be overridden
# by config files in the /etc/initramfs-tools/conf.d directory.

#
# MODULES: [ most | netboot | dep | list ]
#
# most - Add most filesystem and all harddrive drivers.
#
# dep - Try and guess which modules to load.
#
# netboot - Add the base modules, network modules, but skip block devices.
#
# list - Only include modules from the 'additional modules' list
#

MODULES=most

#
# BUSYBOX: [ y | n | auto ]
#
# Use busybox shell and utilities.  If set to n, klibc utilities will be used.
# If set to auto (or unset), busybox will be used if installed and klibc will
# be used otherwise.
#

BUSYBOX=auto

#
# KEYMAP: [ y | n ]
#
# Load a keymap during the initramfs stage.
#

KEYMAP=n

#
# COMPRESS: [ gzip | bzip2 | lz4 | lzma | lzop | xz ]
#

COMPRESS=gzip

#
# NFS Section of the config.
#

#
# DEVICE: ...
#
# Specify a specific network interface, like eth0
# Overridden by optional ip= or BOOTIF= bootarg
#

DEVICE=

#
# NFSROOT: [ auto | HOST:MOUNT ]
#

NFSROOT=auto

#
# RUNSIZE: ...
#
# The size of the /run tmpfs mount point, like 256M or 10%
# Overridden by optional initramfs.runsize= bootarg
#

RUNSIZE=10%

Last edited by ComputerBob (2021-03-15 14:23:24)

dice · 2021-03-15 14:51:32

Ok so you need to remove /etc/initramfs-tools/conf.d/driver-policy "driver-policy" file.

Then

# update-initramfs -u

Then make a new snapshot.

fsmithred · 2021-03-15 15:11:26

Thanks for persevering, dice. I gave up too soon. I did an install with targeted drivers and the initrd was only 8M, but maybe because the hardware was virtual. Let's see how this turns out, but I think you may have solved it.

dice · 2021-03-15 15:19:21

fsmithred wrote:

Thanks for persevering, dice. I gave up too soon. I did an install with targeted drivers and the initrd was only 8M, but maybe because the hardware was virtual. Let's see how this turns out, but I think you may have solved it.

All good, without your knowledge we would have got nowhere. To me this is a good learning exercise and ive gained some knowledge on the files that control the initrd/initramfs.

Its possible that update and upgrade from ascii to beowulf might leave out some needed drivers as well, just a hunch here but there might be something to it.

ComputerBob · 2021-03-15 20:28:48

dice wrote:

Ok so you need to remove /etc/initramfs-tools/conf.d/driver-policy "driver-policy" file.
Then
# update-initramfs -u
Then make a new snapshot.

I had hoped to be writing this from my system, after booting from my USB stick.

I was pretty hopeful, because the whole iso creation and copying it to my USB stick seemed noticeably faster than it had been previously.

But I still I got the exact same frozen-after-the "random:crng init done" line, followed by a long pause, followed by the whole "BOOT FAILED" message -- which still complains that it can't find a bootable system...

I'm really sorry guys. With it working for other people now, it's starting to clearly look and feel like I must be the problem.

computerbob@robinson:~$ ls -lh /boot
total 57M
-rw-r--r-- 1 root root 202K Jan 30 04:35 config-4.19.0-14-amd64
-rw-r--r-- 1 root root 183K Dec 17 07:51 config-4.9.0-14-amd64
drwxr-xr-x 5 root root 4.0K Mar 10 10:30 grub
-rw-r--r-- 1 root root  37M Mar 15 15:58 initrd.img-4.19.0-14-amd64
-rw-r--r-- 1 root root 4.7M Dec 18 13:22 initrd.img-4.9.0-14-amd64
-rw-r--r-- 1 root root 3.3M Jan 30 04:35 System.map-4.19.0-14-amd64
-rw-r--r-- 1 root root 3.1M Dec 17 07:51 System.map-4.9.0-14-amd64
-rw-r--r-- 1 root root 5.1M Jan 30 04:35 vmlinuz-4.19.0-14-amd64
-rw-r--r-- 1 root root 4.1M Dec 17 07:51 vmlinuz-4.9.0-14-amd64

Last edited by ComputerBob (2021-03-15 20:39:51)

ralph.ronnquist · 2021-03-15 20:48:47

Would you mind reporting what ls /dev/ shows at the (initramfs) prompt.

ComputerBob · 2021-03-15 21:01:51

I don't know of any way to copy/paste it all here, but running

ls /dev/

at the (initramfs) prompt showed me 129 files, with several different names.

ralph.ronnquist · 2021-03-15 21:09:42

Ok, let's start with those starting with "s" other than "snapshot", "stderr", "stdin" and "stdout" (irrelevant if they are there or not).

The officially official Devuan Forum!

#51 2021-03-13 14:59:21

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#52 2021-03-13 15:17:36

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#53 2021-03-13 15:23:53

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#54 2021-03-13 18:29:50

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#55 2021-03-13 19:11:01

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#56 2021-03-13 19:20:31

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#57 2021-03-13 19:36:51

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#58 2021-03-13 20:18:40

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#59 2021-03-13 20:28:29

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#60 2021-03-13 20:42:34

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#61 2021-03-13 20:54:42

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#62 2021-03-13 21:15:54

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#63 2021-03-15 02:01:33

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#64 2021-03-15 04:12:24

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#65 2021-03-15 13:22:54

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#66 2021-03-15 14:05:38

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#67 2021-03-15 14:15:03

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#68 2021-03-15 14:20:17

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#69 2021-03-15 14:51:32

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#70 2021-03-15 15:11:26

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#71 2021-03-15 15:19:21

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#72 2021-03-15 20:28:48

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#73 2021-03-15 20:48:47

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#74 2021-03-15 21:01:51

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#75 2021-03-15 21:09:42

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

Board footer