[SOLVED] Backup entire installed Beowulf w/software + configs + /home?

ComputerBob · 2021-03-15 21:16:09

Are you wanting me to go back to the "crashed" (initramfs) prompt and copy down the names of the files, starting with the ones that start with "s"?

I'm about to leave to go do some ministry, but I would be able to do it when I return.

ralph.ronnquist · 2021-03-15 21:32:06

No worries. If you have a camera nearby you could take a picture and then refer to that...

Only those starting with "s" to begin with.

ComputerBob · 2021-03-15 23:12:34

@ralph,
I just took your advice, but I don't know any way to attach my 4 photos to this forum message or to the private forum message that I sent you, and I don't have any photo-displaying service to put them on. ;(

Last edited by ComputerBob (2021-03-15 23:14:16)

ralph.ronnquist · 2021-03-15 23:18:57

Well, there should be less than 20 s* entries in /dev so a little bit of eye-to-hand copying would work for me

ComputerBob · 2021-03-15 23:50:17

They're not especially sharp photos -- the post-its on my monitor are in sharper forcus that the screen itself -- but, as best as I can make out, here are the things that start with "s." It might take me awhile, because I think they are displayed horizontally across 3 of the 4 photos...

sda
sda1
sda2
sda3
sda4
sdb
sdb1

stdin?
sg0?
sg1?
sg2?

tty0
tty1
tty10
tty11
tty12
tty13
tty14
tty15
tty16
tty17
tty18
tty19
tty2
tty20 through tty29
tty3
tty 30 through 39
tty4
tty 40 through 49
urandom
ucs?
ucs1?
ucsa?
ucsa1?
vcsa?
vcsa1?
vcsu?
vcsu1?
vga_arbiter
zero
ztdin?
ztdout?

This was one of the most time-consuming, hardest things I've tried to do in a long, long time -- remembering one number at a time, going back and forth from the forum to blurry photos in GIMP. Whew! I sure hope I could see enough info to help you!

ralph.ronnquist · 2021-03-16 00:20:33

That's good information. Only the s* were interesting and in particular the sda* and sdb* entries which are adapters for SATA devices and their partitions. Apparently there is one "drive", sda, with 4 partitions and another, sdb, with 1 partition.

This kind of presents the same hardware picture as your post #34 execpt that it has sda4 instead of sda6.

That difference is probably the cause of your problem... and now we need @fsmithred to draw conclusions

fsmithred · 2021-03-16 01:07:35

ralph.ronnquist wrote:

This kind of presents the same hardware picture as your post #34 execpt that it has sda4 instead of sda6.
That difference is probably the cause of your problem... and now we need @fsmithred to draw conclusions

I don't see why that would matter. It should be booting the live system, not a disk partition. But it does lead me to a question. What's in the boot command?

Bob, at the boot menu for the live-usb, press TAB and see what it shows at the bottom of the screen. It should be

/live/vmlinuz initrd=/live/initrd.img boot=live username=computerbob

I can boot to initramfs prompt if I remove 'boot=live' from the boot command. I'm wondering if your boot menu got screwed up and lost that piece.

ComputerBob · 2021-03-16 02:18:37

ralph.ronnquist wrote:

That's good information. Only the s* were interesting and in particular the sda* and sdb* entries which are adapters for SATA devices and their partitions. Apparently there is one "drive", sda, with 4 partitions and another, sdb, with 1 partition.
This kind of presents the same hardware picture as your post #34 execpt that it has sda4 instead of sda6.
That difference is probably the cause of your problem... and now we need @fsmithred to draw conclusions

You are correct about sda and sdb. The sda drive with 4 partitions is my HD, my sdb USB stick was totally unallocated space until I copied the Snapshot to it -- I didn't intentionally create any partitions on it - maybe that was created by the copying process?

GParted currently shows me the following on my sda:

/dev/sda1 /
/dev/sda2 /home
/dev/sda3 /linux-swap
unallocated 9.69 GB
/dev/sda6 /media/data (102.54 GB of user data)

ComputerBob · 2021-03-16 02:23:30

fsmithred wrote:

ralph.ronnquist wrote:
This kind of presents the same hardware picture as your post #34 execpt that it has sda4 instead of sda6.
That difference is probably the cause of your problem... and now we need @fsmithred to draw conclusions
I don't see why that would matter. It should be booting the live system, not a disk partition. But it does lead me to a question. What's in the boot command?
Bob, at the boot menu for the live-usb, press TAB and see what it shows at the bottom of the screen. It should be
/live/vmlinuz initrd=/live/initrd.img boot=live username=computerbob
I can boot to initramfs prompt if I remove 'boot=live' from the boot command. I'm wondering if your boot menu got screwed up and lost that piece.

I'll try to do that now, and then report back in this same message. Please hold.... (Muzak: la, la, la, dum-de-dum-dum... Please continue to hold. Your call is very important to us. Estimated wait time is approximately 38 hours...)

I will try to copy what I see in the blurry photo that I just took after pressing TAB at the boot menu:

/live/vmlinuz   initrd=/live/initrd.img boot=live   username=computerbob   _

I suspect that that may be what you said it might be, but I'm too tired tonight (it's been a long, long but rewarding day) to compare the two lines myselt.

Last edited by ComputerBob (2021-03-16 02:39:48)

dice · 2021-03-16 14:14:12

ComputerBob wrote:

dice wrote:
Ok so you need to remove /etc/initramfs-tools/conf.d/driver-policy "driver-policy" file.
Then
# update-initramfs -u
Then make a new snapshot.
I had hoped to be writing this from my system, after booting from my USB stick.
I was pretty hopeful, because the whole iso creation and copying it to my USB stick seemed noticeably faster than it had been previously.
But I still I got the exact same frozen-after-the "random:crng init done" line, followed by a long pause, followed by the whole "BOOT FAILED" message -- which still complains that it can't find a bootable system...
I'm really sorry guys. With it working for other people now, it's starting to clearly look and feel like I must be the problem.
computerbob@robinson:~$ ls -lh /boot
total 57M
-rw-r--r-- 1 root root 202K Jan 30 04:35 config-4.19.0-14-amd64
-rw-r--r-- 1 root root 183K Dec 17 07:51 config-4.9.0-14-amd64
drwxr-xr-x 5 root root 4.0K Mar 10 10:30 grub
-rw-r--r-- 1 root root  37M Mar 15 15:58 initrd.img-4.19.0-14-amd64
-rw-r--r-- 1 root root 4.7M Dec 18 13:22 initrd.img-4.9.0-14-amd64
-rw-r--r-- 1 root root 3.3M Jan 30 04:35 System.map-4.19.0-14-amd64
-rw-r--r-- 1 root root 3.1M Dec 17 07:51 System.map-4.9.0-14-amd64
-rw-r--r-- 1 root root 5.1M Jan 30 04:35 vmlinuz-4.19.0-14-amd64
-rw-r--r-- 1 root root 4.1M Dec 17 07:51 vmlinuz-4.9.0-14-amd64

Can you please do below command and let us now the output.

uname -a

I have a feeling the ascii kernel is the issue here and still possibly in use instead of the updated kernel. If the snapshot is picking that up the initrd.img is way to small. 4.7M Dec 18 13:22 initrd.img-4.9.0-14-amd64

You need to ls -lh the directory of the liveiso when mounted, for me this directory was

ls -lh /media/liveiso/live

Or it could be with user name...

ls -lh /media/dice/liveiso/live

Last edited by dice (2021-03-16 14:22:18)

ComputerBob · 2021-03-16 16:33:26

computerbob@robinson:~$ uname -a
Linux robinson 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64 GNU/Linux
computerbob@robinson:~$

computerbob@robinson:/media/computerbob/liveiso$ ls -lh /media/computerbob/liveiso/live
total 42M
-r--r--r-- 1 computerbob computerbob  37M Mar 15 16:04 initrd.img
-r--r--r-- 1 computerbob computerbob 179K Dec 12  2019 memtest
-r--r--r-- 1 computerbob computerbob 5.1M Mar 15 16:01 vmlinuz
computerbob@robinson:/media/computerbob/liveiso$

ComputerBob · 2021-03-16 16:37:39

I lost track of what we're doing a few days ago. Right now, I'm just a monkey with a keyboard.
I'm sure glad that you guys know what you're doing.

Last edited by ComputerBob (2021-03-16 17:19:16)

dice · 2021-03-17 02:26:58

ComputerBob wrote:

computerbob@robinson:~$ uname -a
Linux robinson 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64 GNU/Linux
computerbob@robinson:~$

computerbob@robinson:/media/computerbob/liveiso$ ls -lh /media/computerbob/liveiso/live
total 42M
-r--r--r-- 1 computerbob computerbob  37M Mar 15 16:04 initrd.img
-r--r--r-- 1 computerbob computerbob 179K Dec 12  2019 memtest
-r--r--r-- 1 computerbob computerbob 5.1M Mar 15 16:01 vmlinuz
computerbob@robinson:/media/computerbob/liveiso$

You dont seem to have a filesystem.squashfs ?? Perhaps you havent created the snapshot correctly..

This is what my snapshot looks like...

/media/dice/liveiso/live $ ls -lh 
total 765M
-r--r--r-- 1 dice dice 731M Mar 14 17:18 filesystem.squashfs
-r--r--r-- 1 dice dice  29M Mar 14 17:16 initrd.img
-r--r--r-- 1 dice dice 179K Dec 13  2019 memtest
-r--r--r-- 1 dice dice 5.1M Mar 14 17:15 vmlinuz

also do you need the older ascii kernel, if not maybe uninstall that and install another fallback kernel that is a bit newer from beowulf. Like maybe the linux-image-4.19.0-13-amd64, then reboot, and do another snapshot and see if this make a difference.

Last edited by dice (2021-03-17 02:33:54)

ComputerBob · 2021-03-17 02:56:17

I'll replace the older kernel with a newer fallback one, as you suggested.

"Perhaps you haven't created the snapshot correctly." IIRC, alll I did was click on the the default option to create the snapshot, and then it did everything else itself. Did I miss something? Do I need to run something else in addition to what I ran, to create the snapshots?

OK, I just removed/installed the kernels, exactly as you suggested. In Synaptic, that caused the following two packages to appear as "Installed (auto removable)":
irqbalance
runit helper

I have not removed those two packages (yet?), just in case you wanted me to NOT do that.

I will reboot and continue on to creating a new snapshot (hopefully, tomorrow). An entire day today, of trying to help in a domestic violence situation, and dealing with someone who just had a stroke, really tired me out... but I'll still be here again, as soon as I can.

Now, it's a few minutes later. I just rebooted. A quick Catfish File Search of the whole File System found 14 items with "squashfs" in their names. Five of them were folders, and 9 of them were files. (???)

Good night! And Thanks.

Last edited by ComputerBob (2021-03-17 03:09:25)

dice · 2021-03-17 12:13:52

Okay maybe delete the snapshot and start again.

so delete the snapshot folder in /home/snapshot.

run refractasnapshot again, creating a new snapshot i think is option 1 with uefi disabled.

I dont know anything about the irqbalance and runit-helper packages so just leave them alone for now.

Doing a catfish file search of the whole file system for squashfs is not needed and just wasting your time, all we want to know is that when you create the snapshot and mount it in your file manager you can see the filesystem.squashfs similar to below code snippet. The filesystem.squashfs was missing in your code snippet so all you seem to have had was a usb stick with only 42 megabytes of files in the form of initrd.img, memtest and vmlinuz.This will definitely not boot into a live session as you dont have a filesystem added to boot it.

/media/dice/liveiso/live $ ls -lh 
total 765M
-r--r--r-- 1 dice dice 731M Mar 14 17:18 filesystem.squashfs
-r--r--r-- 1 dice dice  29M Mar 14 17:16 initrd.img
-r--r--r-- 1 dice dice 179K Dec 13  2019 memtest
-r--r--r-- 1 dice dice 5.1M Mar 14 17:15 vmlinuz

fsmithred · 2021-03-17 12:45:43

One thing that might help is to edit /etc/refractasnapshot.conf to set save_work="yes". Normally, the copy of the filesystem gets deleted after the snapshot.iso is made. If the work dir is not deleted, we can do some forensic inspection and some other tricks, maybe.

Also...
I did not see any indication of a common problem in the log file you sent me, but looking again at your df output has me wondering if you're running out of space. You posted:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1       8.6G  4.0G  4.2G  49% /
/dev/sda2       8.5G  3.0G  5.0G  38% /home
/dev/sda6       247G   98G  138G  42% /media/data

I see 4G of operating system plus 3G of home used. That's a total of 7G unless you added a bunch of stuff to the excludes file. I'm guessing you did not do that. Refractasnapshot makes a copy of the entire system (minus what's listed in the excludes file) and puts that copy in /home/snapshot. But you only have 5G free space in /home.

You can switch the location of work_dir and snapshot_dir in the config file. Change this:

snapshot_dir="/home/snapshot"
work_dir="/home/work"
efi_work="${work_dir}/efi-files"

To this:

snapshot_dir="/media/data/snapshot"
work_dir="/media/data/work"
efi_work="${work_dir}/efi-files"

Then run refractasnapshot.

dice · 2021-03-17 12:54:55

^ definitely looks like a size issue, i wonder how large the snapshot would be if enough space were available?

ComputerBob · 2021-03-17 13:22:07

Messages received, guys! And some of them actually UNDERSTOOD.

I have printed them out, and will start to work on them, as a to-do list.

fsmithred · 2021-03-17 13:28:48

dice wrote:

^ definitely looks like a size issue, i wonder how large the snapshot would be if enough space were available?

I'm not sure how big it will be. It's possible to make the iso smaller by uncommenting one of the mksq_opt lines in the config file.

# Uncomment one of the lines below to use xz compression for smaller iso. 

# small and slow
#mksq_opt="-comp xz"

# smaller and slower:
#mksq_opt="-comp xz -Xbcj x86"

ComputerBob · 2021-03-17 14:21:08

It is SO CLOSE!!!!

It was a scary process. I remember that it seemed to take a lot longer than in the past, to run the snapshot-creation program.

The "squashing" section itself probably took ten minutes, while the little pop-up gui feedback window never showed anything except that it reported 0% complete the whole time.

I think it may have been at the same time that the command line showed a progress bar that stayed at 0% for a long time, then jumped to 20,xxx for a long time, then 40,xxx for a long time, then completed without any further updates.

Then, I think the gui window disappeared for awhile, before returning to tell me that it was all done!

There was no longer any /etc/home/ folder. I found the iso and its SHA file in /media/data/snapshot/

Copying the iso to sdb took much longer than it had before. I considered that to be a GOOD sign, except that the only way I could tell that it was doing anything was by the quickly-flashing (not slowly-flashing) light on the USB stick.

Then, my USB stick BOOTED UP to my complete, auto-logged-in desktop (the autolog-in is intentional). But for only a few seconds.

Then it dropped back to a CLI screen with messages at the bottom, for only about one second, before returning to my desktop, with a login screen, autofilled with my username, for about 3 seconds.

Then back to the CLI screen for about one second, back to the desktop for about 3 seconds. After the 5th of 6th infinite circle or activities, I physically shut down my computer, and booted back to the HD again. That's how I'm typing this right now.

MAN, THIS IS SO COOL! IT IS SO CLOSE TO WORKING!!!!!!!!!

Last edited by ComputerBob (2021-03-17 14:25:24)

fsmithred · 2021-03-17 14:51:48

Cool! Please tell us a little about what desktop environment and display (login) manager is installed. And if you have autologin set for your installed system, please describe how you did that. The live-config scripts set up autologin for the live environment. Maybe two settings are competing.

Optional: At the isolinux boot menu, press TAB to edit the boot line and add the one word, noautologin. And see if it boots to a login screen. Or maybe it'll boot to desktop.

ComputerBob · 2021-03-17 18:31:01

YES! I added noautologin to the end of the boot line, and then rebooted!
SUCCESS - except that my data partition didn't get mounted
so that meant that I had no email or browser, UNTIL I eventually figured out how to mount the data partition (/dev/sda6) as /media/data, because that's where my email and browser profiles are stored.
But I got it figured out, as proven by the fact that I'm writing this very first forum response
FROM MY BEOWULF SYSTEM, FULLY CONFIGURED, RUNNING ON MY USB STICK!

I don't know if it's a real word or not, but you have all been patient, knowlegable, understanding, genii (the plural of "genius," in my mind) throughout this entire, lengthy, time-consuming thread.

It has been a REAL HONOR to work with you. Thank you very, very much. You have no idea how much good it did me to help in my own small way. And I hope that none of you ever have a stroke, and learn, first-hand, how much you've all done for me.

But, please know, in your hearts, from my words, that you've done a tremendous thing for me.

I think there's supposed to be a way to use my WORKING USB stick to INSTALL my system onto a hard drive, but I'm not even going to worry about that right now.

THANK YOU, THANK YOU, THANK YOU!

If you have any unanswered questions, or things for me to test, I'll be happy to keep replying!

fsmithred · 2021-03-17 19:24:11

Thanks for playing until the end. I like my stuff to work, and I'm really glad you stuck around until we all solved it. I might have to adjust the script so that mksquashfs is noisier about failing.

When you're in a mood to read some documentation, take a look here:
https://refracta.org/documents.html

ComputerBob · 2021-03-17 20:35:35

Thank you, again! And thank you for the link -- I look forward to learning a lot more about RefractaSnapshot, once enough of my brain cells have recovered!

ComputerBob · 2021-03-18 17:29:07

IIRC, the whole "squashing" process took several minutes, with virtually no feedback, because the gui feedback progress, back-and-forth bar slowed way, way, down, then stopped altogether, even though (I figured out later) it was actually still "silently" working for several minutes -- while the command-line progress bar only updated two or three times during the whole process. But I'm really glad that they worked, even though, at the time, I suspected that they had locked up.

Last edited by ComputerBob (2021-03-18 18:02:48)

The officially official Devuan Forum!

#76 2021-03-15 21:16:09

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#77 2021-03-15 21:32:06

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#78 2021-03-15 23:12:34

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#79 2021-03-15 23:18:57

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#80 2021-03-15 23:50:17

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#81 2021-03-16 00:20:33

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#82 2021-03-16 01:07:35

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#83 2021-03-16 02:18:37

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#84 2021-03-16 02:23:30

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#85 2021-03-16 14:14:12

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#86 2021-03-16 16:33:26

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#87 2021-03-16 16:37:39

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#88 2021-03-17 02:26:58

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#89 2021-03-17 02:56:17

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#90 2021-03-17 12:13:52

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#91 2021-03-17 12:45:43

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#92 2021-03-17 12:54:55

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#93 2021-03-17 13:22:07

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#94 2021-03-17 13:28:48

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#95 2021-03-17 14:21:08

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#96 2021-03-17 14:51:48

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#97 2021-03-17 18:31:01

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#98 2021-03-17 19:24:11

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#99 2021-03-17 20:35:35

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

#100 2021-03-18 17:29:07

Re: [SOLVED] Backup entire installed Beowulf w/software + configs + /home?

Board footer