[Closed] Problems with Beowulf as second OS on a SSD

rolfie · 2020-04-23 19:31:23

I am trying to add Beowulf as second OS past an Arch installation on the SSD of a brand new PC. I would like to share my experience and need some advice how to deal with the remaining problems.

Basics: HW is a Ryzen 3700x on ASUS X570-Pro mainboard, latest bios. UEFI system, Arch installed as first OS on the SSD (also encrypted with LWM), enough space on the EFI partition, labelled as ESP. CSM off, secure boot off, all keys deleted, Uefi driver first.

The Beowulf beta desktop iso was written to a USB stick with rufus/Windows 7. Got a small 2G boot partition and a large 256G partition for an encrypted LVM setup by gparted past the Arch stuff. The stick boots fine in EFI mode, I used the expert installer and manual partitioning. First installation attempt failed with grub not installing. On the second attempt I paid attention to have a network connection to a mirror. This time the grub installation worked. This is an important topic.

Beowulf is now booting, I have got Lightdm/Mate desktop running. With the patch applied to /lib/cryptdisk.functions the shutdown is fast.

First effect: in the efibootmgr is no more indication of the Arch boot entries. It looks now like this:

# efibootmgr -v
BootCurrent: 0008
Timeout: 1 seconds
BootOrder: 0008,0009,000A,000C,000D
Boot0000* GRUB	HD(1,GPT,44444444-5555-4444-8888-999999999999,0x800,0x400000)/File(\EFI\GRUB\GRUBX64.EFI)
Boot0001* debian	HD(1,GPT,44444444-5555-4444-8888-999999999999,0x800,0x400000)/File(\EFI\DEBIAN\SHIMX64.EFI)
Boot0004* Hard Drive	BBS(HD,,0x0)..GO..NO........o.G.I.G.A.B.Y.T.E. .G.P.-.A.S.M.2.N.E.6.1.0.0.T.T.T.D....................A..........................dy.%.w  .....2..Gd-.;.A..MQ..L.S.N.1.9.3.5.0.8.9.0.0.3.8.5........BO
Boot0005* CD/DVD Drive	BBS(CDROM,,0x0)..GO..NO..........A.S.U.S. . . . .B.W.-.1.6.D.1.H.T....................A.............................................>..Gd-.;.A..MQ..L.L.K.J.P.E.C.3.B.1.4. .3. . . . . . . . ........BO
Boot0008* debian	HD(1,GPT,44444444-5555-4444-8888-999999999999,0x800,0x400000)/File(\EFI\DEBIAN\GRUBX64.EFI)..BO
Boot0009* Hard Drive	BBS(HD,,0x0)..GO..NO........o.G.I.G.A.B.Y.T.E. .G.P.-.A.S.M.2.N.E.6.1.0.0.T.T.T.D....................A..........................dy.%.w  .....2..Gd-.;.A..MQ..L.S.N.1.9.3.5.0.8.9.0.0.3.8.5........BO..NO..........C.T.5.0.0.M.X.5.0.0.S.S.D.1....................A.............................................>..Gd-.;.A..MQ..L.9.1.5.0.1.E.6.E.8.8.7.A. . . . . . . . ........BO
Boot000A* CD/DVD Drive	BBS(CDROM,,0x0)..GO..NO..........A.S.U.S. . . . .B.W.-.1.6.D.1.H.T....................A.............................................>..Gd-.;.A..MQ..L.L.K.J.P.E.C.3.B.1.4. .3. . . . . . . . ........BO
Boot000C* UEFI: SanDisk, Partition 1	PciRoot(0x0)/Pci(0x8,0x1)/Pci(0x0,0x3)/USB(1,0)/HD(1,MBR,0x97df71f4,0x800,0x1ca3800)..BO
Boot000D* USB	BBS(HD,,0x0)..GO..NO........q.S.a.n.D.i.s.k....................A.............................>..Gd-.;.A..MQ..L.4.C.5.3.0.0.0.0.0.4.0.3.2.8.1.1.3.2.3.4........BO

I did manually modify the boot number from 1 to 8. No change. Also removed grub-efi-amd64-signed, did not help. Why is Arch overwritten? I though that two Linux OS could reside on the same drive with one efi partition without conflict.

Second side effect: When booting before loading the grub screen I get this error. I am requested to press a key, or if I wait long enough the grub screen pops up. The output actually is in German, I translated it.

Error: no such device: 76c68ea3-5785-49b4-b93e-66f4de5b3a2b
Error: no server specified

This uuid is the one of the encrypted LVM partition for Beowulf /dev/nvme0n1p4. What is going on here?
[Edit: Thats wrong, its the uuid of the root partition inside the LVM! Edit]

Third side effect: after upgrading to kernel 5.4-bpo.4 kernel panic, due to EFI_MEMMAP is not enabled.

Thanks, rolfie

Edit: some clarification, more info added.

Last edited by rolfie (2020-04-27 15:45:00)

Head_on_a_Stick · 2020-04-23 23:23:08

rolfie wrote:

Ryzen 3700x

You will need the backported kernel for that, 3rd generation Ryzen is poorly supported by kernel 4.19.

What is your graphics hardware? The Devuan firmware-amd-graphics packages don't have the Navi firmware yet.

rolfie wrote:

First installation attempt failed with grub not installing.

Did you check TTY4 or the installer log for the actual error message? It would be nice to know why it failed.

rolfie wrote:

I did manually modify the boot number from 1 to 8. No change.

The only difference between those two entries is Secure Boot support, which is provided by shimx64.efi.

rolfie wrote:

Why is Arch overwritten?

Because your UEFI firmware is a useless, buggy pile of crap. Most are. I can show you how to generate a new NVRAM entry for Arch but your motherboard would probably just forget it again.

rolfie wrote:

I though that two Linux OS could reside on the same drive with one efi partition without conflict.

There is no conflict, as such, the Arch entry has just been lost.

EDIT: actually I think the Arch entry might be Boot0000. What was the exact grub-install command you ran when installing Arch? /EDIT

rolfie wrote:

This uuid is the one of the encrypted LVM partition for Beowulf /dev/nvme0n1p4. What is going on here?

No idea about LUKS, sorry.

rolfie wrote:

after upgrading to kernel 5.4-bpo.4 kernel panic, due to EFI_MEMMAP is not enabled.

As I mentioned earlier you will need that kernel version.

Have you tried the add_efi_memmap kernel parameter?

Last edited by Head_on_a_Stick (2020-04-23 23:51:18)

xinomilo · 2020-04-24 12:18:00

after modifying disk with gparted to prepare 2nd install, the disk uuid probably got modified. check from working system here : /dev/disk/by-uuid,
if it's not the same, chroot to troubling installation, modify grub and run update-grub.. then try booting again.

note, i dont use UEFI. and grub loads luks partition from eg. 'root=/dev/mapper/system--vg-root'

rolfie · 2020-04-24 16:49:06

Head_on_a_Stick wrote:

rolfie wrote:
after upgrading to kernel 5.4-bpo.4 kernel panic, due to EFI_MEMMAP is not enabled.
As I mentioned earlier you will need that kernel version.
Have you tried the add_efi_memmap kernel parameter?

Oh man, found the issue. Its not the parameter, adding that to grub did not work. When I tried to downgrade to the bpo.3 kernel and recalled the command from the bash_history to simply replace 4 by 3, I saw this:

# apt install linux-image-5.4.0-0.bpo.4-686 linux-headers-5.4.0-0.bpo.4-686

Don't know how that happened, copy/paste error from apt list output or if it was too late yesterday... It works fine with -amd64.

Other stuff later...

rolfie

Last edited by rolfie (2020-04-24 16:51:13)

rolfie · 2020-04-24 17:41:05

Head_on_a_Stick wrote:

rolfie wrote:
Ryzen 3700x
You will need the backported kernel for that, 3rd generation Ryzen is poorly supported by kernel 4.19.

Mainboard and Ryzen 7 3700X CPU work fine with 4.19 kernel (except the other glitches). I have not seen any crashes, kernel panic whatsoever. Lightdm/Mate is coming up fine, LO/Firefox ... work flawlessly. The 5.4 kernel is preparation for installing my new Radeon RX 5500XT grahics card which isn't yet installed.

Head_on_a_Stick wrote:

What is your graphics hardware? The Devuan firmware-amd-graphics packages don't have the Navi firmware yet.

For the time being I installed an older passive Radeon card that was available in my cellar.

# lspci | grep VGA
08:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM]

The mainboard wanted CSM enabled with this card, it complained that this card wasn't efi compatible which it is afaik. It was used in an older PC with an efi installation.

Head_on_a_Stick wrote:

rolfie wrote:
First installation attempt failed with grub not installing.
Did you check TTY4 or the installer log for the actual error message? It would be nice to know why it failed.

Can't recall the exact text, just saw on the 2nd console that there was no efi partition present in /target/boot. I tried to mount it and to repeat the grub-install, failed. Well, I searched all my threads in this forum and stumbled somewhere across the bit with the network connection. On the second install I paid explicit attention to if the manual partitioning shows any way to mount the efi partition to /boot, there was none. And against my usual habit I made sure the network was setup for using an external mirror. I checked /target/boot just before the grub-install (again on second console), this time the efi partition was mounted.

Head_on_a_Stick wrote:

rolfie wrote:
I did manually modify the boot number from 1 to 8. No change.
The only difference between those two entries is Secure Boot support, which is provided by shimx64.efi.

Secure boot is disabled (Other OS), and I did delete all key from the database. I always do this.

Head_on_a_Stick wrote:

rolfie wrote:
Why is Arch overwritten?
Because your UEFI firmware is a useless, buggy pile of crap. Most are. I can show you how to generate a new NVRAM entry for Arch but your motherboard would probably just forget it again.
rolfie wrote:
I though that two Linux OS could reside on the same drive with one efi partition without conflict.
There is no conflict, as such, the Arch entry has just been lost.
EDIT: actually I think the Arch entry might be Boot0000. What was the exact grub-install command you ran when installing Arch? /EDIT

I used

grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=GRUB --removable

so Boot0000 should be the Arch entry, but with F8 I do not get this as a choice. I haven't yet tried to use efibootmgr to change the boot order. Will be a separate experiment.

rolfie

rolfie · 2020-04-24 18:55:32

xinomilo wrote:

after modifying disk with gparted to prepare 2nd install, the disk uuid probably got modified. check from working system here : /dev/disk/by-uuid, if it's not the same, chroot to troubling installation, modify grub and run update-grub.. then try booting again.
note, i dont use UEFI. and grub loads luks partition from eg. 'root=/dev/mapper/system--vg-root'

Tried update-grub, made no difference. When I compare the output from blkid and what is listed in /dev/disk/by-uuid I see no difference. What else can I check?

I also had a look at grub.cfg. In this file this uuid is listed twice in the header part:

set root='lvmid/xS2VOe-2irS-1ZJQ-fWAz-FBAn-BYSx-sAs2sG/hfqu6W-qJ0f-iYDd-2Su6-ZgOg-0PXN-deZDHi'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint='lvmid/xS2VOe-2irS-1ZJQ-fWAz-FBAn-BYSx-sAs2sG/hfqu6W-qJ0f-iYDd-2Su6-ZgOg-0PXN-$
else
  search --no-floppy --fs-uuid --set=root 76c68ea3-5785-49b4-b93e-66f4de5b3a2b
fi
    font="/usr/share/grub/unicode.pf2"
fi

if loadfont $font ; then
  set gfxmode=auto
  load_video
  insmod gfxterm
  set locale_dir=$prefix/locale
  set lang=de_DE
  insmod gettext
fi
terminal_output gfxterm
insmod lvm
insmod ext2
set root='lvmid/xS2VOe-2irS-1ZJQ-fWAz-FBAn-BYSx-sAs2sG/hfqu6W-qJ0f-iYDd-2Su6-ZgOg-0PXN-deZDHi'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint='lvmid/xS2VOe-2irS-1ZJQ-fWAz-FBAn-BYSx-sAs2sG/hfqu6W-qJ0f-iYDd-2Su6-ZgOg-0PXN-$
else
  search --no-floppy --fs-uuid --set=root 76c68ea3-5785-49b4-b93e-66f4de5b3a2b
fi

and here is the main entry that also references this uuid (cut off here due to nano):

menuentry 'Debian GNU/Linux' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-76c6$
        load_video
        insmod gzio
        if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
        insmod part_gpt
        insmod ext2
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root  97e1104c-2123-44da-9e09-97a02728d093
        else
          search --no-floppy --fs-uuid --set=root 97e1104c-2123-44da-9e09-97a02728d093
        fi
        echo    'Linux 5.4.0-0.bpo.4-amd64 wird geladen …'
        linux   /vmlinuz-5.4.0-0.bpo.4-amd64 root=/dev/mapper/vgBsys-1_root ro  quiet
        echo    'Initiale Ramdisk wird geladen …'
        initrd  /initrd.img-5.4.0-0.bpo.4-amd64
}

Another observation is that the grub color is the Debian blue, cinnebar does not seem to be used. Further ideas?

Thanks, rolfie

golinux · 2020-04-24 19:59:06

Is desktop-base installed? That should pull in all parts of the cinnabar theme. Sorry, can't help with grub issues. Hope you get it straightened out.

Head_on_a_Stick · 2020-04-24 23:00:14

rolfie wrote:

Mainboard and Ryzen 7 3700X CPU work fine with 4.19 kernel (except the other glitches). I have not seen any crashes, kernel panic whatsoever. Lightdm/Mate is coming up fine, LO/Firefox ... work flawlessly.

Oh really? Do any errors show up in dmesg? How are the temperatures?

We've had quite a few problem threads over at the Arch forums about freezes with 3rd generation Ryzen even with kernels >5 so I just presumed 4.19 wouldn't work well at all.

rolfie wrote:

my new Radeon RX 5500XT grahics card

You're going to have fun with that one because the Devuan firmware packages haven't been updated since last July, even in ceres.

I've been advising people over at fdn with that card to install MX Linux's firmware package, that's been ported from Ubuntu and contains the latest Navi firmware packages: http://mxrepo.com/mx/repo/pool/ahs/l/li … +1_all.deb

rolfie wrote:

On the second install I paid explicit attention to if the manual partitioning shows any way to mount the efi partition to /boot, there was none.

Devuan should be mounting the ESP under /boot/efi rather than /boot.

rolfie wrote:

grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=GRUB --removable

Off-topic for these forums but you don't need to mount the ESP under /boot in Arch if you're using GRUB. I prefer to have it mounted under /efi but with the noauto option applied to save wear on the frangible FAT filesystem.

rolfie · 2020-04-25 19:29:23

golinux wrote:

Is desktop-base installed? That should pull in all parts of the cinnabar theme. Sorry, can't help with grub issues. Hope you get it straightened out.

It is. There is a Cinnabar theme present, but not used by grub. Lightdm/Mate use Cinnabar.

rolfie

rolfie · 2020-04-25 19:39:02

Strange, I did several experiments trying to affect the problems, still the same. Looked into a VM with Beowulf upgraded from ASCII, encrypted with LVM to compare.

Removed the shim entry from the efi, deleted all shim files by uninstalling via Synaptic, ... always used update-grub next and rebooted, still the same. Then I looked at /boot/grub/grub.cfg. That file still is dated April 18th. Shouldn't that file be changed by update-grub?

The VM installation runs 5.4-bpo2 kernel, grub.cfg had a date of end of Feb this year. Did a update-grub, the file now has today timestamp. ?????

Arch was installed with --bootloader-id=GRUB according to the German Arch Beginners Tutorial. Do I have some interference between these two installations?

rolfie

Last edited by rolfie (2020-04-25 19:40:19)

rolfie · 2020-04-25 20:37:42

I guess I looked at the wrong file in the efi directory which belongs to Arch.

Arch is booting fine when I change the boot order to 0.

Also created a new Devuan entry based on doing the opposite to https://dev1galaxy.org/viewtopic.php?id=3280.

Looks like the entries in the efi are duplicated, when using efibootmgr V17 on Arch, suddenly empty entries 2,3,4,5,6 were displayed. Tried to de-activate them under Arch, no help, under efibootmgr V15 (Beowulf) they are back again. Now I am getting more complaints before the blue grub screen comes up. Maybe update-grub under Beowulf ins't doing what its supposed to do?

rolfie

Head_on_a_Stick · 2020-04-25 23:08:07

rolfie wrote:

Arch was installed with --bootloader-id=GRUB according to the German Arch Beginners Tutorial. Do I have some interference between these two installations?

There's a German version of the old Beginner's Guide? Alad had that removed from the English site some time ago.

It won't cause interference but if you have the ESP mounted under /boot then the kernel & initramfs images will be on the FAT filesystem, which isn't ideal (IMO).

If you open a thread over at bbs.archlinux.org I can show you how to migrate /boot to the root partition, it's pretty simple to do.

rolfie wrote:

Looks like the entries in the efi are duplicated, when using efibootmgr V17 on Arch, suddenly empty entries 2,3,4,5,6 were displayed. Tried to de-activate them under Arch, no help, under efibootmgr V15 (Beowulf) they are back again. Now I am getting more complaints before the blue grub screen comes up. Maybe update-grub under Beowulf ins't doing what its supposed to do?

The update-grub command doesn't change the NVRAM entries at all, your problems are almost certainly due to a dodgy UEFI implementation.

rolfie · 2020-04-26 15:09:54

@Head_on_a_Stick:

1.) Lets move the Arch stuff discussion to a separate thread e.g. to DIY.

2.) You can't convince me that the EFI Bios of my ASUS Prime X570-Pro is crap and the one and only reason for my problems.

I have got VM with a Beowulf installed from a netinstall image about a year ago encrypted with LVM that shows the same effect when booting. grub complains about not finding the logical volume that hold the root partition. Traditional grub colors. Just update to the latest status, still the same.

On the other hand, my VM installed from the beta DVD with similar setup = encrypted LVM, just Cinnamon instead of Mate, shows the grub Cinnabar theme and no errors. Both VMs use efi.

There must be something in my setup that screws update-grub. Anyhow, if I want ASUS to fix the efi bios, I need some idea what to complain about, and I have no idea whats causing the problems.

3.) Running a Ryzen 3700X with kernel 4.19: I have rebooted the PC with that kernel and pulled dmesg into a text file. Shall I mail that to your attention (zipped)? What temps are concerned: looks like lmsensors just finds the old passive graphics card thats on the board currently. I would claim temps are fine. In the bios I got then in the mid of 30s. I have the cool aircooler shipped with the CPU installed (with LEDs) on the CPU, rotation is low in operation, its nicely regulated if higher load is applied. The initial burst on reset is there, then the speed is run down.

rolfie

Head_on_a_Stick · 2020-04-26 15:19:30

rolfie wrote:

Lets move the Arch stuff discussion to a separate thread e.g. to DIY.

Please open a thread at bbs.archlinux.org instead, I'm a member there as well :-)

rolfie wrote:

You can't convince me that the EFI Bios of my ASUS Prime X570-Pro is crap and the one and only reason for my problems

I'm not trying to convince you of anything — I'm just stating my opinion, which is based on countless threads reporting problems with half-assed UEFI implementations. But I've never actually used LUKS or LVM so that may be a factor here.

rolfie wrote:

Running a Ryzen 3700X with kernel 4.19: I have rebooted the PC with that kernel and pulled dmesg into a text file. Shall I mail that to your attention (zipped)?

Just upload it to a pastebin and share the link here, lockdown is sooooo boring that I'll probably read it all.

rolfie · 2020-05-22 19:05:16

Late feedback: Beowulf (fresh install from beta3 Desktop DVD) works fine on X570/Ryzen 7 3700X/RS5500XT/backports kernel 5.5 when you know how to get the Navi14 firmware to work:

This link give the requried input: https://bugs.debian.org/cgi-bin/bugrepo … =952710#10
I have learned that the bios 1407 supplied by ASUS insists on telling a PCIe3 GPA as non-efi compatible and switches on CSM. That causes some glitches during the installation. I am still fighting with the hotline, waiting for a fix. The 5500 runs fine without CSM now.

rolfie

Edith: corrected link URL

Last edited by rolfie (2020-06-07 12:33:25)

The officially official Devuan Forum!

#1 2020-04-23 19:31:23

[Closed] Problems with Beowulf as second OS on a SSD

#2 2020-04-23 23:23:08

Re: [Closed] Problems with Beowulf as second OS on a SSD

#3 2020-04-24 12:18:00

Re: [Closed] Problems with Beowulf as second OS on a SSD

#4 2020-04-24 16:49:06

Re: [Closed] Problems with Beowulf as second OS on a SSD

#5 2020-04-24 17:41:05

Re: [Closed] Problems with Beowulf as second OS on a SSD

#6 2020-04-24 18:55:32

Re: [Closed] Problems with Beowulf as second OS on a SSD

#7 2020-04-24 19:59:06

Re: [Closed] Problems with Beowulf as second OS on a SSD

#8 2020-04-24 23:00:14

Re: [Closed] Problems with Beowulf as second OS on a SSD

#9 2020-04-25 19:29:23

Re: [Closed] Problems with Beowulf as second OS on a SSD

#10 2020-04-25 19:39:02

Re: [Closed] Problems with Beowulf as second OS on a SSD

#11 2020-04-25 20:37:42

Re: [Closed] Problems with Beowulf as second OS on a SSD

#12 2020-04-25 23:08:07

Re: [Closed] Problems with Beowulf as second OS on a SSD

#13 2020-04-26 15:09:54

Re: [Closed] Problems with Beowulf as second OS on a SSD

#14 2020-04-26 15:19:30

Re: [Closed] Problems with Beowulf as second OS on a SSD

#15 2020-05-22 19:05:16

Re: [Closed] Problems with Beowulf as second OS on a SSD

Board footer