The officially official Devuan Forum!

You are not logged in.

#26 Re: News & Announcements » Testing for beowulf 3.1 point release » 2020-12-06 15:43:52

fsmithred wrote:

I don't know what decisions went into using start-stop-daemon. There were a few iterations of the patch, and they all seemed to work.

As it stands now, udev seems to work correctly, so it's not a big deal.
The only "problem" here may be a few microseconds and a few kb of ram temporarily used when starting the "start-stop-daemon" binary itself. I was only pointing this out because I believe any unneccessary "extra layer", even one as simple as start-stop-daemon, should be justified, and, as you know it had already caused some problems in the past.

Again, right now everything seems to work correctly wrt udev, so no problems here.

NTFS: If something is missing, I have no idea what it is. I did not remove any packages and ntfs-3g is installed in the live system. Do you maybe need to load a grub module before loading the kernel?

This is not a grub problem - grub loads the kernel and the initrd into memory from "within" the iso when the iso itself is on an ntfs partition.
The problem appears later, I think, when initrd is parsing the "fromiso=/dev/sda3:/path/to.iso" kernel command line parameter. It tries to mount the iso but fails, because it can not mount ntfs filesystem.
Edit:The full grub entry looks like this (has been working since forever with debian,devuan, ubuntu with some changes):

menuentry 'Devuan Ascii 2.1 Live cd' {
        set root="hd0,3"
        set isofile="/devuan_ascii_2.1_amd64_desktop-live.iso"
        set gfxpayload=keep
        loopback loop ($root)$isofile
        linux   (loop)/live/vmlinuz boot=live username=devuan fromiso=/dev/sda3$isofile
        initrd  (loop)/live/initrd.img
    }

This seems to be an initrd problem, it might be necessary to regenerate it and include the ntfs modules. Here's why I think so:
1) Open/unpack the "devuan_beowulf_3.1.0_2020-11-30_amd64_desktop-live.iso" file.
2) Within it: go to "/live" directory.
3) There is a file "initrd.img"
4) Open/unpack this file (on ascii, use "unmkinitramfs" command)
5) Go to "/lib/modules/4.19.0-13-amd64/kernel/fs" within it.
6) Note there are directories like btrfs, ext4, fat, xfs, but there is no "ntfs" directory
7) Now, inspect the "devuan_ascii_2.1_amd64_desktop-live.iso", for example, and see that "ntfs" directory is there. (obviously the path is a bit different due to different kernel version, "/lib/modules/4.9.0-11-amd64/kernel/fs". That one loads the iso from ntfs partition just fine.

Also, when unpacking the initrd from ascii 2.1 live iso when using "unmkinitramfs", we get two directories - "early" and "main", "early" contains processor microcode, "main" contains the initrd tree itself. That means there are basically two (2) archives in the initrd itself.

Shouldn't the "beowulf-3.1" initrd also include processor microcode?
Note: I am using the ascii "unmkinitramfs" script to unpack a "beowulf" initrd, that may be a source of the problem.

Please doublecheck.

acpi-fakekey is still there because I didn't think to remove it. I will do so in the rebuild.

Again, it doesn't cause problems, it's simply not needed if udev has done all the work wrt detecting mouse/kbd/sound etc. As I've said, udev seems to be working correctly.
However acpi-fakekey may be needed on the live image itself, if perhaps, something fails on an unusual configuration. Having an extra layer of reliability is not a bad idea for livecd.
But perhaps you can exclude it from the system which will be installed to hdd itself.

#27 Re: News & Announcements » Testing for beowulf 3.1 point release » 2020-12-05 18:54:36

Testing the amd64 live-iso.

Burned to physical dvd.

Udev seems fine, all modules loaded on first try (with or without "toram").
Network cards and sound working -  the system even detected which output I had plugged and set default output accordingly. This is even more than I expected - usually some tweaking is needed on my setup to select the correct output. Very good.
Switching betweet manual/dhcp, as well as lan/wi-fi using wicd - all ok.
Cryptsetup, lvm work ok.
Added backports (with contrib and non-free sections), built zfs-dkms modules, imported the pools - all ok.
Used "dkms mkbmdeb" to pack the zfs modules into a deb, rebooted and installed the deb to avoid recompile of zfs modules - all ok.

Module count, as indicated by "lsmod | wc -l" is the same w/ or wo/ "toram" parameter.

Problem:
Trying to load the iso from hd through grub entry failed:
"Warning: can not mount /dev/sda3".
That's an ntfs partition where I've placed the iso, and, as I suspected, the ntfs modules are missing from /live/boot.img.
Deliberate policy or accidental omission? I hope the latter.
I used to be able to boot debian/devuan/ubuntu and many others from ntfs partitions like this.
When placing the iso to an ext4 partition, the boot proceeds normally.

Minor issues:
1)Incosistent formatting wrt tabs/spaces in "/etc/init.d/eudev" between start) and other) sections. Has no run-time effect.
2)Usage of "start-stop-daemon" to start eudev is not neccessary. It doesn't do any damage in this configuration, but it's still not needed. Udev will be perfectly fine when started as "udevd --daemon" - the way it was in jessie/ascii, unless I am mistaken.
3)acpi-fakekeyd: not neccessary anymore since eudev startup seems to be in order. May be used as a fail-safe for live-cd, but maybe consider adding it to exclude list, if possible, for the actual refracta-install process. Some people have complained that system installed from live-image uses a bit more resources - it's not much, and does no harm, but acpi-fakekey may be one of the reasons.

Other than that, live-image seems very good.

Keep Devuan rock-solid.

#28 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-06-18 20:11:21

fsmithred wrote:

Put the sleep after start-stop-daemon and right before 'udevadm trigger --action=add'.

Confirming: adding sleep 3 between start-stop-daemon and udevadm trigger --action=add allows to boot livecd with no problem.

Update:
The bug report contains a proposition to try to run start-stop-daemon without --background, but using --daemon argument. That means adding --daemon to the end of start-stop-daemon line, and it will be passed to eudev. I think I've tried that right before the beowulf release, but that didn't seem to work.

#29 Re: News & Announcements » Beowulf Beta is here! » 2020-06-01 17:27:38

Just tested the beowulf hd-media installer located at
https://pkgmaster.devuan.org/devuan/dis … /hd-media/

The gtk installer doesn't boot, enters in a loop throwing xserver errors.
Fixed by specifying 'vga=788' at  the boot command line.

When using expert install, both gtk and text installers will, after detecting the iso, say something like:
'FIXME' iso mounted. The iso 'devuan-beowulf-..blah...iso' mounted at /dev/sda1 (FIXME).

So that's 2 FIXME entries somewhere.

Tested with beowulf dvd install image.

Other than that both text and gtk installers seem to work fine.

#30 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-30 16:01:46

fsmithred wrote:

(using refracta10-beta5)

So what exactly is refracta-beta5 ?
There were so many of them I am getting confused.

#31 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-29 22:59:50

fsmithred wrote:

Right now, I'm leaning toward doing the easiest workaround to get beowulf out the door.

I agree, that should be the top priority.
Especially since the beowulf packages have been ready for some time, and beowulf systems themselves seem to run fine.

Jessie is really getting a bit long in the tooth, time to migrate.

Also:
I checked the ascii 2.1 live dvd, which always worked ok for me. It starts eudev as udevd --daemon.

Last thing: please confirm that my fix works on your machine, not only on mine.

#32 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-29 20:10:18

fsmithred wrote:

Made another refracta iso and set udev.conf log level to debug.

I've already managed to build my own "beowulf live mate edition" iso using refractasnapshot.
It works and behaves the same as refracta/beowulf rc with regards to eudev/modules.

TLDR:
Udev works when started as udevd -d, but doesn't work when wrapped with start-stop-daemon.

In more detail:
When booting, take a look at the messages early on. When displaying "waiting for /dev to be fully populated" there's a message: settle: could not connect to daemon.
When booting to runlevel 1, and looking at the running processes we see udev isn't there. There's also an orphaned pid file left, as well as /run/omitsigs.... link to it.

It probably means that udevd dies/gets killed early on before it finishes detecting everything.

The simplest way to restart eudev is:

udevd -d
udevadm trigger --action=add

Doing service eudev stop is not needed because there is simply nothing to stop, it isn't there.
Doing service eudev restart doesn't help because even if does start the daemon (I am not sure it does), it doesn't call udevadm trigger --action=add.

After tinkering with init script a bit, I decided to replace the start-stop-daemon invocation of eudev with a simple udevd -d invocation. I've burnt a live cd with this version of eudev init script, and it seems to work flawlessly without any other workaround. (I am still testing this).

The start-stop-daemon is needed to grab udevd's pid and write a pid file. Also a link is placed to /run/sendsigs... to prevent sendsigs
from killing udev earlier than needed. As specified in the comments of eudev init script, this is related to debian #791944.
Take a look at this thread here:
https://bugs.debian.org/cgi-bin/bugrepo … bug=791944
It's also related to delays on shutdown when unmounting encrypted stuff.

All of this is probably unneccessary on the live cd, and even if it is, we can just try to manually grab the pid of udevd and create the pid file and sendsigs.omit link ourselves without using start-stop-daemon.

edit:typo

#33 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-29 18:20:05

Try the following:

Edit /etc/init.d/eudev startup script.

In the start) section, comment out the start-stop-daemon command, along with the enclosing if statement and add
the following line below it: udevd -d.

It should look like this:

    # clean up parts of the database created by the initramfs udev
    udevadm info --cleanup-db
   
    # set the SELinux context for devices created in the initramfs
    [ -x /sbin/restorecon ] && /sbin/restorecon -R /dev
  
    log_daemon_msg "Starting $DESC" "$NAME"

#   Commented out code is below this line

#    if start-stop-daemon --start $NAME --user root --quiet \
#      --pidfile $PIDFILE --exec $DAEMON --background --make-pidfile; then
#      # prevent udevd to be killed by sendsigs (see #261 & DEBIAN #791944)
#      mkdir -p $OMITDIR/$NAME
#      ln -sf $PIDFILE $OMITDIR/$NAME
#<----->  log_end_msg $?
#    else
#<----->log_warning_msg $?
#<----->log_warning_msg "Waiting 15 seconds and trying to continue anyway"
#<----->sleep 15
#    fi

#   Commented out code is above this line

#   Inserted code is below this line
    udevd -d
#   Inserted code is above this line

    log_action_begin_msg "Synthesizing the initial hotplug events"
    if udevadm trigger --action=add; then
<------>log_action_end_msg $?
    else
<------>log_action_end_msg $?
    fi

#34 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-27 17:36:39

For this "stopstart" edition of refracta, the module count is the same when booting from dvd with or without the "toram" parameter and everything seems to work.

Also, the module count when booting the iso from hard disk (no "toram"), and when booting from dvd (no "toram"), but also mounting the hard disk partition where the iso resides, is the same.

#35 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-27 16:27:55

fsmithred wrote:

I did something similar - added to /etc/rc.local the following line:

/etc/init.d/eudev stop && /etc/init.d/eudev start

and it works.

I tried editing rc.local as well but it didn't work for me. Maybe I did something incorrectly.
After that I decided to do it "by the book" and create an actual "init script" for this.

Here's a test iso:
https://get.refracta.org/files/testing/ … 7_1508.iso

sha256sum

59aea5d5075815f07f34cae046dc618a9ce99de78c2b737f8db36bd447cf8674  refracta10+stopstart_xfce_amd64-20200527_1508.iso

I'm downloading it and will give it a spin.

It's odd that using "stop" and "start" for eudev works, but using "restart" does not.

I noticed that too.
The "restart" portion of the eudev init script doesn't simply execute "start" and "stop" portions. It behaves differently. It just calls start-stop-daemon 2 times.

The "start" portion of the eudev init script, on the other hand, does a lot of things. It performs mounts, remounts, makes nodes, etc, a lot of important work. It seems to me that something within that portion fails for the first time (on initial boot), but succeeds when we run it a bit later in the boot sequence.

Bad thing that there is not much logging going on.

I was going to suggest that if you were to make a refracta iso again, maybe you could edit /etc/udev/udev.conf and change the udev_log parameter so that we can get as much information as possible. Udev may have something interesting to say about all of this, we should listen.

#36 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-26 19:50:22

Next excercise:
1) Take beowulf rc live-image on dvd
2) Boot to runlevel 1.
3) Enter root password
4) Create a file named

/etc/init.d/restart-eudev

with the following contents:

#!/bin/sh
### BEGIN INIT INFO
# Provides:		restart-eudev
# Required-Start:
# Required-Stop:
# Should-Start:
# Should-Stop:
# Default-Start:	2 3 4 5 
# Default-Stop:
# Short-Description:	Restarts eudev to detect more devices.
# Description:		Runs "service eudev stop" followed by "service eudev start"
#			May be useful on a livecd.
### END INIT INFO

PATH=/sbin:/bin:/usr/sbin:/usr/bin

. /lib/lsb/init-functions

do_restart_eudev () {
	log_action_msg "Restarting eudev"
	service eudev stop
	service eudev start
}

case "$1" in
  start)
	do_restart_eudev
	;;
  restart|reload|force-reload)
	echo "Error: argument '$1' not supported" >&2
	exit 3
	;;
  stop|status)
	# No-op
	;;
  *)
	echo "Usage: $0 start|stop" >&2
	exit 3
	;;
esac

5) Run the following command:

update-rc.d restart-eudev defaults

6) Bonus points if you can do the following to prove the old work-around isn't needed anymore:

apt purge acpi-fakekey

7) Logout to proceed to runlevel 2.
Watch out for the following message:
[info] Restarting eudev

8) Test mouse, keyboard, sound, pc-speaker, etc.

#37 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-26 19:45:42

pcalvert wrote:

That just fixes the keyboard and mouse problem, right? Because I just tried it, and sound still isn't working.

For me this fixes everything, including the sound. All the way up to pc-speaker.
I am testing on the beowulf rc live-image

#38 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-25 17:30:01

pcalvert wrote:

That may help with some issues, but it doesn't look like it will help with the no-sound issue (for me, anyway).

Here's why:

# groupadd kvm
groupadd: group 'kvm' already exists

Phil

Have you done that on the iso that you've created, or on the beowulf rc live-image?

I tested on the beowulf rc live-image burned to dvd.

I boot into runlevel 1. Run those commands.
Eudev instantly detects a lot of things that it didn't detect before that. Including pc-speaker. Also my console font immediately changes to the correct one. Means video-device is properly detected. Doing 'service alsa-utils restart' after that I get the sound immediately.
After that I purge the acpi-fakekey, start the desktop and everything including mouse and kbd works.

Beowulf live-image on dvd, no 'toram', bare-metal.

#39 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-25 14:04:55

I think I've found the fix.
Eudev needs group "kvm" to be present.

Try booting the beowulf live image rc from dvd into runlevel 1, after that

service eudev stop
groupadd kvm
service eudev start

Seems to fix the issue with missing modules. You don't even need acpi-fakekey for the mouse after that.

Update: turns out the kvm group has no effect - but stopping and restarting do.

#40 Re: News & Announcements » Beowulf Beta is here! » 2020-05-24 18:24:08

Booting the beowulf live image rc into runlevel 1, after that

service eudev stop
groupadd kvm
service eudev start

seems to fix the issue with missing modules.

#41 Re: News & Announcements » Beowulf Beta is here! » 2020-05-24 12:17:22

Setting CRYPTSETUP=y will cause cryptsetup to be added to the initramfs if the root filesystem is NOT encrypted. (just tested this to be sure.) If root is encrypted, it'll be added without this setting. The wording in the comment for this setting suggests that in the future, it will be added if cryptsetup is installed, regardless of the root filesystem.

I stand corrected.

I do remember though that when getting the warning

cryptsetup: WARNING: The initramfs image may not contain cryptsetup binaries 
    nor crypto modules. If that's on purpose, you may want to uninstall the 
    'cryptsetup-initramfs' package in order to disable the cryptsetup initramfs 
    integration and avoid this warning.

I did get into trouble (unable to decrypt root filesystem) a few times because cryptsetup bins were not included. So I learned to pay attention to it if I need to decrypt something at the initramfs stage.

The preferrable way of including cryptsetup bins for me is adding 'initramfs' to options column of crypttab. Like this:

# <target name> <source device>         <key file>      <options>
target_device_name   UUID=<..uuid here..>   none  luks,initramfs

And you don't need to add CRYPTSETUP=y. This is for ascii and later.

I hope Lomax was able to boot successfully.

#42 Re: News & Announcements » Beowulf Beta is here! » 2020-05-24 02:17:58

Lomax wrote:

Excellent, then it is as I thought, and the warning is misleading.

The warning isn't misleading, because it says cryptsetup won't be included in the initramfs, and that's exactly what will happen.
But you simply don't need the cryptsetup binaries in the initramfs if your root isn't encrypted. The binaries will be accessible when the root filesystem is mounted.

Adding CRYPTSETUP=y to /usr/share/initramfs-tools/conf-hooks.d/cryptsetup isn't exactly including it manually. This just specifies that cryptsetup will be added automatically if the root filesystem itself is encrypted.

Hope it helps.

#43 Re: News & Announcements » Beowulf Beta is here! » 2020-05-24 01:45:34

Lomax wrote:

The upgrade seems to have gone well, but at the end of it I got a curious warning from update-initramfs:

If your root is unencypted you may boot but will have to unlock crypt partitions manually.

If you've only encrypted home and swap then you don't need cryptsetup in the initramfs and can ignore the warning. The root will be mounted and after that the cryptsetup binaries are accessible to the system in order to unlock the rest of partitions.

If you've described the situation accurately, you'll probably be able to boot and login as root. After that you can unlock the rest and add the required lines to crypttab/fstab.

#44 Re: Installation » Beowulf install » 2020-05-21 14:07:19

Does OP have an nvidia graphics card?

#45 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-21 13:57:28

I myself don't often use optical images for installation in most cases.

My preferred method is to put isos on hard drive and create grub entries to boot into them. Writing the iso to a usb stick is as redundant as burning it to a dvd in most cases.

There are cases where it's not possible, or not convinient, to do it from hd and in those cases external media, dvd/usb, comes to the rescue.

My dvd drive is a very old ide drive. It's by no means "modern bd crap".
Jessie and ascii live/install images, burned to a dvd, always worked flawlessly for me. I also sometimes write an ubuntu lts version to a disc and it always worked without problem as well.

Also, fsmithred said that the problem appears on some usb configurations as well, it's not limited to optical media.

#46 Re: News & Announcements » Beowulf Beta is here! » 2020-05-19 22:44:00

It seems that udev is confused, that's why not all modules are loaded.

The first initscripts to boot on a beowulf system already installed to hd in runlevel S:

/etc/rcS.d:
@S01mountkernfs.sh
@S02eudev 

The first init scripts to boot on the beowulf live-image from dvd in runlevel S:

/etc/rcS.d:
squashfs-root/etc/rcS.d/S01live-config
squashfs-root/etc/rcS.d/S02mountkernfs.sh
squashfs-root/etc/rcS.d/S03eudev

It is possible that live-config does something which works when using "toram", but fails when booting from the dvd.

Can you change the order of execution so that live-config runs after udev, or it HAS to be the first thing to run?

Again, just a wild guess here.

#47 Re: News & Announcements » Beowulf Beta is here! » 2020-05-19 21:08:36

fsmithred wrote:

Now we're cross-posting. See my update above.

Yes, I've read it just now smile We were posting the same info at the same moment.

I think just using single-threaded boot may not be enough here, we probably need unique <X,Y> numbers in /etc/rc.<X>/S<Y>script files, so that the scripts would be sorted and executed in the correct order.

#48 Re: News & Announcements » Beowulf Beta is here! » 2020-05-19 20:57:45

How can you even tell if it's running single or multi-threaded?

Take a look at /etc/init.d/rc file.
In particular, look for "CONCURRENCY=makefile" line.

TL;DR: in beowulf, add concurrency=none to kernel command line to make the boot single threaded.
It will be possible to strictly control boot ordering by assigning every script /etc/rc<X>.d/S<Y>script a unique <X,Y> number, and thus control boot process from start to finish.

I tried using "concurrency=none" on the beowulf-rc image, didn't help. Meaning sound card wasn't working, otherwise it's ok.

Note: I am making wild guesses here. If you feel like this is a waste of time, don't burn your energy on it.
I am digging in this direction only because we both thought of a possible timeout issue.

EDIT:
I've also looked up things like "alsa doesn't detect soundcard", but I couldn't find anything that looks helpful for our particular case.

The fact that so many modules are missing when booting from dvd is, of course, very very puzzling.

BTW, list of modules which I have missing, i.e. modules present when using toram but absent when booting from dvd:

acpi_cpufreq
coretemp
fuse
gameport
iTCO_vendor_support
iTCO_wdt
joydev
pcc_cpufreq
pcspkr
serio_raw
snd
snd_ens1370
snd_pcm
snd_rawmidi
snd_seq_device
snd_timer
soundcore

#49 Re: News & Announcements » Beowulf Beta is here! » 2020-05-19 20:01:58

How can you even tell if it's running single or multi-threaded?

It says "entering makefile-style CONCURRENT boot in runlevel S".

Which is supposed to mean I think that it starts (or tries to start) at least some processes in parallel.

Anyway we can tell init to start only a single process at once, thus doing it all sequentially? Sort of like doing "make -j1" to only execute one job at a time.

What I can do is turn off some services in the default runlevel and see if that changes anything.

Only if you have any particular ideas as to what exactly should be turned off to check what difference it makes.

#50 Re: News & Announcements » Beowulf Beta is here! » 2020-05-19 16:50:37

fsmithred wrote:

I'm thinking maybe there's a timeout somewhere on reading the dvd, and it just stops reading early.

The thought of a timeout did also cross my mind a few days ago.

Besides differences in the amount of modules loaded, there's also a difference of how the root filesystem is mounted between dvd and toram.
Maybe something isn't able to get write access to the rootfs when it's a read-only fs from the cd?

Also:
The boot is multi-threaded, as far as I can see. When loading from a dvd, some processes take a lot longer than from a ramdisk.
Therefore: perhaps the BOOT ORDER is simply different due to read-delays? Thus we get a completely different boot processs.

Would it be hard to create a refracta version which uses a single-threaded boot, and manually fix the boot order?

Surely when booting a live dvd, a predictable and reliable boot order is more important than utilizing MOAR COARZ(tm) for a multi-threaded boot and cutting 0.73 seconds from the boot time.

Board footer

Forum Software