The officially official Devuan Forum!

You are not logged in.

#1 Re: News & Announcements » Testing for beowulf 3.1 point release » 2020-12-07 21:31:22

Got it to work by applying the same changes gparted-live people did.

Patching the file

9990-misc-helpers.sh

with the following patch

474a475,481
>   if [ "${fstype}" = "ntfs" ]; then
>       if type ntfs-3g >/dev/null 2>&1; then
>           return 0
>       else
>           return 1
>       fi
>   fi

seems to be enough.

Pro-tip: for a quick test you can avoid rebuilding the initrd by booting the iso with "break=mount" command line parameter, and applying the patch right in the shell.

#2 Re: News & Announcements » Testing for beowulf 3.1 point release » 2020-12-07 13:36:34

fsmithred wrote:

If you try patching 9990-misc-helpers.sh please let me know how it goes. That file already gets patched for refracta2usb, and I could add the ntfs support next time I touch that package.

I've tried patching it myself, but I am unable to boot with a modified initrd which I've reconstructed by myself.

Maybe you can suggest something?

I unpack the beowulf-live initrd with

unmkinitramfs initrd.img /path/to/extracted/initrd-tree

After I've applied some changes, how do I compress it in exactly the same format?
I've tried the following:

find . | cpio -H newc -o > ../initramfs.cpio
cat initramfs.cpio | gzip > initramfs.igz

But boot failed immediately and dropped me to the shell even though I've only added some debug prints.

Do I have to insert the initrd into the iso image itself, or can I simply modify grub entry to load the custom initrd and it should work?

#3 Re: News & Announcements » Testing for beowulf 3.1 point release » 2020-12-07 13:31:53

I've tested the gparted-live-1.0.0-3 live image, which is being discussed in the forum post I've linked at message #8 in this thread.
The url for the actual live image is:
https://sourceforge.net/projects/gparte … o/download

I was able to boot it to desktop from an ntfs partition.

I looked into the initrd, /lib/live/boot directory and ran a diff command between beowulf's files at the same location in the initrd.
There are a number of differences, I'm not posting full diff because most of them don't seem related to ntfs.
However, here's the diff between the 9990-misc-helpers.sh file:

82c82
<   elif echo "${sysfs_path}" | grep -q '^/block/vd[a-z]$'
---
>   elif echo "${sysfs_path}" | grep -q '^/block/[hsv]d[a-z]$'
474a475,482 
>   # For ntfs, since user space program ntfs-3g will be used. Check ntfs-3g instead of kernel module.
>   if [ "${fstype}" = "ntfs" ]; then
>       if type ntfs-3g >/dev/null 2>&1; then
>           return 0
>       else
>           return 1
>       fi
>   fi
1222c1230
<       if [ "$(cat ${sysblock}/removable)" = "1" ]
---
>       if [ "$(cat ${sysblock}/removable 2>/dev/null)" = "1" ]
1270c1278
<       if [ "$(cat ${sysblock}/removable)" = "0" ]
---
>       if [ "$(cat ${sysblock}/removable 2>/dev/null)" = "0" ]

The second difference looks almost exactly like the post you linked to in message #9 of this thread.

#4 Re: News & Announcements » Testing for beowulf 3.1 point release » 2020-12-07 13:00:10

fsmithred wrote:

I think this post has the magic sauce: https://lists.debian.org/debian-live/20 … 00038.html
That function, is_supported_fs is in /lib/live/boot/9990-misc-helpers.sh

That seems close to the target.
If I have some time and energy in the next few days, maybe I'll try looking into it more. It might be just a few lines of code needed to fix it.

For the moment, we can conclude that it's another feature that has been working for decades but has been cancelled by Debian.
Thus it doesn't block release of beowulf 3.1.

#5 Re: News & Announcements » Testing for beowulf 3.1 point release » 2020-12-06 18:55:16

fsmithred wrote:

Where the hell did ntfs.ko go? On my installed beowulf system, it's missing. Same ntfs packages are installed as in my ascii, which does have ntfs.ko.

Thanks for paying attention and replying, fsmithred. That's why I (and I'm sure many others) use devuan - because of the sensible attitude towards resolving actual issues.

However, in this case:
FALSE ALARM!
I've just tested the debian-live-10.7.0-mate-amd64.iso directrly from debian.

It behaves exactly the same. No ntfs.ko in initramfs. "Warning: unable to mount /dev/sda3" when trying to access iso on ntfs partition.

Therefore: it's a Debian change, and Devuan simply mirrored it, and there's no direct obligation for Devuan team to do anything about it since it isn't connected to *ystemd.

In ascii, apt-file shows that it comes with the kernel. In beowulf, apt-file doesn't find ntfs.ko.

I can see the same results for ascii. I don't have a beowulf system right now near me.

If we were to try:
The simplest way to include ntfs.ko into the initramfs would probably be modifying "/etc/initramfs-tools/modules", but first making sure that the modules themselves are installed on the base system of course. I don't know why they aren't there on beowulf.

Can you tell me how big a partition I would need to install a minimal beowulf 3.1 system from the live image? If I have a partition big enough I can maybe try and find an easy solution.

In the meantime - this isn't a Devuan problem, so it's a non-blocking issue if you are planning to release 3.1.
As I described, the beowulf 3.0 problems with eudev are not present in 3.1, so progress has already been made.

Update:
Debian has dropped the support of ntfs.ko: (see: 29 June 2019: GParted Live 1.0.0-3 Stable Release)
https://gparted.org/news.php?item=224Release

And here people seem to be solving the same problem:
http://gparted-forum.surf4.info/viewtopic.php?pid=35358

#6 Re: News & Announcements » Testing for beowulf 3.1 point release » 2020-12-06 15:43:52

fsmithred wrote:

I don't know what decisions went into using start-stop-daemon. There were a few iterations of the patch, and they all seemed to work.

As it stands now, udev seems to work correctly, so it's not a big deal.
The only "problem" here may be a few microseconds and a few kb of ram temporarily used when starting the "start-stop-daemon" binary itself. I was only pointing this out because I believe any unneccessary "extra layer", even one as simple as start-stop-daemon, should be justified, and, as you know it had already caused some problems in the past.

Again, right now everything seems to work correctly wrt udev, so no problems here.

NTFS: If something is missing, I have no idea what it is. I did not remove any packages and ntfs-3g is installed in the live system. Do you maybe need to load a grub module before loading the kernel?

This is not a grub problem - grub loads the kernel and the initrd into memory from "within" the iso when the iso itself is on an ntfs partition.
The problem appears later, I think, when initrd is parsing the "fromiso=/dev/sda3:/path/to.iso" kernel command line parameter. It tries to mount the iso but fails, because it can not mount ntfs filesystem.
Edit:The full grub entry looks like this (has been working since forever with debian,devuan, ubuntu with some changes):

menuentry 'Devuan Ascii 2.1 Live cd' {
        set root="hd0,3"
        set isofile="/devuan_ascii_2.1_amd64_desktop-live.iso"
        set gfxpayload=keep
        loopback loop ($root)$isofile
        linux   (loop)/live/vmlinuz boot=live username=devuan fromiso=/dev/sda3$isofile
        initrd  (loop)/live/initrd.img
    }

This seems to be an initrd problem, it might be necessary to regenerate it and include the ntfs modules. Here's why I think so:
1) Open/unpack the "devuan_beowulf_3.1.0_2020-11-30_amd64_desktop-live.iso" file.
2) Within it: go to "/live" directory.
3) There is a file "initrd.img"
4) Open/unpack this file (on ascii, use "unmkinitramfs" command)
5) Go to "/lib/modules/4.19.0-13-amd64/kernel/fs" within it.
6) Note there are directories like btrfs, ext4, fat, xfs, but there is no "ntfs" directory
7) Now, inspect the "devuan_ascii_2.1_amd64_desktop-live.iso", for example, and see that "ntfs" directory is there. (obviously the path is a bit different due to different kernel version, "/lib/modules/4.9.0-11-amd64/kernel/fs". That one loads the iso from ntfs partition just fine.

Also, when unpacking the initrd from ascii 2.1 live iso when using "unmkinitramfs", we get two directories - "early" and "main", "early" contains processor microcode, "main" contains the initrd tree itself. That means there are basically two (2) archives in the initrd itself.

Shouldn't the "beowulf-3.1" initrd also include processor microcode?
Note: I am using the ascii "unmkinitramfs" script to unpack a "beowulf" initrd, that may be a source of the problem.

Please doublecheck.

acpi-fakekey is still there because I didn't think to remove it. I will do so in the rebuild.

Again, it doesn't cause problems, it's simply not needed if udev has done all the work wrt detecting mouse/kbd/sound etc. As I've said, udev seems to be working correctly.
However acpi-fakekey may be needed on the live image itself, if perhaps, something fails on an unusual configuration. Having an extra layer of reliability is not a bad idea for livecd.
But perhaps you can exclude it from the system which will be installed to hdd itself.

#7 Re: News & Announcements » Testing for beowulf 3.1 point release » 2020-12-05 18:54:36

Testing the amd64 live-iso.

Burned to physical dvd.

Udev seems fine, all modules loaded on first try (with or without "toram").
Network cards and sound working -  the system even detected which output I had plugged and set default output accordingly. This is even more than I expected - usually some tweaking is needed on my setup to select the correct output. Very good.
Switching betweet manual/dhcp, as well as lan/wi-fi using wicd - all ok.
Cryptsetup, lvm work ok.
Added backports (with contrib and non-free sections), built zfs-dkms modules, imported the pools - all ok.
Used "dkms mkbmdeb" to pack the zfs modules into a deb, rebooted and installed the deb to avoid recompile of zfs modules - all ok.

Module count, as indicated by "lsmod | wc -l" is the same w/ or wo/ "toram" parameter.

Problem:
Trying to load the iso from hd through grub entry failed:
"Warning: can not mount /dev/sda3".
That's an ntfs partition where I've placed the iso, and, as I suspected, the ntfs modules are missing from /live/boot.img.
Deliberate policy or accidental omission? I hope the latter.
I used to be able to boot debian/devuan/ubuntu and many others from ntfs partitions like this.
When placing the iso to an ext4 partition, the boot proceeds normally.

Minor issues:
1)Incosistent formatting wrt tabs/spaces in "/etc/init.d/eudev" between start) and other) sections. Has no run-time effect.
2)Usage of "start-stop-daemon" to start eudev is not neccessary. It doesn't do any damage in this configuration, but it's still not needed. Udev will be perfectly fine when started as "udevd --daemon" - the way it was in jessie/ascii, unless I am mistaken.
3)acpi-fakekeyd: not neccessary anymore since eudev startup seems to be in order. May be used as a fail-safe for live-cd, but maybe consider adding it to exclude list, if possible, for the actual refracta-install process. Some people have complained that system installed from live-image uses a bit more resources - it's not much, and does no harm, but acpi-fakekey may be one of the reasons.

Other than that, live-image seems very good.

Keep Devuan rock-solid.

#8 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-06-18 20:11:21

fsmithred wrote:

Put the sleep after start-stop-daemon and right before 'udevadm trigger --action=add'.

Confirming: adding sleep 3 between start-stop-daemon and udevadm trigger --action=add allows to boot livecd with no problem.

Update:
The bug report contains a proposition to try to run start-stop-daemon without --background, but using --daemon argument. That means adding --daemon to the end of start-stop-daemon line, and it will be passed to eudev. I think I've tried that right before the beowulf release, but that didn't seem to work.

#9 Re: News & Announcements » Beowulf Beta is here! » 2020-06-01 17:27:38

Just tested the beowulf hd-media installer located at
https://pkgmaster.devuan.org/devuan/dis … /hd-media/

The gtk installer doesn't boot, enters in a loop throwing xserver errors.
Fixed by specifying 'vga=788' at  the boot command line.

When using expert install, both gtk and text installers will, after detecting the iso, say something like:
'FIXME' iso mounted. The iso 'devuan-beowulf-..blah...iso' mounted at /dev/sda1 (FIXME).

So that's 2 FIXME entries somewhere.

Tested with beowulf dvd install image.

Other than that both text and gtk installers seem to work fine.

#10 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-30 16:01:46

fsmithred wrote:

(using refracta10-beta5)

So what exactly is refracta-beta5 ?
There were so many of them I am getting confused.

#11 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-29 22:59:50

fsmithred wrote:

Right now, I'm leaning toward doing the easiest workaround to get beowulf out the door.

I agree, that should be the top priority.
Especially since the beowulf packages have been ready for some time, and beowulf systems themselves seem to run fine.

Jessie is really getting a bit long in the tooth, time to migrate.

Also:
I checked the ascii 2.1 live dvd, which always worked ok for me. It starts eudev as udevd --daemon.

Last thing: please confirm that my fix works on your machine, not only on mine.

#12 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-29 20:10:18

fsmithred wrote:

Made another refracta iso and set udev.conf log level to debug.

I've already managed to build my own "beowulf live mate edition" iso using refractasnapshot.
It works and behaves the same as refracta/beowulf rc with regards to eudev/modules.

TLDR:
Udev works when started as udevd -d, but doesn't work when wrapped with start-stop-daemon.

In more detail:
When booting, take a look at the messages early on. When displaying "waiting for /dev to be fully populated" there's a message: settle: could not connect to daemon.
When booting to runlevel 1, and looking at the running processes we see udev isn't there. There's also an orphaned pid file left, as well as /run/omitsigs.... link to it.

It probably means that udevd dies/gets killed early on before it finishes detecting everything.

The simplest way to restart eudev is:

udevd -d
udevadm trigger --action=add

Doing service eudev stop is not needed because there is simply nothing to stop, it isn't there.
Doing service eudev restart doesn't help because even if does start the daemon (I am not sure it does), it doesn't call udevadm trigger --action=add.

After tinkering with init script a bit, I decided to replace the start-stop-daemon invocation of eudev with a simple udevd -d invocation. I've burnt a live cd with this version of eudev init script, and it seems to work flawlessly without any other workaround. (I am still testing this).

The start-stop-daemon is needed to grab udevd's pid and write a pid file. Also a link is placed to /run/sendsigs... to prevent sendsigs
from killing udev earlier than needed. As specified in the comments of eudev init script, this is related to debian #791944.
Take a look at this thread here:
https://bugs.debian.org/cgi-bin/bugrepo … bug=791944
It's also related to delays on shutdown when unmounting encrypted stuff.

All of this is probably unneccessary on the live cd, and even if it is, we can just try to manually grab the pid of udevd and create the pid file and sendsigs.omit link ourselves without using start-stop-daemon.

edit:typo

#13 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-29 18:20:05

Try the following:

Edit /etc/init.d/eudev startup script.

In the start) section, comment out the start-stop-daemon command, along with the enclosing if statement and add
the following line below it: udevd -d.

It should look like this:

    # clean up parts of the database created by the initramfs udev
    udevadm info --cleanup-db
   
    # set the SELinux context for devices created in the initramfs
    [ -x /sbin/restorecon ] && /sbin/restorecon -R /dev
  
    log_daemon_msg "Starting $DESC" "$NAME"

#   Commented out code is below this line

#    if start-stop-daemon --start $NAME --user root --quiet \
#      --pidfile $PIDFILE --exec $DAEMON --background --make-pidfile; then
#      # prevent udevd to be killed by sendsigs (see #261 & DEBIAN #791944)
#      mkdir -p $OMITDIR/$NAME
#      ln -sf $PIDFILE $OMITDIR/$NAME
#<----->  log_end_msg $?
#    else
#<----->log_warning_msg $?
#<----->log_warning_msg "Waiting 15 seconds and trying to continue anyway"
#<----->sleep 15
#    fi

#   Commented out code is above this line

#   Inserted code is below this line
    udevd -d
#   Inserted code is above this line

    log_action_begin_msg "Synthesizing the initial hotplug events"
    if udevadm trigger --action=add; then
<------>log_action_end_msg $?
    else
<------>log_action_end_msg $?
    fi

#14 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-27 17:36:39

For this "stopstart" edition of refracta, the module count is the same when booting from dvd with or without the "toram" parameter and everything seems to work.

Also, the module count when booting the iso from hard disk (no "toram"), and when booting from dvd (no "toram"), but also mounting the hard disk partition where the iso resides, is the same.

#15 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-27 16:27:55

fsmithred wrote:

I did something similar - added to /etc/rc.local the following line:

/etc/init.d/eudev stop && /etc/init.d/eudev start

and it works.

I tried editing rc.local as well but it didn't work for me. Maybe I did something incorrectly.
After that I decided to do it "by the book" and create an actual "init script" for this.

Here's a test iso:
https://get.refracta.org/files/testing/ … 7_1508.iso

sha256sum

59aea5d5075815f07f34cae046dc618a9ce99de78c2b737f8db36bd447cf8674  refracta10+stopstart_xfce_amd64-20200527_1508.iso

I'm downloading it and will give it a spin.

It's odd that using "stop" and "start" for eudev works, but using "restart" does not.

I noticed that too.
The "restart" portion of the eudev init script doesn't simply execute "start" and "stop" portions. It behaves differently. It just calls start-stop-daemon 2 times.

The "start" portion of the eudev init script, on the other hand, does a lot of things. It performs mounts, remounts, makes nodes, etc, a lot of important work. It seems to me that something within that portion fails for the first time (on initial boot), but succeeds when we run it a bit later in the boot sequence.

Bad thing that there is not much logging going on.

I was going to suggest that if you were to make a refracta iso again, maybe you could edit /etc/udev/udev.conf and change the udev_log parameter so that we can get as much information as possible. Udev may have something interesting to say about all of this, we should listen.

#16 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-26 19:50:22

Next excercise:
1) Take beowulf rc live-image on dvd
2) Boot to runlevel 1.
3) Enter root password
4) Create a file named

/etc/init.d/restart-eudev

with the following contents:

#!/bin/sh
### BEGIN INIT INFO
# Provides:		restart-eudev
# Required-Start:
# Required-Stop:
# Should-Start:
# Should-Stop:
# Default-Start:	2 3 4 5 
# Default-Stop:
# Short-Description:	Restarts eudev to detect more devices.
# Description:		Runs "service eudev stop" followed by "service eudev start"
#			May be useful on a livecd.
### END INIT INFO

PATH=/sbin:/bin:/usr/sbin:/usr/bin

. /lib/lsb/init-functions

do_restart_eudev () {
	log_action_msg "Restarting eudev"
	service eudev stop
	service eudev start
}

case "$1" in
  start)
	do_restart_eudev
	;;
  restart|reload|force-reload)
	echo "Error: argument '$1' not supported" >&2
	exit 3
	;;
  stop|status)
	# No-op
	;;
  *)
	echo "Usage: $0 start|stop" >&2
	exit 3
	;;
esac

5) Run the following command:

update-rc.d restart-eudev defaults

6) Bonus points if you can do the following to prove the old work-around isn't needed anymore:

apt purge acpi-fakekey

7) Logout to proceed to runlevel 2.
Watch out for the following message:
[info] Restarting eudev

8) Test mouse, keyboard, sound, pc-speaker, etc.

#17 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-26 19:45:42

pcalvert wrote:

That just fixes the keyboard and mouse problem, right? Because I just tried it, and sound still isn't working.

For me this fixes everything, including the sound. All the way up to pc-speaker.
I am testing on the beowulf rc live-image

#18 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-25 17:30:01

pcalvert wrote:

That may help with some issues, but it doesn't look like it will help with the no-sound issue (for me, anyway).

Here's why:

# groupadd kvm
groupadd: group 'kvm' already exists

Phil

Have you done that on the iso that you've created, or on the beowulf rc live-image?

I tested on the beowulf rc live-image burned to dvd.

I boot into runlevel 1. Run those commands.
Eudev instantly detects a lot of things that it didn't detect before that. Including pc-speaker. Also my console font immediately changes to the correct one. Means video-device is properly detected. Doing 'service alsa-utils restart' after that I get the sound immediately.
After that I purge the acpi-fakekey, start the desktop and everything including mouse and kbd works.

Beowulf live-image on dvd, no 'toram', bare-metal.

#19 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-25 14:04:55

I think I've found the fix.
Eudev needs group "kvm" to be present.

Try booting the beowulf live image rc from dvd into runlevel 1, after that

service eudev stop
groupadd kvm
service eudev start

Seems to fix the issue with missing modules. You don't even need acpi-fakekey for the mouse after that.

Update: turns out the kvm group has no effect - but stopping and restarting do.

#20 Re: News & Announcements » Beowulf Beta is here! » 2020-05-24 18:24:08

Booting the beowulf live image rc into runlevel 1, after that

service eudev stop
groupadd kvm
service eudev start

seems to fix the issue with missing modules.

#21 Re: News & Announcements » Beowulf Beta is here! » 2020-05-24 12:17:22

Setting CRYPTSETUP=y will cause cryptsetup to be added to the initramfs if the root filesystem is NOT encrypted. (just tested this to be sure.) If root is encrypted, it'll be added without this setting. The wording in the comment for this setting suggests that in the future, it will be added if cryptsetup is installed, regardless of the root filesystem.

I stand corrected.

I do remember though that when getting the warning

cryptsetup: WARNING: The initramfs image may not contain cryptsetup binaries 
    nor crypto modules. If that's on purpose, you may want to uninstall the 
    'cryptsetup-initramfs' package in order to disable the cryptsetup initramfs 
    integration and avoid this warning.

I did get into trouble (unable to decrypt root filesystem) a few times because cryptsetup bins were not included. So I learned to pay attention to it if I need to decrypt something at the initramfs stage.

The preferrable way of including cryptsetup bins for me is adding 'initramfs' to options column of crypttab. Like this:

# <target name> <source device>         <key file>      <options>
target_device_name   UUID=<..uuid here..>   none  luks,initramfs

And you don't need to add CRYPTSETUP=y. This is for ascii and later.

I hope Lomax was able to boot successfully.

#22 Re: News & Announcements » Beowulf Beta is here! » 2020-05-24 02:17:58

Lomax wrote:

Excellent, then it is as I thought, and the warning is misleading.

The warning isn't misleading, because it says cryptsetup won't be included in the initramfs, and that's exactly what will happen.
But you simply don't need the cryptsetup binaries in the initramfs if your root isn't encrypted. The binaries will be accessible when the root filesystem is mounted.

Adding CRYPTSETUP=y to /usr/share/initramfs-tools/conf-hooks.d/cryptsetup isn't exactly including it manually. This just specifies that cryptsetup will be added automatically if the root filesystem itself is encrypted.

Hope it helps.

#23 Re: News & Announcements » Beowulf Beta is here! » 2020-05-24 01:45:34

Lomax wrote:

The upgrade seems to have gone well, but at the end of it I got a curious warning from update-initramfs:

If your root is unencypted you may boot but will have to unlock crypt partitions manually.

If you've only encrypted home and swap then you don't need cryptsetup in the initramfs and can ignore the warning. The root will be mounted and after that the cryptsetup binaries are accessible to the system in order to unlock the rest of partitions.

If you've described the situation accurately, you'll probably be able to boot and login as root. After that you can unlock the rest and add the required lines to crypttab/fstab.

#24 Re: Installation » Beowulf install » 2020-05-21 14:07:19

Does OP have an nvidia graphics card?

#25 Re: Devuan Derivatives » Can live isos be burned to optical media? » 2020-05-21 13:57:28

I myself don't often use optical images for installation in most cases.

My preferred method is to put isos on hard drive and create grub entries to boot into them. Writing the iso to a usb stick is as redundant as burning it to a dvd in most cases.

There are cases where it's not possible, or not convinient, to do it from hd and in those cases external media, dvd/usb, comes to the rescue.

My dvd drive is a very old ide drive. It's by no means "modern bd crap".
Jessie and ascii live/install images, burned to a dvd, always worked flawlessly for me. I also sometimes write an ubuntu lts version to a disc and it always worked without problem as well.

Also, fsmithred said that the problem appears on some usb configurations as well, it's not limited to optical media.

Board footer

Forum Software