The officially official Devuan Forum!

You are not logged in.

#1 2024-07-07 12:02:22

vazhnov
Member
From: Wrocław, Poland
Registered: 2020-05-31
Posts: 24  

OpenSSH fails to start with the latest update in Devuan 6 Excalibur

Hi!

I have a Proxmox host with KVM virtual machine with Devuan GNU/Linux 6 (excalibur/ceres) + OpenRC init.

Yesterday I've installed updates in the virtual machine guest and after reboot, SSHD couldn't start:

/var/log/syslog:

2024-07-07T01:00:42.824732+02:00 devuan6-kube01 /etc/init.d/ssh[1822]: start-stop-daemon: pam error: Authentication failure
2024-07-07T01:00:42.830420+02:00 devuan6-kube01 /etc/init.d/ssh[1821]: start-stop-daemon: failed to start `/usr/sbin/sshd'

Updates, from /var/log/apt/history.log:

Start-Date: 2024-07-07  00:59:42
Commandline: apt-get -V --no-install-recommends dist-upgrade
Requested-By: alexey (1000)
Upgrade: base-passwd:amd64 (3.6.3, 3.6.4), libeinfo1:amd64 (0.54-2, 0.54.2-1), gping:amd64 (1.16.1-1, 1.16.1-2), debian-keyring:amd64 (2024.03.24, 2024.06.24), librc1t64:amd64 (0.54-2, 0.54.2-1), openrc:amd64 (0.54-2, 0.54.2-1), ruby-xmlrpc:amd64 (0.3.2-2, 0.3.3-2)
End-Date: 2024-07-07  00:59:56

When I login through KVM console and start the service manually with sudo service ssh start, it starts.

I will investigate this. Just wanted to notify users to not install the latest updates if they have similar configuration.

Offline

#2 2024-07-07 12:19:06

vazhnov
Member
From: Wrocław, Poland
Registered: 2020-05-31
Posts: 24  

Re: OpenSSH fails to start with the latest update in Devuan 6 Excalibur

Also I found that after the upgrade, Devuan fails to stop some services at shutdown:

2024-07-07T01:00:07.694393+02:00 devuan6-kube01 shutdown[2776]: shutting down for system reboot
2024-07-07T01:00:07.802340+02:00 devuan6-kube01 init: Switching to runlevel: 6
2024-07-07T01:00:07.803922+02:00 devuan6-kube01 kernel: Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
2024-07-07T01:00:07.810263+02:00 devuan6-kube01 init: No inittab.d directory found
2024-07-07T01:00:07.916679+02:00 devuan6-kube01 start-stop-daemon: Will stop /usr/sbin/sshd
2024-07-07T01:00:07.916702+02:00 devuan6-kube01 start-stop-daemon: Will stop PID 1833
2024-07-07T01:00:07.917112+02:00 devuan6-kube01 start-stop-daemon: Sending signal 15 to PID 1833
2024-07-07T01:00:07.950597+02:00 devuan6-kube01 /etc/init.d/rsync[2806]: ERROR: rsync failed to stop
2024-07-07T01:00:08.032347+02:00 devuan6-kube01 start-stop-daemon: Will stop PID 1765
2024-07-07T01:00:08.032929+02:00 devuan6-kube01 start-stop-daemon: Sending signal 15 to PID 1765
2024-07-07T01:00:08.216425+02:00 devuan6-kube01 avahi-daemon[1643]: Got SIGTERM, quitting.
2024-07-07T01:00:08.217919+02:00 devuan6-kube01 avahi-daemon[1643]: Leaving mDNS multicast group on interface eth0.IPv6 with address 2a01:112f:xxx:yyy:zzz:xxx:yyy:zzz.
2024-07-07T01:00:08.218882+02:00 devuan6-kube01 avahi-daemon[1643]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.168.155.137.
2024-07-07T01:00:08.219253+02:00 devuan6-kube01 avahi-daemon[1643]: Leaving mDNS multicast group on interface lo.IPv6 with address ::1.
2024-07-07T01:00:08.219426+02:00 devuan6-kube01 avahi-daemon[1643]: Leaving mDNS multicast group on interface lo.IPv4 with address 127.0.0.1.
2024-07-07T01:00:08.236026+02:00 devuan6-kube01 avahi-daemon[1643]: avahi-daemon 0.8 exiting.
2024-07-07T01:00:08.364213+02:00 devuan6-kube01 start-stop-daemon: Will stop PID 1625
2024-07-07T01:00:08.364234+02:00 devuan6-kube01 start-stop-daemon: Will stop processes owned by UID 100
2024-07-07T01:00:08.364783+02:00 devuan6-kube01 start-stop-daemon: Sending signal 15 to PID 1625
2024-07-07T01:00:08.387275+02:00 devuan6-kube01 /etc/init.d/dbus[2928]: ERROR: dbus failed to stop

Last edited by vazhnov (2024-07-07 12:19:31)

Offline

#3 2024-07-08 13:35:28

fsmithred
Administrator
Registered: 2016-11-25
Posts: 2,458  

Re: OpenSSH fails to start with the latest update in Devuan 6 Excalibur

I'm not seeing this problem before or after upgrade today of excalibur amd64 on hardware with sysvinit. SSH login from remote works, rsync and dbus stop/start normally as do other services.

Offline

#4 2024-07-08 22:34:16

vazhnov
Member
From: Wrocław, Poland
Registered: 2020-05-31
Posts: 24  

Re: OpenSSH fails to start with the latest update in Devuan 6 Excalibur

fsmithred, thanks. This is OpenRC-related issue, probably connected with an upgrade openrc:amd64 (0.54-2, 0.54.2-1).

I've added rc_verbose=yes into /etc/rc.conf and now I see in /var/log/boot:

Tue Jul  9 00:07:18 2024: Starting system message bus: dbus * start-stop-daemon: fopen `/var/run/dbus/pid': No such file or directory
Tue Jul  9 00:07:18 2024:  * Detaching to start `/usr/bin/dbus-daemon' ...
Tue Jul  9 00:07:18 2024: .
Tue Jul  9 00:07:18 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/avahi-daemon start
Tue Jul  9 00:07:18 2024: Starting Avahi mDNS/DNS-SD Daemon: avahi-daemon.
Tue Jul  9 00:07:18 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/bootlogs start
Tue Jul  9 00:07:19 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/console-setup.sh start
Tue Jul  9 00:07:19 2024: Setting up console font and keymap...done.
Tue Jul  9 00:07:19 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/cron start
Tue Jul  9 00:07:19 2024: Starting periodic command scheduler: cron.
Tue Jul  9 00:07:19 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/earlyoom start
Tue Jul  9 00:07:19 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/elogind start
Tue Jul  9 00:07:19 2024: Starting session management daemon: elogind * start-stop-daemon: fopen `/run/elogind.pid': No such file or directory
Tue Jul  9 00:07:19 2024:  * Detaching to start `/usr/libexec/elogind' ...
Tue Jul  9 00:07:19 2024: .
Tue Jul  9 00:07:19 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/sudo start
Tue Jul  9 00:07:19 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/rmnologin start
Tue Jul  9 00:07:19 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/rsync start
Tue Jul  9 00:07:19 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/ssh start
Tue Jul  9 00:07:19 2024: Starting OpenBSD Secure Shell server: sshd * start-stop-daemon: fopen `/run/sshd.pid': No such file or directory
Tue Jul  9 00:07:19 2024:  * Detaching to start `/usr/sbin/sshd' ...
Tue Jul  9 00:07:19 2024:  *   start-stop-daemon: pam error: Authentication failure
Tue Jul  9 00:07:20 2024:  *   start-stop-daemon: failed to start `/usr/sbin/sshd'
Tue Jul  9 00:07:20 2024:  failed!
Tue Jul  9 00:07:20 2024:  * ERROR: rc.local needs service(s) 0, 6
Tue Jul  9 00:07:20 2024:  * Executing: /usr/lib/rc/sh/openrc-run.sh /usr/lib/rc/sh/openrc-run.sh /etc/init.d/stop-bootlogd start

I don't have custom records in /etc/fstab to mount /run

Analysing git diff debian/0.54-2 debian/0.54.2-1 in https://salsa.debian.org/debian/openrc (web diff).

Offline

#5 2024-07-10 22:03:51

vazhnov
Member
From: Wrocław, Poland
Registered: 2020-05-31
Posts: 24  

Re: OpenSSH fails to start with the latest update in Devuan 6 Excalibur

Currently there are two versions of start-stop-daemon binaries from two packages:

 % dpkg -S start-stop-daemon|sort                      
dpkg: /usr/sbin/start-stop-daemon
dpkg: /usr/share/man/de/man8/start-stop-daemon.8.gz
dpkg: /usr/share/man/fr/man8/start-stop-daemon.8.gz
dpkg: /usr/share/man/man8/start-stop-daemon.8.gz
dpkg: /usr/share/man/nl/man8/start-stop-daemon.8.gz
dpkg: /usr/share/man/pt/man8/start-stop-daemon.8.gz
dpkg: /usr/share/man/sv/man8/start-stop-daemon.8.gz
openrc: /usr/lib/rc/sbin/start-stop-daemon
openrc: /usr/lib/rc/sh/start-stop-daemon.sh

They are different:

 % file /usr/sbin/start-stop-daemon /usr/lib/rc/sbin/start-stop-daemon
/usr/sbin/start-stop-daemon:        ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=fb0de15cbe08c53aa308fddb2925afe3423ed34e, for GNU/Linux 3.2.0, stripped
/usr/lib/rc/sbin/start-stop-daemon: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=6b69feea4068615a048ce10c763efb5869f798f8, for GNU/Linux 3.2.0, stripped

As an experiment, I decided to make OpenRC version of start-stop-daemon as the main one:

sudo cp -pv /usr/sbin/start-stop-daemon /usr/sbin/start-stop-daemon.dpkg
sudo cp -pv /usr/lib/rc/sbin/start-stop-daemon /usr/sbin/start-stop-daemon

After this change, I see no difference in Devuan shutdown/start (the same errors and SSHD can't start), but this time I couldn't start SSHD manually with sudo /etc/init.d/ssh start, I received the same error "start-stop-daemon: pam error: Authentication failure". Before the binaries swap, I was able to start the SSHD manually.

Then I returned binaries back and made second experiment:

sudo cp -pv /usr/sbin/start-stop-daemon.dpkg /usr/lib/rc/sbin/start-stop-daemon

And now my system starts and stops normally, SSHD starts automatically!

Offline

Board footer