Shutdown encrypted LVM on Beowulf

rolfie · 2019-02-19 21:01:42

Looks like the delays as described in https://dev1galaxy.org/viewtopic.php?pid=7675#p7675 are back in Beowulf. The patch described in https://dev1galaxy.org/viewtopic.php?pid=11522#p11522 which works fine with ASCII does no more apply, the file cryptdisks.functions is no more present in Beowulf.

Looked through the scripts in /lib/cryptsetup/, but could not identify where to patch what.

Any proposals how to attack the issue in Beowulf?

Thanks, Rolf

fsmithred · 2019-02-20 13:32:59

Looks like the file name was changed to /lib/cryptsetup/cryptdisks-functions. (there's a dash instead of a dot)

I have it on two beowulf installs - one was upgrade from ascii, one was from mini.iso install.

I don't know if that fix still works (i.e. copying part of the jessie version into the ascii file.) You could try the earlier fix of changing the timings on line 190 to 'for i in 1'. I haven't tested this yet, because I have't been able to boot into my encrypted lvm install.

# Removes all mappings in crypttab
181 do_stop() {
182     dmsetup mknodes
183     log_action_begin_msg "Stopping $INITSTATE crypto disks"
184
185     crypttab_foreach_entry _do_stop_callback
186     log_action_end_msg 0
187 }
188 _do_stop_callback() {
189     local i rv=0
190     for i in 1 2 4 8 16 32; do
191         remove_mapping "$CRYPTTAB_NAME" 3<&- && break || rv=$?
192         if [ $rv -eq 1 ] || [ $rv -eq 2 -a $i -gt 16 ]; then
193             log_action_end_msg $rv
194             break
195         fi
196         log_action_cont_msg "$CRYPTTAB_NAME busy..."
197         sleep $i

xinomilo · 2019-02-20 13:41:13

having the same problem since installing devuan... bug was also present in debian without-systemd, but not in MX linux (they have packaged a patched version).

there is a bug #271 for cryptsetup / delays in shutdown.
https://bugs.devuan.org//cgi/bugreport.cgi?bug=271
as mentioned, new file in beowulf is /lib/cryptsetup/cryptdisks-functions, and no ascii patch applies there now.

rolfie · 2019-02-21 21:00:25

Tried to find the line

local dst src key opts opencount major minor

in the new file as a hook or any hint for the patch that shuts down the LVM before the luks container, and failed.

I was able to apply the patch for ASCII, the patch and the description was sufficient for me. But for the new situation in Beowulf I do not understand enough about scripting and scripts to understand in detail what is happening there and how to possibly attack the problem.

To clarify: I got an LVM inside the luks container.

My impression: the new cryptdisks-functions just looks at encrypted volumes and does not take into account that a LVM may be included.

Thanks, Rolf

Remark: I use the classical setup with an un-encrypted boot partition, a system partition which is encrypted, and on top of that a LVM with at least two volumes for root and swap.

Last edited by rolfie (2019-02-22 19:03:59)

fsmithred · 2019-02-22 00:21:09

Something has to happen here. I'm not sure what. I tried changing line 164 to just 'for i in 1 ; do' but I don't think it really worked. Only tried it once before I reinstalled over it. I've done a bunch of installs today, and it looks like grub can't do full-disk encryption or encrypted lvm. I need to run the tests again, because I just got a new version of grub. If I can get an encrypted lvm install to boot, I'll play with this some more.

   156  do_stop() {
   157      dmsetup mknodes
   158      log_action_begin_msg "Stopping $INITSTATE crypto disks"
       
   159      crypttab_foreach_entry _do_stop_callback
   160      log_action_end_msg 0
   161  }
   162  _do_stop_callback() {
   163      local i rv=0
   164      for i in 1 2 4 8 16 32; do
   165          remove_mapping "$CRYPTTAB_NAME" 3<&- && break || rv=$?
   166          if [ $rv -eq 1 ] || [ $rv -eq 2 -a $i -gt 16 ]; then
   167              log_action_end_msg $rv
   168              break
   169          fi
   170          log_action_cont_msg "$CRYPTTAB_NAME busy..."
   171          sleep $i
   172      done
   173  }

rolfie · 2019-02-22 19:05:30

I have tried with 1 2 3 4 5 6, the messages seem to come a bit faster, the general appearance still is the same.

fsmithred · 2019-02-23 22:58:31

Forget about the timeouts. The old patch still works. Here's what my do_stop function looks like now. I added the lines between 'dmsetup mknodes' and 'log_action_begin_message...' I did not add the local variable declarations. Those are no longer in this function. Maybe I should.

do_stop() {
    dmsetup mknodes
    if [ -x /sbin/lvm ]; then
        vgs="$(/sbin/lvm vgscan | sed -n '/"/s/^.*"\([^'\'']*\)".*$/\1/p')"
        if [ -n "${vgs}" ]; then
          for vg in ${vgs}; do
          /sbin/lvm vgchange -a n ${vg} >/dev/null 2>&1 
         done 
        fi
    fi
    log_action_begin_msg "Stopping $INITSTATE crypto disks"

    crypttab_foreach_entry _do_stop_callback
    log_action_end_msg 0
}

xinomilo · 2019-02-24 12:51:22

added these lines and there are no cryptdisks delays, system shutdown is almost instant, but cryptdisks stop fails for some reason. too quick to view exactly on shutdown - just a glimpse of "cryptdisks: failed".
so, not exactly sure if that works.

fsmithred · 2019-02-24 14:20:17

It goes by too fast for me to see what it says. There are only about four lines of output, and there's no red. I did add 'local vgs vg' to the beginning of do_stop() but I don't know if that would make a difference.

xinomilo · 2019-02-24 16:33:24

no it doesn't, unfortunately.. shutdown is very fast, but cryptdisk & cryptdisk-early fail (red). no other messages there..
maybe someone else can try this too?

diff from default :

diff cryptdisks-functions.orig cryptdisks-functions
181a182,183
>     local vgs vg
> 
182a185,192
>     if [ -x /sbin/lvm ]; then
>         vgs="$(/sbin/lvm vgscan | sed -n '/"/s/^.*"\([^'\'']*\)".*$/\1/p')"
>         if [ -n "${vgs}" ]; then
>           for vg in ${vgs}; do
>           /sbin/lvm vgchange -a n ${vg} >/dev/null 2>&1 
>          done 
>         fi
>     fi

chris2be8 · 2019-02-24 20:05:10

Pressing control-S just after the messages come out might enable you to read them (then control-Q to continue).

If that fails could you take a video of the console during shutdown, then play it back slowly?

Chris

xinomilo · 2019-02-25 00:59:43

ctrl+S doesn't work, not sure why. here's a blurry screeshot (sorry about quality):
https://share.riseup.net/#3XVZ8rGqhJ1uyPczRhVcXg

rolfie · 2019-02-25 20:11:10

Would be easier if I could paste in a screenshot. I don't like these external file sharing hosters.

With the local declaration, I get the following display:

[ ok ] Unmounting local filesystems ... done
* ERROR: cryptdisks failed to stop
* ERROR: cryptdisks-early failed to stop

The shutdown is fast as expected now.

~~Without the declaration there are additional messages.~~
When checking my secondary PC again versus the encrypted VM on my main PC, it now looks like the messages I see are related to missing devices in a USB multi-card-reader. They also appear during the boot in an early phase.

The messages look like this:

  /dev/sdc: open failed: No medium found
  /dev/sdd: open failed: No medium found
  /dev/sde: open failed: No medium found
  /dev/sdf: open failed: No medium found
  /dev/sdg: open failed: No medium found

Rolf

Last edited by rolfie (2019-02-25 21:05:45)

rolfie · 2019-02-25 21:13:16

Just run an apt update/upgrade, that brought about 180 updates, that killed the patch again. Just redid it.

Rolf

rolfie · 2019-02-26 20:37:52

Are the cryptdisk erros to be expected?

Thanks, Rolf

kuleszdl · 2019-05-29 22:43:57

I tried the proposed workaround on a freshly installed beowulf system. Unfortunately, it seems to make no difference. Anyone tried this recently?

Last edited by kuleszdl (2019-05-29 22:51:28)

rolfie · 2019-05-30 06:44:57

kuleszdl wrote:

I tried the proposed workaround on a freshly installed beowulf system. Unfortunately, it seems to make no difference. Anyone tried this recently?

Works fine on my Beowulf x64 system. Look at entry #7, filename to patch is /lib/cryptsetup/cryptdisks-functions. Shutdown is really fast.

Rolf

kuleszdl · 2019-05-30 10:40:17

I have the exact same patch. I assume the problem is that I don't use LVM but just "plain" encrypted disks.

fsmithred · 2019-05-30 11:14:51

For plain encrypted filesystems without lvm, you can do the fix that changes the timeout.

Edit this function in /lib/cryptsetup/cryptdisks-functions. (And keep a spare copy of this file those times when an update wipes out your edits. )

_do_stop_callback() {
    local i rv=0
    for i in 1 2 4 8 16 32; do
        remove_mapping "$CRYPTTAB_NAME" 3<&- && break || rv=$?
        if [ $rv -eq 1 ] || [ $rv -eq 2 -a $i -gt 16 ]; then
            log_action_end_msg $rv
            break
        fi
        log_action_cont_msg "$CRYPTTAB_NAME busy..."
        sleep $i
    done

To look more like this. You can make it 'for i in 1' instead of 'for i in 1 2' if you want.

_do_stop_callback() {
    local i rv=0
    for i in 1 2 ; do
        remove_mapping "$CRYPTTAB_NAME" 3<&- && break || rv=$?
        if [ $rv -eq 1 ] || [ $rv -eq 2 -a $i -gt 16 ]; then
            log_action_end_msg $rv
            break
        fi
        log_action_cont_msg "$CRYPTTAB_NAME busy..."
        sleep $i
    done

xinomilo · 2019-07-31 22:33:05

so, i've installed cryptsetup-modified-functions from antix testing repo (no other deps) and system poweroff is fast as expected (deb link: http://ftp.cc.uoc.gr/mirrors/linux/mx/a … functions/ ).
there's still a failed message on shutdown, so maybe antix is using a similar solution.. still IMO, a better way to deal with this issue (#271 in devuan), than "random patches" that will be ovewritten in next upgrade. (this is testing/ceres afterall..)

fsmithred · 2019-08-01 02:00:28

Yes, it's the same solution. Here's the diff against the same file in ascii. The advantage of the mx/antix package is that it uses dpkg-divert to keep the file from being clobbered on upgrades. Do they have a version for buster?

diff -u /usr/lib/cryptsetup/cryptdisks.functions  cryptdisks-functions 
--- /usr/lib/cryptsetup/cryptdisks.functions	2017-05-09 11:50:59.000000000 +0000
+++ cryptdisks-functions	2018-11-30 03:02:16.000000000 +0000
@@ -769,7 +769,7 @@
 	log_action_begin_msg "Stopping $INITSTATE crypto disks"
 
 	egrep -v "^[[:space:]]*(#|$)" "$TABFILE" | while read dst src key opts; do
-		for i in 1 2 4 8 16 32; do
+		for i in 1 ; do
 			handle_crypttab_line_stop "$dst" "$src" "$key" "$opts" <&3 && break || ret=$?
 			if [ $ret -eq 1 ] || [ $ret -eq 2 -a $i -gt 16 ]; then
 				log_action_end_msg $ret

xinomilo · 2019-08-01 08:56:36

fsmithred wrote:

Do they have a version for buster?

from what i've seen, it's the same version for all dists (stable/testing/unstable). so yes, it should work on buster/beowulf too, but not stretch/ascii.

fsmithred · 2019-08-01 22:45:26

The fix appears to be the same, but the file is not. The last diff I posted was between the copy
of cryptdisks-functions in the mx package, which is named same as the original file in beowulf, against the cryptdisks.functions file from ascii. I didn't notice the different filenames when I ran the diff because I used tab-completion.

The diff between the mx version of the file and the original file in beowulf, looks a lot different. I haven't examined it closely, but it's obviously not the same file. It's too big to copy the whole thing from a terminal, so I pasted it here:
https://termbin.com/xle7
This is really the diff between cryptdisks-functions in beowulf and cryptdisks.functions+patch in ascii
Unless antix or mx is still using cryptsetup 1.7, they should update their package to use the file from cryptsetup 2.1. for buster.

They also didn't include the fix for lvm. Here's what I changed in beowulf:

diff -u cryptdisks-functions.orig-2.1.0-5  cryptdisks-functions.patched 
--- cryptdisks-functions.orig-2.1.0-5	2019-08-01 22:24:09.400000000 +0000
+++ cryptdisks-functions.patched	2019-07-19 02:03:11.972000000 +0000
@@ -179,7 +179,18 @@
 
 # Removes all mappings in crypttab
 do_stop() {
+
+    local vgs vg
+
     dmsetup mknodes
+       if [ -x /sbin/lvm ]; then
+        vgs="$(/sbin/lvm vgscan | sed -n '/"/s/^.*"\([^'\'']*\)".*$/\1/p')"
+         if [ -n "${vgs}" ]; then
+          for vg in ${vgs}; do
+           /sbin/lvm vgchange -a n ${vg} >/dev/null 2>&1 
+          done 
+         fi
+        fi
     log_action_begin_msg "Stopping $INITSTATE crypto disks"
 
     crypttab_foreach_entry _do_stop_callback
@@ -187,7 +198,7 @@
 }
 _do_stop_callback() {
     local i rv=0
-    for i in 1 2 4 8 16 32; do
+    for i in 1 2 ; do
         remove_mapping "$CRYPTTAB_NAME" 3<&- && break || rv=$?
         if [ $rv -eq 1 ] || [ $rv -eq 2 -a $i -gt 16 ]; then
             log_action_end_msg $rv

rolfie · 2019-08-05 19:36:49

Any explanation why the LVM isn't considered?

Vizitor · 2019-08-18 07:39:00

Maybe this is impropriety comment, but no pun intended:
Recently, when Debian 10 was released, I tested it with (simple, not-LVM) Full Disk Encription, and it shutdown immediately; so, after all this years, they finally fixed this bug.

Maybe somebody who know how to do it, can see there how it is done.
I am just at user-level, so I can't do it myself.

The officially official Devuan Forum!

#1 2019-02-19 21:01:42

Shutdown encrypted LVM on Beowulf

#2 2019-02-20 13:32:59

Re: Shutdown encrypted LVM on Beowulf

#3 2019-02-20 13:41:13

Re: Shutdown encrypted LVM on Beowulf

#4 2019-02-21 21:00:25

Re: Shutdown encrypted LVM on Beowulf

#5 2019-02-22 00:21:09

Re: Shutdown encrypted LVM on Beowulf

#6 2019-02-22 19:05:30

Re: Shutdown encrypted LVM on Beowulf

#7 2019-02-23 22:58:31

Re: Shutdown encrypted LVM on Beowulf

#8 2019-02-24 12:51:22

Re: Shutdown encrypted LVM on Beowulf

#9 2019-02-24 14:20:17

Re: Shutdown encrypted LVM on Beowulf

#10 2019-02-24 16:33:24

Re: Shutdown encrypted LVM on Beowulf

#11 2019-02-24 20:05:10

Re: Shutdown encrypted LVM on Beowulf

#12 2019-02-25 00:59:43

Re: Shutdown encrypted LVM on Beowulf

#13 2019-02-25 20:11:10

Re: Shutdown encrypted LVM on Beowulf

#14 2019-02-25 21:13:16

Re: Shutdown encrypted LVM on Beowulf

#15 2019-02-26 20:37:52

Re: Shutdown encrypted LVM on Beowulf

#16 2019-05-29 22:43:57

Re: Shutdown encrypted LVM on Beowulf

#17 2019-05-30 06:44:57

Re: Shutdown encrypted LVM on Beowulf

#18 2019-05-30 10:40:17

Re: Shutdown encrypted LVM on Beowulf

#19 2019-05-30 11:14:51

Re: Shutdown encrypted LVM on Beowulf

#20 2019-07-31 22:33:05

Re: Shutdown encrypted LVM on Beowulf

#21 2019-08-01 02:00:28

Re: Shutdown encrypted LVM on Beowulf

#22 2019-08-01 08:56:36

Re: Shutdown encrypted LVM on Beowulf

#23 2019-08-01 22:45:26

Re: Shutdown encrypted LVM on Beowulf

#24 2019-08-05 19:36:49

Re: Shutdown encrypted LVM on Beowulf

#25 2019-08-18 07:39:00

Re: Shutdown encrypted LVM on Beowulf

Board footer