mikas blog

Update on 2023-03-23: thanks to Daniel Roschka for mentioning the Mandos and TPM approaches, which might be better alternatives, depending on your options and needs. Peter Palfrader furthermore pointed me towards clevis-initramfs and tang.

A customer of mine runs dedicated servers inside a foreign data-center, remote hands only. In such an environment you might need a disk replacement because you need bigger or faster disks, though also a disk might (start to) fail and you need a replacement. One has to be prepared for such a scenario, but fully wiping your used disk then might not always be an option, especially once disks (start to) fail. On the other hand you don’t want to end up with (partial) data on your disk handed over to someone unexpected.

By encrypting the data on your disks upfront you can prevent against this scenario. But if you have a fleet of servers you might not want to manually jump on servers during boot and unlock crypto volumes manually. It’s especially annoying if it’s about the root filesystem where a solution like dropbear-initramfs needs to be used for remote access during initramfs boot stage. So my task for the customer was to adjust encrypted LUKS devices such that no one needs to manually unlock the encrypted device during server boot (with some specific assumptions about possible attack vectors one has to live with, see the disclaimer at the end).

The documentation about this use-case was rather inconsistent, especially because special rules apply for the root filesystem (no key file usage), we see different behavior between what’s supported by systemd (hello key file again), initramfs-tools and dracut, not to mention the changes between different distributions. Since tests with this tend to be rather annoying (better make sure to have a Grml live system available :)), I’m hereby documenting what worked for us (Debian/bullseye with initramfs-tools and cryptsetup-initramfs).

The system was installed with LVM on-top of an encrypted Software-RAID device, only the /boot partition is unencrypted. But even if you don’t use Software-RAID nor LVM the same instructions apply. The system looks like this:

% mount -t ext4 -l
/dev/mapper/foobar-root_1 on / type ext4 (rw,relatime,errors=remount-ro)

% sudo pvs
  PV                    VG     Fmt  Attr PSize   PFree
  /dev/mapper/md1_crypt foobar lvm2 a--  445.95g 430.12g

% sudo vgs
  VG     #PV #LV #SN Attr   VSize   VFree
  foobar   1   2   0 wz--n- 445.95g 430.12g

% sudo lvs
  LV     VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root_1 foobar -wi-ao---- <14.90g

% lsblk
NAME                  MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
[...]
sdd                     8:48   0 447.1G  0 disk
├─sdd1                  8:49   0   571M  0 part  /boot/efi
├─sdd2                  8:50   0   488M  0 part
│ └─md0                 9:0    0   487M  0 raid1 /boot
└─sdd3                  8:51   0 446.1G  0 part
  └─md1                 9:1    0   446G  0 raid1
    └─md1_crypt       253:0    0   446G  0 crypt
      ├─foobar-root_1 253:1    0  14.9G  0 lvm   /
[...]
sdf                     8:80   0 447.1G  0 disk
├─sdf1                  8:81   0   571M  0 part
├─sdf2                  8:82   0   488M  0 part
│ └─md0                 9:0    0   487M  0 raid1 /boot
└─sdf3                  8:83   0 446.1G  0 part
  └─md1                 9:1    0   446G  0 raid1
    └─md1_crypt       253:0    0   446G  0 crypt
      ├─foobar-root_1 253:1    0  14.9G  0 lvm   /

The actual crypsetup configuration is:

% cat /etc/crypttab
md1_crypt UUID=77246138-b666-4151-b01c-5a12db54b28b none luks,discard

Now, to automatically open the crypto device during boot we can instead use:

% cat /etc/crypttab 
md1_crypt UUID=77246138-b666-4151-b01c-5a12db54b28b none luks,discard,keyscript=/etc/initramfs-tools/unlock.sh

# touch /etc/initramfs-tools/unlock.sh
# chmod 0700 /etc/initramfs-tools/unlock.sh
# $EDITOR etc/initramfs-tools/unlock.sh
# cat /etc/initramfs-tools/unlock.sh
#!/bin/sh
echo -n "provide_the_actual_password_here"

# update-initramfs -k all -u
[...]

The server will then boot without prompting for a crypto password.

Note that initramfs-tools by default uses an insecure umask of 0022, resulting in the initrd being accessible to everyone. But if you have the dropbear-initramfs package installed, its `/usr/share/initramfs-tools/conf-hooks.d/dropbear` sets `UMASK=0077`, so the resulting /boot/initrd* file should automatically have proper permissions (0600). The cryptsetup hook warns about a permissive umask configuration during update-initramfs runs, but if you want to be sure, explicitly set it via e.g.:

# cat > /etc/initramfs-tools/conf.d/umask << EOF
# restrictive umask to avoid non-root access to initrd:
UMASK=0077
EOF
# update-initramfs -k all -u

Disclaimer: Of course you need to trust users with access to /etc/initramfs-tools/unlock.sh as well as the initramfs/initrd on your system. Furthermore you should wipe the boot partition (to destroy the keyfile information) before handing over such a disk. But that is a risk my customer can live with, YMMV.

This entry was posted by mika on Wednesday, March 22nd, 2023 at 14:30 and is filed under Computer, Debian, English. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.