Repository navigation
Conversation
59a2544 to
97bf779
Compare
|
Would there be any instructions, how this can be tested? I'd like to aid in this implementation, but am unsure how. Is there something that can be done by third-parties? |
I've packaged my changes for Ubuntu in a PPA. After you install that, it's essentially just using
Testing would be good. But I only have this specific initramfs configuration. |
|
Thank you for the quick reply. I have found the PPA. Unfortunately I have never used Clevis and cannot unpack the sentence
to infer what a test setup would look like. I'm able to set up an LXD/Incus virtual network that contains an Ubuntu VM/system container and a block devices that contains an encrypted ZFS pool plus a separate instance for Clevis. As a tester I would ask myself:
I am having the intuition that Would it seem suitable to add some documentation with this cycle? The information could go to the |
Ah, OK. Have a look at the Arch wiki. In my case, I want to bind the key with both my TPM and with a Yubikey, so I used
If you want a server, I think you're referring to Tang. I haven't set that up; my key unlock is purely held on the TPM and with a Yubikey, there's no network involved in my setup.
Essentially Clevis does most of the heavy lifting. All this PR does is teach Clevis how to unlock ZFS datasets, which is why there isn't significant documentation. The two arguments:
Calling |
|
I'm on it, but give a few months. Thanks for the link and for helping me understand the relationship between Tang and Clevis better. |
fc9e6a6 to
4317bd7
Compare
|
this would be a huge help for our organization 👍 |
|
@lowjoel any help you need to wrap this pr up ? |
|
More testing and code review is good. I've been using it for half a year now, but works well for me |
|
@lowjoel As you are requesting more testing I will be playing around with this shortly. Not an Ubuntu user but Debian. My plain is PVE on encrypted ZFS mirror. I have taken a look at your |
|
@deatharse that looks about right! The changes here are written in shell so you probably could get away with just installing your distro clevis, and then just manually install using dpkg |
|
Thanks, I will let you know how I get on as it would be really nice to get this merged.
Not so as
Which isnt a problem as the Debian version hasn't been updated to 21 yet anyway. |
|
@lowjoel So I've had a play. Much like my earlier suggested plan I:
Now I'm not sure if the following is due to PVE or not but upon reboot i recieved:
Okay, so I configured initramfs by adding a file Updated initramfs again and rebooted. Same thing occurred, however after entering the passphrase I saw the network configured output so it seemed like it was processing out-of-order. I decided to install and configure
Configured dropbear, updated initramfs again, and rebooted. Success, unlocked using the tang server. I know its advisable but I do not remember needing the additional steps with LUKS, but was likely just Debian. I will have a further play later by setting up Debian with ZFS on root. I am not an "official reviewer" for the project but one thing i would suggest is in keeping consistency with I hope this is helpful. |
|
@lowjoel Another consistency point @sarroutbi or @sergio-correia would probably pick up on as a nit-pick is the needless use of the |
|
@lowjoel one bug? I have noticed is if you pass the
it seems counter intuitive to me that this flag would cause it to fail in this circumstance. |
|
From further investigation the need for properly configured
which from the file 155 if [ $netcfg_attempted -eq 0 ] && has_tang_pin ${CRYPTTAB_SOURCE}; then
156 netcfg_attempted=1
157 do_configure_networking
158 fiwhere there is obvious LUKS dependencies 108 has_tang_pin() {
109 local dev="$1"
110
111 clevis luks list -d "${dev}" | grep -q tang
112 }Replicating both of these files and modifying the bottom of - clevisloop &
- echo $! >/run/clevis.pid
+ do_configure_networkingAllows it to work as expected after updating initramfs. It might be worthwhile abstracting the networking functions into a |
|
So I've found an issue with unbinding. If you do not use # clevis zfs bind -d rpool -l boot01 tang '{"url": "http://192.168.1.2/"}'
# clevis zfs bind -d rpool -l boot02 tang '{"url": "http://192.168.1.3/"}'You end up with 3 labels in the form: # zfs get all rpool | tail -3
rpool latchset.clevis.label:boot01 [tang server 1 data] local
rpool latchset.clevis:labels boot01 boot02 local
rpool latchset.clevis.label:boot02 [tang server 2 data] local you can then unbind them in the reverse order: # clevis zfs unbind -l boot02 -d rpool
Loading existing key...
Enter existing ZFS password for rpool:
Wiping Clevis data... okunbinds # clevis zfs unbind -l boot01 -d rpool
Loading existing key...
Enter existing ZFS password for rpool:
Wiping Clevis data... okunbinds however if you unbind in sequential order # clevis zfs unbind -l boot01 -d rpool
Loading existing key...
Enter existing ZFS password for rpool:
Wiping Clevis data... okremoves the label and therefor attempting # clevis zfs unbind -l boot02 -d rpool
ERROR: dataset is not bound with Clevis: rpool
Usage: clevis zfs unbind [-f] [-k KEY] -d DATASET [-a] -l LABEL
Unbinds a label from a ZFS dataset:
-f Force unbinding dataset
-d DATASET The ZFS dataset on which to perform unbinding
-a Unbind all labels
-l LABEL The label to unbind
-k KEY Non-interactively read ZFS password from KEY file
-k - Non-interactively read ZFS password from standard inputthe # zfs inherit latchset.clevis.label:boot02 rpoolif having both bound you attempt to unbind all: # clevis zfs unbind -a -d rpoolremoves the label for |
|
Thanks for this @deatharse -- you've given lots of good observations. Let me find a weekend to address them. |
|
I've had a bit of a further play by creating a second encrypted pool and noticed that was not unlocked. I notice The diff for 22c22,41
< clevis zfs unlock -d "${ENCRYPTIONROOT}"
---
> zpool import -a
>
> for pool in $(zpool list -H -o name); do
> # if pool encryption is active and the zfs command understands '-o encryption'
> if [ "$(zpool list -H -o feature@encryption ${pool})" = 'active' ]; then
> # if the root dataset has encryption enabled
> ENCRYPTIONROOT=$(zfs get -H -o value encryptionroot "${pool}")
> if ! [ "${ENCRYPTIONROOT}" = "-" ]; then
> KEYSTATUS="$(zfs get -H -o value keystatus "${ENCRYPTIONROOT}")"
> # continue only if the key needs to be loaded
> [ "$KEYSTATUS" = "unavailable" ] || continue
> # decrypt them
> TRY_COUNT=5
> while [ $TRY_COUNT -gt 0 ]; do
> clevis zfs unlock -d "${ENCRYPTIONROOT}" && break
> TRY_COUNT=$((TRY_COUNT - 1))
> done
> fi
> fi
> doneN.B. I have yet to create an unencrypted pool with an encrypted dataset, maybe I will get round to that on a weekend. |
|
Heres an updated diff for 22c22,58
< clevis zfs unlock -d "${ENCRYPTIONROOT}"
---
> attempt_unlock() {
> local dataset=$1
>
> KEYSTATUS=$(zfs get -H -o value keystatus "${dataset}")
> # continue only if the key needs to be loaded
> [ "$KEYSTATUS" = "unavailable" ] || break
> # decrypt them
> TRY_COUNT=5
> while [ $TRY_COUNT -gt 0 ]; do
> clevis zfs unlock -d "${dataset}" && break
> TRY_COUNT=$((TRY_COUNT - 1))
> done
> }
>
> zpool import -a
> for pool in $(zpool list -H -o name); do
> # if pool encryption is active and the zfs command understands '-o encryption'
> if [ $(zpool list -H -o feature@encryption "${pool}") = 'active' ]; then
> # if the root dataset has encryption enabled
> ENCRYPTIONROOT=$(zfs get -H -o value encryptionroot "${pool}")
> if ! [ "${ENCRYPTIONROOT}" = "-" ]; then
> attempt_unlock "${ENCRYPTIONROOT}"
> else
> # encryption in child dataset, lets get list of datasets in pool
> for dataset in $( zfs list -r -H -o name "${pool}" ); do
> # first entry will not be a child so we will ignore it
> if [ "${dataset}" != "${pool}" ]; then
> # test for encrypted dataset
> ENCRYPTIONROOT=$(zfs get -H -o value encryptionroot "${dataset}")
> if ! [ "${ENCRYPTIONROOT}" = "-" ]; then
> attempt_unlock "${ENCRYPTIONROOT}"
> fi
> fi
> done
> fi
> fi
> doneCreated test datasets via: # zpool create \
-o ashift=12 \
-o autotrim=on \
-O acltype=posixacl -O xattr=sa -O dnodesize=auto \
-O compression=zstd \
-O normalization=formD \
-O relatime=on \
-O canmount=off \
-O recordsize=1M storage-pool2 /dev/vdd
# zfs create storage-pool2/unencrypted
# zfs create -o encryption=on -o keylocation=prompt -o keyformat=passphrase storage-pool2/encrypted
# zfs create storage-pool2/encrypted/backup
# clevis zfs bind -d storage-pool2/encrypted -l boot02 tang '{"url": "http://192.168.1.3/"}' |
|
Just curious here why the initramfs-tools integration has not added a script into The zfs-initramfs package code executed in initramfs is here. |
|
The recently linked issue has a comment #462 (comment) that reads:
Would that observed behaviour of this branch be intentional, misunderstood or an error? |
Yep, that is my current workaround: #!/bin/sh
# /etc/zfs/initramfs-tools-load-key.d/clevis
# Unlock a ZFS encryption root using Clevis-wrapped key material.
#
# How to provision the clevis properties on a dataset:
# Example (TPM+SB):
# 1. Generate a new wrapping for your dataset key:
# - For passphrase keyformat:
# echo -n 'your-passphrase' | clevis encrypt tpm2 '{"pcr_bank":"sha256","pcr_ids":"7"}' > passphrase.jwe
# zfs set clevis:passphrase="$(cat passphrase.jwe)" pool/dataset
#
# - For hex keyformat:
# echo -n 'your-hex-key' | clevis encrypt {tpm2,tang,sss} '{/*config*/}' > hex.jwe
# zfs set clevis:hex="$(cat hex.jwe)" pool/dataset
#
# - For raw keyformat:
# cat your-raw.key | clevis encrypt {tpm2,tang,sss} '{/*config*/}' > raw.jwe
# zfs set clevis:raw="$(cat raw.jwe)" pool/dataset
#
# 2. Verify:
# zfs get clevis:passphrase pool/dataset
# zfs get clevis:hex pool/dataset
# zfs get clevis:raw pool/dataset
#
# 3. Ensure the dataset's keyformat matches the property you set.
# Example: zfs get keyformat pool/dataset
#
# Environment (from zfs-initramfs):
# ENCRYPTIONROOT : encryption root dataset
# ZFS : helper to run the zfs binary
#
# Contract:
# Return 0 if we did nothing or successfully unlocked; 1 if we tried and failed.
# Helper: read a property; empty if '-' or error.
get_prop() {
val="$($ZFS get -H -o value "$1" "$ENCRYPTIONROOT" 2>/dev/null || true)"
[ "$val" = "-" ] && val=""
printf '%s' "$val"
}
# passphrase | hex | raw | none
KEYFORMAT="$(get_prop keyformat)"
# Build clevis property name directly from keyformat
CLEVIS_PROP="clevis:$KEYFORMAT"
# Fetch the JWE from that property; log whether found or not.
PROP="$(get_prop "$CLEVIS_PROP")"
if [ -n "$PROP" ]; then
log_success_msg "ZFS: found $CLEVIS_PROP on $ENCRYPTIONROOT"
else
log_warning_msg "ZFS: no $CLEVIS_PROP found on $ENCRYPTIONROOT"
return 0
fi
# Optional: log the Clevis pin type (e.g., tpm2/tang/sss) if jose is present
if command -v jose >/dev/null 2>&1; then
PIN="$(printf %s "$PROP" \
| jose jwe fmt -i- \
| jose fmt -j- -Og protected -yOg clevis -Og pin -Su- 2>/dev/null || true)"
if [ -n "$PIN" ]; then
log_success_msg "ZFS: clevis pin=$PIN keyformat=$KEYFORMAT for $ENCRYPTIONROOT"
fi
fi
# Need clevis to decrypt
if ! command -v clevis >/dev/null 2>&1; then
log_warning_msg "ZFS: clevis not available in initramfs; skipping"
return 0
fi
# Unlock using clevis → zfs (zfs interprets bytes according to keyformat)
log_begin_msg "ZFS: unlocking $ENCRYPTIONROOT with clevis ($CLEVIS_PROP)"
printf %s "$PROP" \
| clevis decrypt \
| $ZFS load-key -L prompt "$ENCRYPTIONROOT"
ret=$?
log_end_msg $ret
return $ret |
|
I'm currently working on adding Clevis support to boot and user pools in TrueNAS Community (a.k.a. Scale) via ZFSBootMenu and would love to see this merged! |
|
I hear all of you -- I'm hoping to work on this when time allows, but Life Happened 😓 |
|
@oldium Thanks I was unaware of / had forgetten about @aadnehovda Thanks for adding your script I have taken it as a starting point and reworked it to unlock all datasets not just the root. |
Sounds good. In my case I just used the normal file based keylocation (e.g. |
|
Sorry this has been languishing for a very long time. It's been working well for me, but since Ubuntu 26.04 has migrated to Dracut I'd do one more push to get both the initramfs-tools and dracut versions working (since I have to migrate my system from Ubuntu 24.04 to 26.04). Trying to summarise all the questions that's been brought up so far:
I think you're asking me to define the flag, but not actually change any other behaviour since this doesn't really take any inputs or ask questions.
Yeah it's inconsistent, I'll remove them all.
I think I've added a commit that fixes that.
clevis-zfs-initramfs does not supersede clevis-initramfs, both need to be installed. I'll add clevis-initramfs as a dependency of clevis-zfs-initramfs in my next deb build (which I'm rebasing on top of master, as an upcoming v24)
I've added another commit that fixes that, good catch.
I can't decide what the right behaviour here is: on one hand, if I were using initramfs/dracut I really only need the root volume to be unlocked. All other pools can be unlocked subsequently an init.d script or a systemctl unit when the root volume has been unlocked. In my scenario, I unlock the boot volume, and all my encrypted pools used for backups directly call On the other hand, you do have a point. But I imagine adding support for "make this one pool optional but unlock the others" would add substantial complexity. Perhaps we can get the singular pool reviewed and merged first, and figure out the multiple pool case later? As you've added patches for handling multiple pools and different permutations of encryption: there's many things to consider. @oldium @almereyda @aadnehovda:
It is installed exactly there. See I'm trying to rebase my changes onto the latest Clevis, and getting my debs to build again. Hang on for a bit (but you could also apply my latest few commits manually too) |
6fa7185 to
2d2a79c
Compare
|
Usually, it is possible to review the work commit by commit (and commit description giving you the lead - reasoning for the implementation). The Git history of this feature looks like a mess. Just saying, I am not a maintainer of this repo. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| . @initramfstoolsdir@/hook-functions | ||
|
|
||
| die() { | ||
| code="$1" |
| echo >&2 "The following ZFS datasets have been bound with Clevis:" | ||
| # we should not quote ${dataset:-} in case it is empty | ||
| # shellcheck disable=SC2086 | ||
| zfs get -H -o "${output}" -slocal "${zfs_labels_prop}" ${dataset:-} |
There was a problem hiding this comment.
This is an external in clevis-zfs-common
| # load the kernel command line vars | ||
| [ -z "$root" ] && root="$(getarg root=)" | ||
| # If root is not ZFS= or zfs: or rootfstype is not zfs then we are not supposed to handle it. | ||
| [ "${root##zfs:}" = "${root}" ] && [ "${root##ZFS=}" = "${root}" ] && [ "$rootfstype" != "zfs" ] && exit 0 |
There was a problem hiding this comment.
Good question: I don't think that we should assume the root commandline arg is set, and we should explicitly call getarg root=. I don't know what rootfstype is though?
There was a problem hiding this comment.
rootfstype is also a possible argument accepted by Systemd and also by Dracut:
- Systemd: https://www.freedesktop.org/software/systemd/man/latest/kernel-command-line.html#root=
- Dracut-ng: https://github.com/search?q=repo%3Adracut-ng%2Fdracut+rootfstype&type=code
Moreover, the root and fstype (taken from rootfstype) is globally set in non-Systemd boot here and also in Systemd boot here (saved into /dracut-state.sh reused in other Dracut hooks inside Systemd boot, like pre-mount hook here).
So for you it should be sufficient to check just the root and fstype values?
| fi | ||
|
|
||
| # if pool encryption is active and the zfs command understands '-o encryption' | ||
| if [ "$(zpool list -H -o feature@encryption $(echo "${BOOTFS}" | awk -F\/ '{print $1}'))" = 'active' ]; then |
| # if the root dataset has encryption enabled | ||
| ENCRYPTIONROOT=$(zfs get -H -o value encryptionroot "${BOOTFS}") | ||
| # where the key is stored (in a file or loaded via prompt) | ||
| KEYLOCATION=$(zfs get -H -o value keylocation "${ENCRYPTIONROOT}") |
| mktemp \ | ||
| jose | ||
|
|
||
| inst_hook pre-mount 90 "${moddir}/clevis-zfs-hook.sh" |
9acc71a to
c1df71d
Compare
|
I've got I've tested the earlier error cases:
These now work properly. I've also fixed the following:
I agree with you; I've had to pick up from where @techhazard left off. I can squash them down or even split them up for easier review when 1) this is demonstrated to work 2) there's consensus that this has a path to merge into Clevis. However, going by the fact that #373 has been open since 2022, and this has been open from 2024 with only 3 or 4 people testing this out, I'm unsure if this is a feature that actually is desired to be within Clevis. Therefore, I would hesitate to think that cleaning up for review is the highest priority now. I'd love to be wrong on this, though. |
I understand that, but this approach discourages others to have a look at your work and even try it. When I see this mess I ask the same question - can it ever get merged? I personally would answer no and go away. Think positive and keep it in a good shape, be prepared for the best - rewrite the commits, keep it mergeable all the time (I did this with TPM1 pin, kept up to 18 logically structured commits in shape the whole time for almost 2 years, regularly rebasing to master). This will attract the attention by itself. |
Co-authored-by: Joel Low <joel@joelsplace.sg>
By storing the list of labels in a well-known location we can use different keys to unlock the same encryptionroot. Co-authored-by: Joel Low <joel@joelsplace.sg>
Split the ZFS initramfs hook from the LUKS hook Installing the ZFS integration should not imply installing the LUKS integration. Co-authored-by: Vince van Oosten <techhazard@codeforyouand.me>
de29d17 to
bfb65c1
Compare
|
@oldium fair: I've rearranged all the commits logically. Took me a while but here it is. |
Co-authored-by: Joel Low <joel@joelsplace.sg>
bfb65c1 to
e3fe571
Compare
| . "$dracutlib" | ||
|
|
||
| # load the kernel command line vars | ||
| root="$(getarg root=)" |
There was a problem hiding this comment.
root should be already set, same as fstype.
There was a problem hiding this comment.
$root was missing the ZFS= at the start that getarg root didn't: the following line would then ignore the dataset. Not sure where the bug is though.
| [ "${root##zfs:}" = "${root}" ] && [ "${root##ZFS=}" = "${root}" ] && [ "${fstype}" != "zfs" ] && [ "${fstype}" != "auto" ] && exit 0 | ||
|
|
||
| # There is a race between the zpool import and the pre-mount hooks, so we wait for a pool to be imported | ||
| systemctl start zfs-import.target |
There was a problem hiding this comment.
This is making it Systemd-only and breaks non-Systemd installations. Dracut is also able to work on systems without Systemd. I would personally select a different approach for Systemd - install a simple separate unlocking service and ensure correct startup order by defining/injecting dependencies.
There was a problem hiding this comment.
Ack, but I don't know where else to place this now. This was in the original code @techhazard wrote.
There was a problem hiding this comment.
I will check it when I have time. I will setup VM with native encrypted ZFS root for testing.
There was a problem hiding this comment.
Installing ZFS is really untrivial and when you need to rescue, you need to compile ZFS in live distro... And have Secure Boot disabled, otherwise you will not be able to load the ZFS modules. Well... Why would anyone want that? 😅
Note: You can certainly install the signing certificate to your UEFI BIOS, but new distro - new certificate (unless you keep laying it somewhere near around). Live distro for rescuing - no way to install the certificate (I assume each boot it is different - not verified).
|
I am currently testing solution using ask-password (Dracut/systemd case), zfs wrapper handling
Requirements, drawbacks:
TODO: report the bug/fix to Update: Tested TPM1.2 as well. |
|
So I am done with testing, I will create a separate MR with that work. It went surprisingly smoothly with help of Opus 5.5, it saved me a lot of time actually. Features:
And as a bonus:
The work needs to be pushed, Pull/Merge Requests created. To be done in the afternoon. |
|
FYI
|
10 participants
Supersedes #373. Description copied:
Further work by @lowjoel:
clevis-zfs-bind,clevis-zfs-unlock(at reboot), andclevis-zfs-unbindOnce we're happy with the code I can squash the commits to those by @techhazard and myself.