[arch-general] Installation: How to get HDD > LUKS > GPT working in a clean way
Hey everybody!
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided for a rather unusual partition scheme: A single LUKS container directly on the disk (SSD) with a GPT partition table and two partitions inside it: one for SWAP, the other one for the system and everyting else, formatted with Btrfs.
The laptop runs libreboot, so I have GRUB2 as a payload inside the flash chip which I use to decrypt the LUKS container and load a GRUB configfile located at /boot/grub/grub.cfg (generated by grub-mkconfig). This works fine.
While experimenting with GPT inside LUKS before the installation I've noticed two issues, with at least one of them being present also after installation:
First, after unlocking the LUKS containter the two GPT partitions don't become visible to the kernel automatically. I have to manually do partprobe /dev/mapper/<dmname> to inform the kernel about the two new partitions. partprobe is part of parted. My idea was to create a custom hook just after the 'encrypt' hook, which would simply run the above command. I tested this and it seems to work.
Question: Is there an even simpler solution to that problem? For example an alternative to partprobe which is already in 'base'?
The second issue was that I could not (after unmounting the Btrfs partition and deactivating the swap partition of course) directly close the LUKS mapping via cryptsetup luksClose <dmname> It gave me: device-mapper: remove ioctl on <dmname> failed: Device or resource busy [...] device-mapper: remove ioctl on <dmname> failed: Device or resource busy Device <dmname> is still in use.
Instead, I had to remove the partition mappings first via dmsetup remove <dmname>1 <dmname>2 This was getting me rid of the aforementioned error messages.
As expected, I get these error messages also during system shutdown -- but only whith the shutdown hook in initramfs. Without it, I presume the system does not even try to close the LUKS container (which would make sense, since there is no initramfs created by default for shutdown afaik), therefore also resulting in no error messages being shown.
What could I do about this? I'd like to have my system closing the LUKS container correctly -- therefore I need to remove the partition mappings before that.
I've read a lot in the last days and weeks about Btrfs, SSDs, coreboot, etc. to make sure I don't run into many issues. Though these two don't come unexpected, I don't know how to solve the latter one, because systemd shutdown and shutdown initramfs are still a little miracle to me...
I'd really appreciate any help!
This is all on an up-to-date vanilla 4.8.10-1-ARCH. I attached two shutdown logs with debugging enabled: one with and one without the shutdown hook applied. They look very similar though. (I made sure to reboot twice after building the initramfs before taking the shutdown log.) My HOOKS array is: HOOKS="base udev autodetect modconf block keyboard keymap encrypt pp \ filesystems fsck shutdown" (pp being the hook which runs partprobe against the mapped LUKS container)
Best Regards,
Merlin
On Sun, Nov 27, 2016 at 04:48:18PM +0100, Merlin Büge wrote:
Hey everybody!
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided for a rather unusual partition scheme: A single LUKS container directly on the disk (SSD) with a GPT partition table and two partitions inside it: one for SWAP, the other one for the system and everyting else, formatted with Btrfs.
I'm not really answering your question here, but just out of curiosity what's the appeal of this approach over doing one of the options described at https://wiki.archlinux.org/index.php/Dm-crypt/Swap_encryption:
(1) dm-encrypted swap with a new key generated each boot (what I generally use, is the most secure, and works in any case where you don't need to hibernate) (2) The same thing you're trying to do but with LVM instead of a GPT table (3) a swap file on your Btrfs volume itself
I've done all these at various points and they all work well.
Jeremy
Hi,
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided for a rather unusual partition scheme: A single LUKS container directly on the disk (SSD) with a GPT partition table and two partitions inside it: one for SWAP, the other one for the system and everyting else, formatted with Btrfs.
I'm not really answering your question here, but just out of curiosity what's the appeal of this approach over doing one of the options described at https://wiki.archlinux.org/index.php/Dm-crypt/Swap_encryption:
(1) dm-encrypted swap with a new key generated each boot (what I generally use, is the most secure, and works in any case where you don't need to hibernate)
Hibernation :P
(2) The same thing you're trying to do but with LVM instead of a GPT table
I don't need any LVM functionality and would consider it a little bit overkill for just dividing available space into two partitions.
(3) a swap file on your Btrfs volume itself
That's (currently) not possible, see the link you posted.
By the way, please try to add line breaks at about 75 characters, see [0].
Regards,
Merlin
[0] https://wiki.archlinux.org/index.php/Code_of_conduct#Mailing_lists
On Sun, Nov 27, 2016 at 09:43:57PM +0100, Merlin Büge wrote:
(2) The same thing you're trying to do but with LVM instead of a GPT table
I don't need any LVM functionality and would consider it a little bit overkill for just dividing available space into two partitions.
I completely disagree. As another poster said in their response, LVM is really the canonical/supported/more flexible way to do this. Even on systems where I don't need encryption, I generally use LVM to divide up the space instead of traditional partitions.
(3) a swap file on your Btrfs volume itself
That's (currently) not possible, see the link you posted.
Gah. Sorry. You are correct. Should have read my own link all the way through before sharing it. I usually use good ole ext4.
By the way, please try to add line breaks at about 75 characters, see [0].
[0] https://wiki.archlinux.org/index.php/Code_of_conduct#Mailing_lists
Ah, got it. Haven't posted to this list much, but noted for this and future posts :)
Cheers,
Jeremy
On 11/27/2016 09:48 AM, Merlin Büge wrote:
Hey everybody!
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided for a rather unusual partition scheme: A single LUKS container directly on the disk (SSD) with a GPT partition table and two partitions inside it: one for SWAP, the other one for the system and everyting else, formatted with Btrfs.
The laptop runs libreboot, so I have GRUB2 as a payload inside the flash chip which I use to decrypt the LUKS container and load a GRUB configfile located at /boot/grub/grub.cfg (generated by grub-mkconfig). This works fine.
While experimenting with GPT inside LUKS before the installation I've noticed two issues, with at least one of them being present also after installation:
First, after unlocking the LUKS containter the two GPT partitions don't become visible to the kernel automatically. I have to manually do partprobe /dev/mapper/<dmname> to inform the kernel about the two new partitions. partprobe is part of parted. My idea was to create a custom hook just after the 'encrypt' hook, which would simply run the above command. I tested this and it seems to work.
Question: Is there an even simpler solution to that problem? For example an alternative to partprobe which is already in 'base'?
The second issue was that I could not (after unmounting the Btrfs partition and deactivating the swap partition of course) directly close the LUKS mapping via cryptsetup luksClose <dmname> It gave me: device-mapper: remove ioctl on <dmname> failed: Device or resource busy [...] device-mapper: remove ioctl on <dmname> failed: Device or resource busy Device <dmname> is still in use.
Instead, I had to remove the partition mappings first via dmsetup remove <dmname>1 <dmname>2 This was getting me rid of the aforementioned error messages.
As expected, I get these error messages also during system shutdown -- but only whith the shutdown hook in initramfs. Without it, I presume the system does not even try to close the LUKS container (which would make sense, since there is no initramfs created by default for shutdown afaik), therefore also resulting in no error messages being shown.
What could I do about this? I'd like to have my system closing the LUKS container correctly -- therefore I need to remove the partition mappings before that.
I've read a lot in the last days and weeks about Btrfs, SSDs, coreboot, etc. to make sure I don't run into many issues. Though these two don't come unexpected, I don't know how to solve the latter one, because systemd shutdown and shutdown initramfs are still a little miracle to me...
I'd really appreciate any help!
This is all on an up-to-date vanilla 4.8.10-1-ARCH. I attached two shutdown logs with debugging enabled: one with and one without the shutdown hook applied. They look very similar though. (I made sure to reboot twice after building the initramfs before taking the shutdown log.) My HOOKS array is: HOOKS="base udev autodetect modconf block keyboard keymap encrypt pp \ filesystems fsck shutdown" (pp being the hook which runs partprobe against the mapped LUKS container)
Best Regards,
Merlin
Hi Merlin,
I'd set up two partitions: Your EFI system partition and the LUKS container. Then inside LUKS, format the whole thing as LVM and then set up from there, rather than make the LUKS container another GPT "disk." Then you just use the crypt and lvm2 hooks.
You should only really use partition tables on a physical disk, in my opinion, not a LUKS container.
The reason for this is that LVM works with a lot more flexibility and is more readily automated than trying to get the system to re-read partition tables.
If you were on a system where you could add disks, I'd even suggest reversing it: LVM on the metal and LUKS on a logical volume spanning the whole VG, that way you could grow the whole thing across multiple disks pretty easily without having to make any dramatic changes. I know btrfs can do multiple disks, but I've always preferred how LVM does it, honestly.
Yaro
Hi Yaro,
thanks for your answer!
I'd set up two partitions: Your EFI system partition and the LUKS container. Then inside LUKS, format the whole thing as LVM and then set up from there, rather than make the LUKS container another GPT "disk." Then you just use the crypt and lvm2 hooks.
I have no EFI system partition because I don't need one.
You should only really use partition tables on a physical disk, in my opinion, not a LUKS container.
The reason for this is that LVM works with a lot more flexibility and is more readily automated than trying to get the system to re-read partition tables.
Hm. I can see your points. But I don't need the flexibility LVM provides, I have enough flexibility through Btrfs. And yeah, it's readily automated, and that's indeed practical for many people. Personally, I'd rather modify the start-up process a tiny bit so that GPT inside LUKS gets parsed. I just try to strip off unnecessary 'overhead' / layers of my system.
If you were on a system where you could add disks [...]
Since it's on my laptop, I don't need that functionality :)
Best Regards,
Merlin
On 11/27/2016 08:09 PM, Merlin Büge wrote:
Hi Yaro,
thanks for your answer!
I'd set up two partitions: Your EFI system partition and the LUKS container. Then inside LUKS, format the whole thing as LVM and then set up from there, rather than make the LUKS container another GPT "disk." Then you just use the crypt and lvm2 hooks.
I have no EFI system partition because I don't need one.
My mistake. I assumed you had a typical UEFI setup, which would require one.
You should only really use partition tables on a physical disk, in my opinion, not a LUKS container.
The reason for this is that LVM works with a lot more flexibility and is more readily automated than trying to get the system to re-read partition tables.
Hm. I can see your points. But I don't need the flexibility LVM provides, I have enough flexibility through Btrfs. And yeah, it's readily automated, and that's indeed practical for many people. Personally, I'd rather modify the start-up process a tiny bit so that GPT inside LUKS gets parsed. I just try to strip off unnecessary 'overhead' / layers of my system.
Okay, then.
Here's my opinion on this approach.
If you have 8 GiB or more and not hibernating, don't bother with swap, it'd be a waste of disk space. In that case you could just put a btrfs volume straight on the LUKS container without the GPT. Problem solved as you don't need any more volume management than opening LUKS containers.
Otherwise WITH swap: Unfortunately btrfs (still) doesn't support swap files properly, otherwise I'd suggest using them. You can write a custom hook. Unless you plan to share it, I'd make it a dead simple shell script that simply reruns the command to scan for added GPT partitions for your specific setup. Make sure you have a setup hook that gets the dependencies in there.
Personally, I still think you should just use LVM, for the simple reason you're having trouble with GPT, which is not meant for being used like this, since it can work as a more flexible "partition table" inside the LUKS container and is better supported all around. btrfs really doesn't act as a good replacement for logical volumes, in my experience. Having something with more features than you need is better than trying to coerce something into working ways it's not really intended.
If you were on a system where you could add disks [...]
Since it's on my laptop, I don't need that functionality :)
Best Regards,
Merlin
Hi,
Hm. I can see your points. But I don't need the flexibility LVM provides, I have enough flexibility through Btrfs. And yeah, it's readily automated, and that's indeed practical for many people. Personally, I'd rather modify the start-up process a tiny bit so that GPT inside LUKS gets parsed. I just try to strip off unnecessary 'overhead' / layers of my system.
Okay, then.
Here's my opinion on this approach.
If you have 8 GiB or more and not hibernating, don't bother with swap, it'd be a waste of disk space. In that case you could just put a btrfs volume straight on the LUKS container without the GPT. Problem solved as you don't need any more volume management than opening LUKS containers.
Otherwise WITH swap: Unfortunately btrfs (still) doesn't support swap files properly, otherwise I'd suggest using them. You can write a custom hook. Unless you plan to share it, I'd make it a dead simple shell script that simply reruns the command to scan for added GPT partitions for your specific setup. Make sure you have a setup hook that gets the dependencies in there.
Personally, I still think you should just use LVM, for the simple reason you're having trouble with GPT, which is not meant for being used like this, since it can work as a more flexible "partition table" inside the LUKS container and is better supported all around. btrfs really doesn't act as a good replacement for logical volumes, in my experience. Having something with more features than you need is better than trying to coerce something into working ways it's not really intended.
Thanks for your input!
I've reconsidered LVM twice now but still feel better without it.
Cheers,
Merlin
On 2 December 2016 at 22:29, Merlin Büge toni@bluenox07.de wrote:
Personally, I'd rather modify the start-up process a tiny bit so that GPT inside LUKS gets parsed. I just try to strip off unnecessary 'overhead' / layers of my system.
If you have 8 GiB or more and not hibernating, don't bother with swap, it'd be a waste of disk space. In that case you could just put a btrfs volume straight on the LUKS container without the GPT. Problem solved as you don't need any more volume management than opening LUKS containers.
If another opinion helps, I've done some funky disk layouts at various times, and I also think that if you need partitioning above the LUKS layer, you'd do better to use LVM than GPT. GPT is intended to be used at the lowest level of the stack, whereas LVM is well-supported at pretty much any level. If you do go ahead with it, double-check that you won't get block alignment issues in that stack that could affect IO performance.
However, if you say that you don't need the flexibility of LVM, I'd certainly first try btrfs directly on top of LUKS.
Final consideration: if you want GRUB to open a LUKS container and then load stage 2 from btrfs, you'll need a decent amount of storage for the GRUB 1st stage, which on a traditional setup goes in free space you need to account for after the MBR (or on the EFI partition for UEFI setups). In your case, as the whole disk is LUKS and you have no partition table, have you considered where the GRUB 1st stage will be stored? I use a USB stick to boot GRUB stage 1 on my encrypted machines, and that may work for you too.
Paul
Hi Paul,
If another opinion helps, I've done some funky disk layouts at various times, and I also think that if you need partitioning above the LUKS layer, you'd do better to use LVM than GPT. GPT is intended to be used at the lowest level of the stack, whereas LVM is well-supported at pretty much any level. If you do go ahead with it, double-check that you won't get block alignment issues in that stack that could affect IO performance.
Thanks for your input. (I already checked alignment of my setup.)
However, if you say that you don't need the flexibility of LVM, I'd certainly first try btrfs directly on top of LUKS.
If I did not want to have a swap partition, I'd to that for sure. Another possible layout which just comes to mind is
GPT +-LUKS | +-Btrfs +-LUKS +-SWAP
I think that should work with hibernation, too, and GPT would be on the right place + still no LVM :) Maye I'll just try different layouts over time, haven't experimented much yet.
Final consideration: if you want GRUB to open a LUKS container and then load stage 2 from btrfs, you'll need a decent amount of storage for the GRUB 1st stage, which on a traditional setup goes in free space you need to account for after the MBR (or on the EFI partition for UEFI setups). In your case, as the whole disk is LUKS and you have no partition table, have you considered where the GRUB 1st stage will be stored? I use a USB stick to boot GRUB stage 1 on my encrypted machines, and that may work for you too.
As mentioned in my initial post, I have GRUB2 along with (deblobbed) coreboot stored in the SPI flash chip (so no BIOS here). It's very convenient :)
Regards,
Merlin
On Sun, Nov 27, 2016 at 04:48:18PM +0100, Merlin Büge wrote:
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided for a rather unusual partition scheme: A single LUKS container directly on the disk (SSD) with a GPT partition table and two partitions inside it: one for SWAP, the other one for the system and everyting else, formatted with Btrfs.
Why is it unusual? Everyone does this: LUKS on a raw block device, then LVM or btrfs subvolumes inside it.
The laptop runs libreboot, so I have GRUB2 as a payload inside the flash chip which I use to decrypt the LUKS container and load a GRUB configfile located at /boot/grub/grub.cfg (generated by grub-mkconfig). This works fine.
Where is /boot physically located? Can grub2 boot from LV these days?
Cheers,
Hi,
On Sun, Nov 27, 2016 at 04:48:18PM +0100, Merlin Büge wrote:
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided for a rather unusual partition scheme: A single LUKS container directly on the disk (SSD) with a GPT partition table and two partitions inside it: one for SWAP, the other one for the system and everyting else, formatted with Btrfs.
Why is it unusual? Everyone does this: LUKS on a raw block device, then LVM or btrfs subvolumes inside it.
Because in my case LUKS occupies the hole physical disk. I think not everyone does this. Or, because I have GPT inside a LUKS container, which obviously does not run out-of-the-box.
The laptop runs libreboot, so I have GRUB2 as a payload inside the flash chip which I use to decrypt the LUKS container and load a GRUB configfile located at /boot/grub/grub.cfg (generated by grub-mkconfig). This works fine.
Where is /boot physically located? Can grub2 boot from LV these days?
/boot is physically located on my only storage drive in the laptop. It's not a seperate partition, just on the Btrfs filesystem.
Can grub2 boot from LV these days?
I don't know (nor do I need).
Regards
On 11/27/2016 08:18 PM, Merlin Büge wrote:
Where is /boot physically located? Can grub2 boot from LV these days?
/boot is physically located on my only storage drive in the laptop. It's not a seperate partition, just on the Btrfs filesystem.
Can grub2 boot from LV these days?
I don't know (nor do I need).
Just to answer: Yes, grub2 can boot from LVM and LUKS. UEFI typically requires an open boot partition, but that could just be used to launch grub2 which could in turn read things from an encrypted logical volume.
My current computer is a legacy BIOS system (Ugh, I prefer UEFI.) so I can put the whole thing in LVM/LUKS no issues, since BIOS doesn't use a system partition. Arch's wiki should provide adequate instructions, I believe.
Regards
participants (5)
-
Jeremy Brown
-
Leonid Isaev
-
Merlin Büge
-
Paul Gideon Dann
-
Yaro Kasear