[arch-releng] btrfs support in aif

[Dieter Plaetinck]

On Tue, 7 Dec 2010 09:35:29 -0600
C Anthony Risinger <anthony at extof.me> wrote:

> where does btrfs stand in terms of AIF?

currently no btrfs support whatsoever.
let me share some aif design insights with you.

my acronyms:
LV = logical volume
VG = volume group
PV = physical volume.
DM = device mapper
BD = block device
FS = file system
DF = devicefile

"normal" FS'es ("do something on the BD represented by
DF /dev/foo, so that you can then call `mount /dev/foo
<somedir>`") are _trivial_ to add to aif.

how aif works is this: it uses a "model" that represents how your
DF/FS structure will look like.
i personally usually have a layout like this:
a boot partition, and a partition on which i do
dm_crypt, which results in a DM BD, which I make a PV, then put a VG on
it, which contains multiple LV's, one for swap, and two containing
some FS'es which get mounted as / and /home.

you can see that model on the bottom of this file:
https://github.com/Dieterbe/aif/blob/master/examples/fancy-install-on-sda

you probably have noticed in the installer how you first configure all
your filesystems in the dialog interface, but only after confirming, it
does all the required actions, step by step.  since it also supports
automatic installs where you define your FS's in abitrary order, aif
figures out the dependencies and processes things in the right order.
(in my example: first create the dm_crypt, then the PV, then the VG,
then the LV's, then the FS'es on those LV's)
then mount all mountpoints in the right order( first /, then /home
and /boot)

I choose this model-based approach initially because I wanted to get rid
of the ugly, hacky original installer code, but still provide a lot of
control through the nice dialog interfaces. (and i wanted to allow
automautic installs where you could just specify how you wanted your
FS/BD structure to look like, not a series of commands)

advantages:
- provides some abstraction, it's easy to add new (simple) filesystems.
  support for "buildup" and rollback comes for free (for simple
  filesystems)
- makes dialog-based "configurator" a bit easier.
disadvantages:
- the more control you want to give users, the more you're just putting
  effort in wrapping commandline arguments in fancy dialog interfaces
  (although there is also a textbox where you can enter which ever
  additional arguments you want, so this is a compromise)
- pretty hard to implement fancier filesystems, you usually need to
  take the common use cases. (read on)
- bash datastructures are very limited. it's not easy to model such a
  datastructure (which, if you simplify things is a tree, but in real
  life leafs can have multiple parents, like a VG can be using multiple
  PV's).  so quite some code is needed to update and parse text files
  to mimic the datastructure, although I do consider using a specific
  optimized text format and an external tool to update/query the data.
 (https://bugs.archlinux.org/task/15640)
- users cannot do their own stuff outside aif and expect to see the
  results inside aif.

here are some examples why this design can lead to complicated code.
- since all the modeling in the UI happens first, and the actual
  applying is later, this means you need to add code that detects
  "okay, you just added a dm_crypt, we need to ask a label, okay now we
  use that label to write a 'fake' /dev/mapper/<label> to the file
  which will appear in the menu".
- for PV's, you need a way to differentiate in the menu between the
  real BD and the actual PV, so i write an entry as <PV DF>+ to the
  file. the '+' suffix represents the actual PV on which you can put
  VG's.
- for VG's, which can contain 0 to n LV's, I needed to hack/extend my
  model for a "BD" to support multiple "FS"'es at once. (line 34 in the
  textfile)
- a VG can actually span multiple PV's, but that would be really hard
  to implement (and not often used) so I just ignored that.
- consider all these complications, then consider what needs to
  happen when something goes wrong and aif needs to do a rollback
 the rollback does the inverse of what i described
  earlier (umount all filesystems in the right order, then destroy the
  devices which need wiping/destroying in the right order).  Cool
  feature but requires a bunch of hard to maintain code and I doubt
  it's used often.



Because of all this, I have sweared quite a bit over the last few
years.  And thought about maybe we should KISS and let users do
everything on the commandline, or provide a minimal layer of
abstraction, like provide some scripts which they can modify that setup
a system in a certain way. (for example, a series of mkfs; mount;pacman;
calls)
I guess it's a tradeoff between making it easy for users and
not overloading the brain of people who want to hack on aif.


This is why softraid hasn't been implemented yet, nor btrfs.
Also, I'm not very familiar with either one (although I am pretty
interested in btrfs).
I would need to know the most common/recommended use cases, and
figure out the best way to implement them.
Or maybe just provide a few predefined wizards for specific setups
(but there are so many possibilities this would be unfeasible, I think)

Or I need to take a different approach (see a bit above)

maybe we can do btrfs in a reasonably unpainful way, what are the
common ways to set it up? (or at least as starting point, knowing you
can further add snapshots and whatnot without worrying about that in
aif)
if we can do it in a similar way like i did with lvm (don't a "thing"
spanning multiple things "below it", but allow multiple things "on top
of it") it could be pretty easy actually, but btrfs seems so advanced I
don't like to only provide half-baked support for it.

maybe we should get in touch on IRC or IM to discuss possible
approaches.  your initcpio support definitely looks cool (although
that's not really an arch-releng topic)

> at the very least, if it's not already, i'd like to see the
> btrfs-progs be included.

that would be trivial.  but not on my agenda yet.  on any
archiso medium you can just do `pacman -Sy btrfs-progs` (if you have
networking)

Dieter