[arch-releng] Draft - Beyond AIF, long live AIF...

Mon Aug 17 14:49:05 EDT 2009

On Mon, 17 Aug 2009 19:19:22 +0200
Gerhard Brauer <gerbra at archlinux.de> wrote:

> Installer paradigms
> ====================
> Today installers (either if graphically or text based) are mostly
> interactive tools. They ask the user for information and the user is
> giving data to the installer. Also the user must make decisions in some
> steps, say Yes or No. Also in current AIF in interactive mode.
> 
> Usual a installation is a top-down and sequent work-flow. But this don't
> prevent that user do "silly" things. So installers (also AIF) must do
> many things "behind the view" to intercept mistakes. 

I don't think aif does much checking for mistakes from the user.
The only things I can think of: notices for missing swap/root/home etc,
and checking if steps are completed succesfully. Little code overhead here IIRC.

> Or need user
> invention/decision based on information from a procedure before current
> step...
> As a example the way grub install is handled by old arch-installer (AIF
> does this a little other): We have to ask about root/boot devices to
> generate a matching menu.lst file. This file is presented to the user
> again to have a look or change something. From the point that the user
> may have changed something some/most informations must be asked again
> (or be parsed from the changed menu.lst file).
> 
> Main problem IMHO is: a today installer doesn't know at the beginning
> with what information it will conclude at the end. So each needed
> information for a step has to be asked/determined before this step.

maybe we should estimate how much extra code is needed for these things.  I would guess a few hundred lines maximum.

> ... we are frogs!
> =================
> During last images build and bugfixing i thought a lot about this. I
> know that our "old Devs" always prefer quickinst from old arch-installer
> package. Now coming familar with AIF, myself look most times and primary
> on the interactive part; but doing some work with automatic install
> scripts IMHO there is more potential than with quickinst.

yeah.

> 
> So my question: Am i totally wrong with my thoughts and - if not - have
> we the pluck to confront the user with... an text editor?
> 
> Take a breath before hit Reply key... ;-)
> 
> First: The above mentioned aif scripts are not the answer - syntax,
> presentation and documentation are more the "proof-concept" to look for
> answers.

what do you mean exactly?  the syntax is too hard for the config files?
I do think the disk section of the configs can be made more simple. (see also http://bugs.archlinux.org/task/15640?project=6), but i like the other stuff.

> 
> Pros & Contras
> ==================
> 
> 1. Making the settings for installation in a control file gives not that
> pressure to the user what a User-Interface(UI) based installer has.
> Working with this file ("editing") does not any bad things. Look for ex.
> on the grub part currently in interactive AIF: The user is confronted in
> this step with questions in a very isolated manner. This step is very
> interdependent from the previous blockdevice setup - but these
> information's were entered before 10 minutes and out-of-users-sight in
> this step. In a editor all information is easy browseable and
> corrections could make at any point.
> IMHO a pro.
If the information we need was entered earlier, we can probably reuse it by storing it in a variable.
(like we do now for some things like proxy server, keymap etc)
> 
> 2. Install information could be saved at any time to external place by
> the user - or come complete from external medium. Currently our install
> procedure has to be restarted every time if the user got "stucked" in
> one step, or canceled the process, or when the installer has a bug. Most
> (and so in fact all) informations have to be re-entered...
> Think further about the capabilities: We (and the community) could offer
> templates for nearly each installation setup - without the need to bring
> this in a UI...

yeah this sounds good

> If one has the idea to write a web-based interface to generate such
> installation control file - why not?

i hate webinterfaces.  I get your point, but I would do it with shellscripts.
> 
> 3. On the road to 2009.08 (and AIF) many of reported problems were UI
> related problems. Such things like "Cursor not automatically placed in
> Menu xyz". But also during the grub refactoring we have seen: to get
> feature or procedure X to work we must have here a new dialog, and there
> we have to change a other dialog... and so on.
> I guess on most interactive programs the UI-based part is the biggest
> part. And often the part with the most possible bugs.
> Current interactive AIF hast 2 UIs: the ncurses dialog based and a pure
> read/print based one. Ok, we will never ever have a graphical installer,
> but we hear often: why do you use such a "antique" UI? And in fact: Most
> today machines/environments could handle a graphically installer and
> also modern frameworks give even better widgets to interact with the
> user... But: Another UI gives potential more bugs and needs more
> manpower which could be better used to make the "base" more stable.
> If we use the control file based install we get rid off from such
> discussions of the "neatest" installer... And IMHO the step away from
> "how the others do it" could make us very modern...
> IMHO a pro.
I don't think there is anything wrong with ncurses / std{in,out,err} per se.
But in general, yes: the simpler the better (as long as it's still easy)
> 
> 4. Yes, i have a contra... Control files/variables tend to be not very
> self-explanation. For example take the current syntax for blockdevice
> setup:
> --------------
> GRUB_DEVICE=/dev/sda
> PARTITIONS='/dev/sda 100:ext2:+ 512:swap *:ext4'
> BLOCKDATA='/dev/sda1 raw no_label
> ext2;yes;/boot;target;no_opts;no_label;no_params
> /dev/sda2 raw no_label
> swap;yes;no_mountpoint;target;no_opts;no_label;no_params
> /dev/sda3 raw no_label ext4;yes;/;target;no_opts;no_label;no_params'
> -------------
> Here we have to invest a lot of work to bring such a syntax to the user
> in a better way. But on such complex steps i could imagine on "helper
> applications". Maybe a similar dialog-based we currently use in
> interactive AIF. But instead to "feed" a installer with this
> informations this helper produces for ex. the BLOCKDATA string for above
> control file.
> The user then import this generated piece into the editor.
> Similar helper could be one for timezone strings; all information a user
> normally not know by rote and so must be "picked" from a list.
> IMHO: if we make this good, the contra could went to a pro - cause at
> the end the user has still a (correctly) control file to start
> installation.
yeah, but note this syntax is not definite (see http://bugs.archlinux.org/task/15640?project=6)
> 
> Read until here....?
> =======================
> Wow. Yes, I'm really thinking (and like to discuss) about a work-flow:
> a) User boots the image, login.
> b) Select/start a text editor with a base control file or select one of
> the other pre-defined control files.
> c) Edit his settings in this control file
> d) Uses the other tty's for getting infos, start helpers. Helper output
> he could load into the current control file editor.
> e) After "editing the setup" he could save it extern and come back later
> or
> f) start the installation. No interaction is needed after this (Ok,
> before fdisk'ing and formatting we should make a "break and review").
> g) Base system is installed - user is happy... Cookies and taccos...
> 
> What do you think? Of course, this is nothing for the next images... 
> It's a massive U-turn against the main stream, but at the end IMHO the
> benefits could be great - KISS like...
> 
> These are only my personal thoughts as a releng member, no position of
> "official politic".
> 
> And now: Fire at will! ;-)
> 
> Regards
> 	Gerhard
> 
> 
> [1]
> http://github.com/Dieterbe/aif/blob/2ef92eaa5f1106d8f1b379a8a94a477a0877bb1f/examples/generic-install-on-sda
> [2]
> http://github.com/Dieterbe/aif/blob/2ef92eaa5f1106d8f1b379a8a94a477a0877bb1f/examples/fancy-install-on-sda
> 

I've also been thinking the installer currently does too much hand holding.
Especially if you compare to an arch system after it is installed: it lets the user manually invoke pacman and let him edit rc.conf himself etc.
There are no fancy wrappers then, so we probably don't need them during the installation either.

I do think it's very beneficial to have a method to install arch:
- that requires you to do no single keystroke more then needed to give you the system you want.
- that does not take any second longer then needed.
- that assures you you didn't forget anything
This is where -p interactive is very good at. No matter my love for automated installs, it's nice that you can almost brainlessly do the interactive one and end up very quickly with a customized install.

I like the idea of simple scripts that generate config files for -p automatic. based on a few questions, output of fdisk -l etc.
(actually I think this is how Gentoo is doing it now, or at least how one of the Gentoo relengs planned to do it.  I've been chatting with one when working on early aif)
But if you want to generate all fine nuances into the config file, the script will become as complex as -p interactive.

note that the "store disk layouts in a nice format, and programatically parse those formats and act correctly on them (invoke mkfs, mount etc)" we currently do requires quite some code (rough estimate: about 1000 sloc).
I've been thinking if it wouldn't be better to let the user write scripts in which he does 'mkfs.. mount..' himself.  it would definitely be easier to code, but the current approach offers some features like rollbacks etc which you otherwise would have to do manually (or script it also), which would make the users scripts/configs uglier.

Dieter