[arch-releng] Helping with AIF

[Jeremiah Dodds]

Alexander Rødseth <rodseth at gmail.com> writes:

> Hi,
>
> 2012/7/27 Jeremiah Dodds <jeremiah.dodds at gmail.com>:
>> I find the "configuration bundles" idea interesting, but I wonder how
>> much of it is needed on a per-machine basis. I haven't run into many
>> machine-specific issues, but then again I'm sure they exist.
>
> I've encountered a lot of machine-specific issues, setting up (Arch)
> Linux on several computers. Especially laptops often need extra
> modules loaded to make the sound card, network card or video camera to
> work, possibly with a couple of firmware-packages thrown into the mix.
> Specifying all it takes to set up a machine in a single, easy to read,
> text file, shared among Arch Linux users in the same fashion as
> AUR-packages is shared, would be an improvement, IMO.

A fair amount lot of this can be automatically detected, I think. I'm
not opposed to the idea of hardware-specific "profiles" for some
installation defaults, I just haven't had use for them.

>
>
>> The end-result I'm working toward is providing a service that allows
>> people to generate linux live and install images that have what they
>> want on them already -- I've written a proof-of-concept webapp that
>> installs a package on a live and install cd that I intend to start a
>> kickstarter with[1]. My goal is to allow college kids and newer linux
>> users to create something that will give them access to the system
>> they need or want in case of failure or circumstance, and allow it to
>> perform (if desired) unattended installs.
>
> I think this approach would work too.

I don't think this approach would work well for a "distro-canon"
installation media. It works as a service, but I think it's far out of
scope for "official" installs -- I would think that a distro-provided
install disk should make generic, interactive installation possible and
try to ensure that all relevant install situations can be covered while
providing sane defaults.

>> A couple of questions on acu based off the github page:
>>
>> 1. Why create a configuration file format? There are many standard
>>    formats ranging from simplistic to very complex, and even if go
>>    didn't have an implementation of one laying around, it'd probably be
>>    simpler (and less confusing/more reliable/quicker) to write a parser
>>    for one.
>
> As far as I can tell, not that much information is needed in order to
> specify what it takes to configure a computer. A list of packages to
> install, a few changed configuration options and perhaps the addition
> of a few lines in a few configuration files should do the trick. This
> could be expressed either as a regular shell script or as some sort of
> list of declarations. Both could work, but it we imagined there was a
> webpage with hundreds of these configurations, it would be easier
> compare configurations and base new configurations on old ones if the
> file format was declarative. Also, it would be easier to parse for the
> web-system. The AUR web page struggles to parse the non-declarative
> PKGBUILD textfiles.

A listing on a web-page and a configuration file are just two different
representations of the same data, they just require a
transformation. That transformation can be well-defined and simple if
you use formats that are already standardized for config files.

Could you give me an example of a declarative format that is
significantly easier to parse than (for example), json or yaml or .ini
or the "normal" shell-script-thats-just-assignments?

>
>
>> 2. What, exactly would you have machine-specific configs entail? There's a
>>    difference between purpose-specific and hardware-specific ... I feel
>>    that the former has more common use cases, but on the other hand a
>>    *lot* of the configs and specific packages that I can think of for a
>>    "purpose" machine are going to vary wildly from user to user.
>
> ...
>
> The personal configuration, like time zone, locale
> and keyboard layout should only need to be specified once ever and
> stored as a short string or as some sort of id. I can't could how many
> times I've specified the same settings, and it should not be needed in
> our day and age. Once should be enough.

Where are you storing these? Online profiles? How does the tool access
them the next time an install needs done? What about for installs that
don't have net access? You currently only need to set those once per
install anyhow -- I can see some benefit in allowing people to store
these, but I feel like it would be out of place in an installation
media, unless perhaps as an option if there was a popular
configuration-storage service that could be used.

If I was going to try to solve this problem, and this problem
specifically, I'd create a site that allowed people to upload that data,
and then a package that could sync to/from a machine. Kinda like
etckeeper. Something like this probably already exists.

>The third and last part would
> be the information that is specific to the installation in question.
> Which disk should the installation be on? Is this supposed to be a
> media server? Should the filesystems handle small files well? Is
> wireless network preferred? In my opinion, this third part is what
> users _should_ be worrying about when installing Arch, not selecting
> keyboard layout for the n'th time and not go hunting for forum posts
> about how to get the soundcard to work on this particular brand of
> laptop.
> Hope this answered your question.

I agree in spirit.

>> 3. Why go?
>
> The non-subjective reason for choosing Go, in this connection, is that
> it easily compiles to a static binary that does not need any
> dependencies (packages).

This means we need to compile for each platform supported, no?

> For an installer, this is great. Also, the
> execution speed is fast. If there are several developers involved,
> there are no arguments about tabs vs spaces, since the standard (and
> tool that formats the code) is clear and it has good modularity.

I don't think the execution speed of the tool is going to be a
bottleneck for an installation tool. It's going to be disk and network,
unless you're a very poor coder :P.

Also, I never have arguments about tabs vs spaces. I use spaces for all
my own stuff, but when entering an existing project or community, it's
generally unspoken that you follow the convention that exists for that
project or community.


> My next plan is to rewrite the code I wrote for the web-interface for
> machine configurations, but in Go. I'm in the process of moving this
> to a server that makes this possible.

I'm a little confused as to what's being made possible -- that is, what
can't you write right now on your machine with a possible addition of a
VM?

> Either that or use the PHP-code for AUR as a basis. Or some other
> solution. Time will tell. (The current PHP-code is bad, but crappy
> prototypes keeps the momentum up).

I haven't looked at the current PHP code, so I can't comment on it. Lots
of deployed code is "bad" though. I'm more of a fan of well-done
test-suites than clean code these days, although clean code is always
desirable.

> In either case: no matter the approach and no matter the authors, I'm
> looking forward to the next generation of installers.

For sure!

A couple of general comments:

I think you underestimate the complexity of some of the problems here,
or overestimate the complexity of some of the existing solutions,
particularly in configuration management. A lot of the complexity in
software comes from people attempting to create elegant solutions, and
then running into all the edge cases that made existing solutions
complex, and then attempting to solve them in a
probably-less-complete-than-others fashion. It would be a shame to waste
a lot of time reinventing the wheel where not necessary.

Also, you do a good job above of separating configuration types in this
email. I'm fairly certain that the types that Dieter was referring to
elsewhere in the thread were more of the "stuff relevant to a particular
user" types where as you include "stuff relevant to a particular
machine" in the definition of configuration. I'm having a hard time
articulating the specifics of what I'm thinking right now, but I do feel
that there are distinctions to be made wrt types of "configurable"
things, what they're relevant to, and where it's appropriate to handle
them.

There are things relevant to a specific set of hardware.
There are things relevant to a purpose for a set of hardware.
There are things relevant to a particular user of a set of hardware.

These may have some overlap in them. *In general* though, I tend to
think that for generic install media the "specific set of hardware" set
is the only one that has a large benefit in being present in an
installation tool. The few attempts I've seen at providing
purpose-specific "generic" install media still required a silly
amount of tweaking afterwards, and user-specific installs require a
service that I doubt that many distros would want to maintain, nor do I
think that they should be present for canon install-media.

Well, that's my hopefully-sensical two cents for the moment....

-- 
Jeremiah Dodds

github     : https://github.com/jdodds
freenode   : exhortatory