[arch-releng] Fw: aif -p automatic

Begin forwarded message: Date: Tue, 31 Mar 2009 13:21:51 +0200 From: Dieter Plaetinck To: Jud Subject: Re: aif -p automatic On Tue, 31 Mar 2009 18:03:44 +1000 Jud wrote:

Hi Dieter,

How's it going?

Okay, my right shoulder hurts a bit atm. I hate chairs with "arm rests" that bother me more then they help. Other then that all okay.

Noticed you have started working on an automatic install feature for Arch. IMHO this the most important feature Arch needs.

Yeah it's something I really want myself too.

I took the liberty of editing your example file to how, from my experience, I would like it to work. Are these ideas possible? There are many questions inline, hope this helps and I am available to answer any of your questions. If you need some help implementing any of the ideas, I like to try and help out.

Thanks. Any help is welcome. I also have a big todo list and various tickets open on the bugtracker. Feel free to explore the codebase, ask me questions and hack on. Note that in general I like to adhere to the kiss and especially the unix philosophy/ies. Eg I want to make aif able to output a little "report" at the end (there is a function for it, but it's a work in progress), however I don't believe things like sending mails belongs in aif.

Regards

Jud

# AIF Automatic.example # these variables are optional, here are the defaults (feel free to omit them) #

# Goal: Setup a very workable machine(s) automatically

that goal is not specific to this profile (==config file). you can say this for the complete procedure and all configs for it. in fact, aif's procedures and their goals are explained in the documentation.

#

SOURCE=ftp usb # [rsync http webdav ftp usb disc] if can't get net connection fallback to packages on boot usb stick

FILE_URL=/dev/sdb1 # does the install script know the boot media and /dev address? Not sure what exactly you mean. SYNC_URL=192.168.1.2 # The ftp:// and /${REPO}/os/${CARCH}/ should be done automatically, fallback to pacman-mirrorlist well, you can't know the full url automatically because there can be

what's all this for? we support ftp and "having a repo available in the VFS", mounted at /src/core/pkg afaik. getting the packages in place (either when generating the medium, or at runtime) is not something that belongs in aif imho. besides, aif supports what pacman supports here. parts specific per distro. I'll think I'll just stick to the same format as the mirrorlist. referring to the mirrorlist seems like an okay idea, though if you configure the config file for aif, why would anyone configure the mirrorlist file if you're creating the aif config file already anyway?

LOGGING_SERVER= # Address to connect to a remote logging server/service maybe i could use syslog for aif logging. you can then configure syslog for remote logging and whatever EMAIL="Admin Guy " # send errors, finished log, etc # verify address first? EMAIL_SERVER="smtp.example.com:25" belongs in a script that wraps around aif imho. feel free to write this (or convince me why "one program to do it all" is better)

# Do you want to have additional pacman repositories or packages available at runtime (during installation)? RUNTIME_REPOSITORIES= RUNTIME_PACKAGES=openssh ntp ssmtp

# you can optionally also override some functions... worker_intro () { infofy "Automatic procedure running the generic-install-on-sda example config. THIS WILL ERASE AND OVERWRITE YOUR /DEV/SDA. IF YOU DO NOT WANT THIS PRESS CTRL+C WITHIN 10 SECONDS" sleep 10 # countdown onscreen? } # Could/Can this be run from GRUB menu on the boot media - ie no user interaction at all?

this is just an example of how to override worker functions from the procedure. you can leave this out. invoking "aif -p automatic -c <some config file>", *can* be done automatically when booting the cd. but it's not something i would enable by default on arch official media. (unless maybe as non-default grub option but then it's not 100% automatic anymore)

PROFILES=TestMachine BuildMachine Laptop Server VirtualTest

# Profiles would be picked at start of script and/or grub menu? # Detect by MAC address?

[TestMachine] MAC=01:E5:17:1C:28:47

well, an aif profile == config file. i'm sure you can write a little script that calls parses dmesg/ifconfig whatever, and calls aif with the correct config file. (or as an option through grub, sure. but you'll have to write your own script for that)

# Packages to Install TARGET_REPOS=core extra community # Could add a list of repos to enable eg: testing, kdemod

sure that's what it's for

TARGET_GROUPS=base xorg gnome #all packages in this group will be installed (defaults to base if no group and no packages are specified) TARGET_PACKAGES=openssh openssl ntp di memtest86+ alsa-lib alsa-utils xf86-video-ati gdm gnupg firefox thunderbird openoffice-base cups cups-pdf ghostscript gsfonts xsane mc-mp rsync ssmtp ttf-liberation ttf-freefont # you can also specify separate packages (this is empty by default) TARGET_PACKAGES_EXCLUDE=jfsutils reiserfsprogs ppp rp-pppoe xterm epiphany gnome2-user-docs # These packages will not be installed unless needed as a dependency for TARGET_PACKAGES

good idea. i suggest: 1) resolve TARGET_GROUPS into list of packages 2) filter out all TARGET_PACKAGES_EXCLUDE 3) add all TARGET_PACKAGES you can easily modify aif to follow such a behaviour. just add step 2.

# /etc/rc.conf Settings HOSTNAME="TestMachine"

# Could also be prompted as script starts? # i don't like adding interactivity if we don't need it. let's make $HOSTNAME a mandatory variable. Also auto append to /etc/hosts yes this has been added to aif recently (aif core libraries. so you also get this behavior with aif -p interactive). now i think of it, it could be that i only added it for -p interactive. i'll look into fixing

LOCALE="en_AU.utf8" # /etc/local.gen should be automatically set to this option only HARDWARECLOCK="UTC" TIMEZONE="Australia/Brisbane" # Set TZ Environment Variable KEYMAP="us" eth0="dhcp" MODULES=(powernow-k8 cpufreq-ondemand !ipv6) DAEMONS=(@alsa @hal @fam @sshd @gdm) # Append to defaults or supply whole line? looks all good to me. maybe we could let the user specify DAEMONS (override) and DAEMONS_APPEND (add a few daemons to the defaults). same for modules maybe?

GROUPS='staff accounts' # groups to be created good idea, but i'm not sure how we would add more advanced options (like specifying gid's, members of the group etc). maybe we could just

USERS='root:[hashed_password]' # Not sure if this is a good idea - how did you handle root? don't think this is a bad idea. the automatic profile doesn't do anything special for the root user yet (so you get the default pass, which is empty i think). similar thoughts as with groups though. 'fred:Fred Nerk:staff,audio:[hashed_password]' # Might be good to add other options like full name, UID, secondary groups etc 'mary:Mary Smith:staff,accounts,audio,optical,scanner:[hashed_password]' # user would be prompted to change password on first login

# These variables are mandatory

GRUB_DEVICE='/dev/sda' GRUB_OPTIONS='TIMEOUT=2 0:vga=775 quiet # Add to the kernel= line for grub item 0 1:vga=773' hmmm.. okay maybe. PARTITIONS='/dev/sda 100:ext2:+ 512:swap 2048:ext4 4096:ext4 *:ext4' # The :+ makes it bootable? yes, though it's not _needed_. it toggles a flag in the MBR but it doesn't do much '/tmp tmpfs' a ticket to support tmpfs is open on the bugtracker. didn't implement

BLOCKDATA='/dev/sda1 raw boot ext2;yes;/boot;target;no_opts;no_label;noatime,nodiratime /dev/sda2 raw swap swap;yes;no_mountpoint;target;no_opts;no_label;no_params /dev/sda3 raw root ext4;yes;/;target;-m 1;no_label;relatime /dev/sda5 raw usr ext4;yes;/usr;target;-m 1;no_label;relatime,nodev /dev/sda6 raw home ext4;yes;/home;target;-m 0;no_label;relatime,nodev,nosuid' '/tmp tmpfs relatime,nodev,nosuid,noexec,size=75% 0 0' # The '-m 0' is the tune2fs command for setting the RESERVED_BLOCKS (default is 5%) These are options for mk* (mke2fs in the case of ext. it supports -m). I don't think (i would need to read the source) you can put mount

yeah, we need a var for hostname this globally. put the stuff from GROUPS into /etc/group directly (as is, no additional parsing), so something like: GROUPS='audio::92:dieter,mpd optical::93:hal floppy:x:94:hal storage:x:95:hal' though i'm not sure how we would handle conflicts with groups already being added by packages (i don't want duplicates). writing code that parses this and checks for duplicates, checking available gids etc seems a bit over the top to me. this yet. options here.

# /dev/sda{5,6} would be logical partitions # Could this be expaned to work with RAID? i don't care much about raid. feel free to implement.

# /etc conf files - from manual installation menu MKINITCPIO= MODPROBE="# Disable IPv6 alias net-pf-10 off alias ipv6 off" RESOLV= HOSTS="192.168.0.1 Firewall" HOSTS_ALLOW="sshd: 192.168.0.0/255.255.255.0" RC_LOCAL="# Syncing Clock if ifconfig | grep eth0 &> /dev/null ; then ntpd -s -d &> /dev/null #ntpdate 192.168.0.1 &> /dev/null fi"

mm yeah.. maybe. see concluding note.

# These are user files added to the USB Stick/Disc and transferred after installation but before reboot UPDATE_CONFIG_FILES=.bashrc /root/.bashrc /etc/ssh/sshd_config /etc/X11/xorg.conf # the .bashrc would be added to each created user .bashrc -> use linux skeleton directory before making users. other files -> see concluding note.

COPY_FILES=some_printer.ppd # $ cp some_printer.ppd /usr/share/cups/model/ welcome.txt, group_bookmarks.html # these would be copied to each user's home # could be many files with target dir

I don't think i'll add parameters where you can specify files/folders to copy. remember that you can add worker functions easily (by defining them in the profile), where you can do stuff like this.

AUTO_REBOOT=yes # yes|no|wait xx

not in the scope of aif. call aif && reboot if you want this.

POST_INSTALLATON_SCRIPT= # This could be a .sh script run only on first reboot of new system - ideas: alsa-config, pacman -Sy, db update, setup printer, some tests, email admin, install more packages, etc

[End of TestMachine] # could be something else or nothing

[BuildMachine] # Repeated from above; adjust options

[Laptop] # Repeated from above; adjust options

[Server] # Repeated from above; adjust options

[VirtualTest] # Repeated from above; adjust options

# Another idea would be to have a [Default] profile then each following profile would just list the specific alterations

# End of File i'm not pro this. see earlier for my thoughts over profiles. maybe i could make a "depend" statement like i already do for

yeah good idea libraries/procedures where you can depend on an other profile. (which basically means to source an other file first). this gives you what you want, it's easy to implement. everybody happy :) Concluding notes about additional config files: many times a user could want to specify some customisations for certain config files. you propose a lot of these variables which will just make aif parse the variables, generate some output and append/replace into a certain config file. I'm just wondering if it's better to implement that for a lot of files (/etc/group, mkinitcpio.conf, ...) or just let a user keep such files in a svn repo or something and to just check them out. i guess it's a question of "let the user maintain "state" (or even generate: in that case putting it in aif would make more sense) himself (the config files themselves), or let aif generate the "state" files from input "metadata". I think letting aif do this may be better, though it would be quite some work, and you lack the "feedback loop" (eg now i can just commit my config files into svn and aif will check them out on the next install), with your suggested things, you need to update the aif profile yourself. I'll need to think a bit about this. :) Dieter