- Arch-releng - lists.archlinux.org

[arch-releng] who wants to write me a relatively simple webapp?
by Dieter Plaetinck 07 Feb '11

07 Feb '11

Hi, for Arch releng, we have recently started automatically building test builds (http://releng.archlinux.org/isos/) These images are built from the archiso and aif git repositories, and the current state of the repos. The idea is that people can test these images, and once in a while, I will promote a set of testbuilds and make them the officially supported media. Because everything evolves all the time, a lot of different things can cause regressions or new bugs, but at the same time it would be too much work to try to test all possible scenarios for each specific testbuild version. So I would like a web application that gives me "a pretty good idea" of the quality of current/recent images. I request someone other then me to make this app for me, I do not have the time. (I do have time for feedback or making adjustments to the codebase you provide) I'm not sure which is best, php or python, or whatever. I think both will be fine (Dan? Pierre?) Basically it would have 2 modes: 1) the "input" mode (after logging in with your wiki or forum account, not sure what's best here. Dan?): the user would select a version (maintain a cached list of the directories at http://releng.archlinux.org/isos/ the format will always be YYYY.MM.DD[suffix]) and select a list of options which are verified to be working fine. that is: (between braces are additional instructions for the user) = image arch = * dual, option i686 * dual, option x86_64 * i686 * x86_64 = image type = * core * net = image boot = * optical * usb * pxe = hardware type = * virtualbox * qemu * physical intel i686 * physical intel x86_64 * physical amd i686 * physical amd x86_64 = install type = * automatic install generic example * automatic install fancy example * automatic install custom config (specify in comments) * interactive install = source selection = * net install manual networking config (check that it works + rc.conf, resolv.conf, mirrorlist) * net install dhcp (check that it works + rc.conf) * core = clock = * left as is * reconfigured manually * reconfigured ntp = partitioning/filesystems = * autoprepare (check the installed system, incl fstab) * manual * from config file = fancy stuff (checklist, not selection) = * lvm2 * dm_crypt * softraid * nilfs2 * btrfs (check also fstab, menu.lst, initcpio.conf and crypttab if appropriate) = rollback tested = * yes * no = if rollback done, another partitioning/filesystems list and another "fancy stuff" list (same options as before) for after the rollback = = bootloader = * grub * syslinux * other/manual = result = * everything works fine * something failed (elaborate in comments) = comments = <textarea> Then, a second interface would need to represent the "current state of things.. (sort of)" that is, for every important thing (like, all image arches, image types, image boots, install types, source selections, clock, partitioning/filesystems, fancy stuff, rollback tested, grub/syslinux bootloader) it needs to show the last recent tests (or the last images for which it worked), along with which user tested it. (note: not all combinations of all variables, just any test with the given variable). if there was any failed test for a specific feature for an as recent (or more recent) image then the last successful one, there should be a warning about that. I'm not sure yet how exactly this page would look like, but basically I need to know all important features are tested with recent images, and that they all performed well. other factors (like hardware type, bootloader other/manual bootloader option) are not very important, they are more for reference. Thanks, Dieter

1 0

[arch-releng] Patch for archiso-pxe-server
by Charles Vejnar 07 Feb '11

07 Feb '11

Hi, I am proposing a patch for archiso-pxe-server. - It adds one option to avoid killing and strating dnsmasq (usefull if you already have one). - It corrects one important bug: with "set -e", the pkill are failing (no previous dnsmasq or nbd-server) and the program stops instead of continuing as it should. Thanks Charles 12a13 > DNSMASQ_SKIP=false 46a48,49 > echo " -k Skip Dnsmasq start up" > echo " (default: ${DNSMASQ_SKIP} )" 80c83 < pkill dnsmasq > /dev/null 2>&1 --- > pkill dnsmasq > /dev/null 2>&1 || true 91c94 < pkill nbd-server > /dev/null 2>&1 --- > pkill nbd-server > /dev/null 2>&1 || true 130c133 < while getopts 'i:s:d:l:m:t:h' arg; do --- > while getopts 'i:s:d:l:m:t:kh' arg; do 137a141 > k) DNSMASQ_SKIP=true ;; 145c149,151 < start_pxe_server --- > if ! ${DNSMASQ_SKIP} ; then > start_pxe_server > fi

2 2

[arch-releng] AIF talk at Fosdem 2011
by Dieter Plaetinck 07 Feb '11

07 Feb '11

I gave an AIF (related) talk at Fosdem this year. For more info, video and slides: http://dieter.plaetinck.be/can_we_build_a_simple_cross-distribution_install… Dieter

1 0

[arch-releng] nbd problem with archlinux-2011.02.01
by Charles Vejnar 04 Feb '11

04 Feb '11

Hi, I tried the new archlinux-2011.02.01-core-x86_64.iso but it failed after saying "Waiting for boot device" and "Waiting 30 seconds for device /dev/nbd0". I am booting through PXE. This step was working with archlinux-2011.01.28-core-x86_64.iso. I don't know exactly what changed between the two versions (except the kernel version). Do you have an idea what could be the problem ? Thanks Charles

2 2

[arch-releng] [PATCH] [syslinux-iso] Add automated_script support
by Charles Vejnar 03 Feb '11

03 Feb '11

Hi, Sorry for the delay. > - it checks if the kernel has a "script" argument (as given by the > bootloader - never in the official builds - You mean this option we are working on won't be usable with the next official build ?! > > Ideally, I think the script would set up auto login as root, and then put the aif invocation in /root/.bash_profile, or something like that. > > But it's probably too late to set up auto login as root, because that needs to be done by modifying /etc/inittab, but init is already running... > This sound much better, running from /root/.bash_profile. Or just having > a modified inittab that runs agetty with automatic login. Well in my opinion, automatic login should be always activated or not at all. The patch I proposed is designed for automatic install and is working for this purpose. But if you want to introduce root autologin, and put something in .bash_profile to autostart a manual or a automatic install, it is a good idea and not only for the case of an automatic install. I would really appreciate to be able to use this script option without modifying the iso. In my opinion, always activated auto-login would be great. Should I propose patches ? > I guess we/Charles will just need to try a few approaches and see what works best. Yes, sure. No problem. Charles

2 1

[arch-releng] [testiso 2011.01.28] test experience
by Thomas Dziedzic 01 Feb '11

01 Feb '11

tested another install iso: source: netinstall in virtualbox using eth0 with dhcp mirror: http://mirrors.kernel.org/ editor: vi time: chicago, with utc hwclock partitions: /boot 200mb; swap 200mb; / rest picked mount by uuid filesystems: /boot ext2; swap swap; / reiserfs package selection: base group installation: no problems configs: left them vanilla except changing the hostname package selection still suffers from https://bugs.archlinux.org/task/22428 FS#22428 - [test iso 2011.01.09] aif package selection should be sorted more naturally. I didn't try btrfs so I can't comment on FS#22427 - [test iso 2011.01.09] aif fails to ask for mountpoint when creating btrfs partition. booted fine. That's all. Thanks!

1 0

[arch-releng] [PATCH] [syslinux-iso] Add automated_script support
by Gerardo Exequiel Pozzi 30 Jan '11

30 Jan '11

Via kernel parameter "script=/path/to/script". /path/to/script can be a ftp:// or http:// or an absolute path. Original idea by "Charles", I just changes few things to use initscripts hooks. https://bbs.archlinux.org/viewtopic.php?id=111925 Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386(a)yahoo.com.ar> --- .../overlay/etc/rc.d/functions.d/automated_script | 21 ++++++++++++++++++++ 1 files changed, 21 insertions(+), 0 deletions(-) create mode 100644 configs/syslinux-iso/overlay/etc/rc.d/functions.d/automated_script diff --git a/configs/syslinux-iso/overlay/etc/rc.d/functions.d/automated_script b/configs/syslinux-iso/overlay/etc/rc.d/functions.d/automated_script new file mode 100644 index 0000000..4637ee1 --- /dev/null +++ b/configs/syslinux-iso/overlay/etc/rc.d/functions.d/automated_script @@ -0,0 +1,21 @@ +. /etc/archiso/functions + +automated_script () +{ + script="$(cmdline_param script)" + if [ -n $script ] ; then + stat_busy "Running $script" + if [[ $script =~ ^http:// || $script =~ ^ftp:// ]] ; then + wget "$script" -q -O /tmp/startup_script >/dev/null + else + cp $script /tmp/startup_script + fi + if [ -s /tmp/startup_script ] ; then + chmod +x /tmp/startup_script + /tmp/startup_script + fi + stat_done + fi +} + +add_hook multi_end automated_script -- 1.7.3.5

2 5

[arch-releng] [PATCH] Use globbing and parameter expansion to get the raid slave devices
by pyther＠pyther.net 29 Jan '11

29 Jan '11

Uses bash globbing and parameter expansion to find all of the slaves for a raid device. This is a much better method then using ls. Also, by looking at DEVNAME in the uevent file we provide support for block device that are not in the root of /dev. --- src/core/libs/lib-blockdevices-filesystems.sh | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diff --git a/src/core/libs/lib-blockdevices-filesystems.sh b/src/core/libs/lib-blockdevices-filesystems.sh index 021f41b..7731d9f 100644 --- a/src/core/libs/lib-blockdevices-filesystems.sh +++ b/src/core/libs/lib-blockdevices-filesystems.sh @@ -943,10 +943,14 @@ mdraid_slave0 () # ex: /dev/md0 has slaves: "/dev/sda1 /dev/sdb2 /dev/sdc2" mdraid_all_slaves () { + shopt -s nullglob local slave= - local slaves= - for slave in $(ls /sys/class/block/$(basename $1)/slaves/); do - slaves=$slaves"/dev/"$slave" " + local slaves= + for slave in /sys/class/block/${1##*/}/slaves/*; do + source "$slave/uevent" + slaves="$slaves/dev/$DEVNAME " + unset DEVNAME done + shopt -u nullglob echo $slaves } -- 1.7.3.5

3 2

[arch-releng] [PATCH] Moved kernel parameter generation into get_kernel_parameters
by pyther＠pyther.net 29 Jan '11

29 Jan '11

The function now stores parameters into var kernel_parameters. The new function does not set the kernel name (vmlinuz). This must now be done in the bootloader configuration code. --- src/core/libs/lib-ui-interactive.sh | 134 +++++++++++++++++++---------------- 1 files changed, 74 insertions(+), 60 deletions(-) diff --git a/src/core/libs/lib-ui-interactive.sh b/src/core/libs/lib-ui-interactive.sh index 829556a..0449d99 100644 --- a/src/core/libs/lib-ui-interactive.sh +++ b/src/core/libs/lib-ui-interactive.sh @@ -1048,76 +1048,27 @@ generate_grub_menulst() { # keep the file from being completely bogus [ "$grubdev" = "DEVICE NOT FOUND" ] && grubdev= if [ -z "$grubdev" ]; then - notify "Your root boot device could not be autodetected by setup. Ensure you adjust the 'root (hd0,0)' line in your GRUB config accordingly." - grubdev="(hd0,0)" - fi - # remove default entries by truncating file at our little tag (#-*) - sed -i -e '/#-\*/q' $grubmenu - - # find label or uuid of the root partition - case $PART_ACCESS in - label) - local _label="$(getlabel $_rootpart)" - if [ -n "${_label}" ]; then - _rootpart="/dev/disk/by-label/${_label}" - fi - ;; - uuid) - local _uuid="$(getuuid $_rootpart)" - if [ -n "${_uuid}" ]; then - _rootpart="/dev/disk/by-uuid/${_uuid}" - fi - ;; - esac + notify "Your root boot device could not be autodetected by setup. Ensure you adjust the 'root (hd0,0)' line in your GRUB config accordingly." + grubdev="(hd0,0)" + fi - # handle dmraid/mdadm,lvm,dm_crypt etc. replace entries where needed automatically - kernel="kernel $subdir/vmlinuz26 root=${_rootpart} ro" - if get_anchestors_mount ';/;' - then - if echo "$ANSWER_DEVICES" | sed -n '1p' | grep -q 'dm_crypt$' && echo "$ANSWER_DEVICES" | sed -n '2p' | grep -q 'raw$' - then - debug 'FS' 'Grub kernel line? Found / on dm_crypt on raw' - raw_device=`echo "$ANSWER_DEVICES" | sed -n '2p' | cut -d ' ' -f1` - crypt_device=`echo "$ANSWER_DEVICES" | sed -n '1p' | cut -d ' ' -f1` - kernel="kernel $subdir/vmlinuz26 root=$crypt_device cryptdevice=$raw_device:`basename $crypt_device` ro" - elif echo "$ANSWER_DEVICES" | sed -n '1p' | grep -q 'lvm-lv$' && echo "$ANSWER_DEVICES" | sed -n '4p' | grep -q 'dm_crypt$' && echo "$ANSWER_DEVICES" | sed -n '5p' | grep -q 'raw$' - then - debug 'FS' 'Grub kernel line? Found / on lvm on dm_crypt on raw' - lv_device=`echo "$ANSWER_DEVICES" | sed -n '1p' | cut -d ' ' -f1` - crypt_device=`echo "$ANSWER_DEVICES" | sed -n '4p' | cut -d ' ' -f1` - raw_device=`echo "$ANSWER_DEVICES" | sed -n '5p' | cut -d ' ' -f1` - kernel="kernel $subdir/vmlinuz26 root=$lv_device cryptdevice=$raw_device:`basename $crypt_device` ro" - elif echo "$ANSWER_DEVICES" | sed -n '1p' | grep -q 'dm_crypt$' && echo "$ANSWER_DEVICES" | sed -n '2p' | grep -q 'lvm-lv$' && echo "$ANSWER_DEVICES" | sed -n '5p' | grep -q 'raw$' - then - debug 'FS' 'Grub kernel line? Found / on dm_crypt on lvm on raw' - crypt_device=`echo "$ANSWER_DEVICES" | sed -n '1p' | cut -d ' ' -f1` - lv_device=`echo "$ANSWER_DEVICES" | sed -n '2p' | cut -d ' ' -f1` - kernel="kernel $subdir/vmlinuz26 root=$crypt_device cryptdevice=$lv_device:`basename $crypt_device` ro" - elif echo "$ANSWER_DEVICES" | sed -n '1p' | grep -q 'raw$' - then - debug 'FS' 'Grub kernel line? Found / on raw' - elif echo "$ANSWER_DEVICES" | sed -n '1p' | grep -q 'lvm-lv$' && echo "$ANSWER_DEVICES" | sed -n '4p' | grep -q 'raw$' - then - debug 'FS' 'Grub kernel line? Found / on lvm on raw' - else - debug 'FS' 'Grub kernel line? Could not figure this one out' - show_warning "Disk setup detection" "Are you using some really fancy dm_crypt/lvm/softraid setup?\nI could not figure it out, so the kernel line will be the default: you'll probably need to edit it.\nPlease file a bug for this and supply all files from /tmp/aif/" - fi - else - show_warning "Disk setup detection" "Could not find out where your / is. Did you setup filesystems/blockdevices? manual/autoprepare? If yes, please file a bug and tell us about this" - fi - cat >>$grubmenu <<EOF + # remove default entries by truncating file at our little tag (#-*) + sed -i -e '/#-\*/q' $grubmenu + + get_kernel_parameters + + cat >>$grubmenu <<EOF # (0) Arch Linux title Arch Linux root $grubdev -$kernel +kernel $subdir/vmlinuz26 $kernel_parameters initrd $subdir/kernel26.img # (1) Arch Linux title Arch Linux Fallback root $grubdev -$kernel +kernel $subdir/vmlinuz26 $kernel_parameters initrd $subdir/kernel26-fallback.img # (2) Windows @@ -1179,6 +1130,69 @@ EOF fi } +get_kernel_parameters() +{ + get_device_with_mount '/' || return 1 + local _rootpart="$ANSWER_DEVICE" + + case "$PART_ACCESS" in + label) + local _label="$(getlabel $_rootpart)" + if [[ -n "${_label}" ]]; then + _rootpart="/dev/disk/by-label/${_label}" + fi + ;; + uuid) + local _uuid="$(getuuid $_rootpart)" + if [[ -n "${_uuid}" ]]; then + _rootpart="/dev/disk/by-uuid/${_uuid}" + fi + ;; + esac + + kernel_parameters="root=$_rootpart ro" + + # No seperate /boot partition + if [[ -z "$bootdev" ]]; then + subdir='/boot' + fi + + if get_anchestors_mount ';/;' + then + if echo "$ANSWER_DEVICES" | sed -n '1p' | grep -q 'dm_crypt$' && echo "$ANSWER_DEVICES" | sed -n '2p' | grep -q 'raw$' + then + debun 'FS' 'get_kernel_options - Found / on dm_crypt on raw' + raw_device="$(echo "$ANSWER_DEVICES" | sed -n '2p' | cut -d ' ' -f1)" + crypt_device="$(echo "$ANSWER_DEVICES" | sed -n '1p' | cut -d ' ' -f1)" + kernel_parameters="root=$crypt_device cryptdevice=$raw_device:`basename $crypt_device` ro" + elif echo "$ANSWER_DEVICES" | sed -n '1p' | grep -q 'lvm-lv$' && echo "$ANSWER_DEVICES" | sed -n '4p' | grep -q 'dm_crypt$' && echo "$ANSWER_DEVICES" | sed -n '5p' | grep -q 'raw$' + then + debug 'FS' 'get_kernel_options - Found / on lvm on dm_crypt on raw' + lv_device="$(echo "$ANSWER_DEVICES" | sed -n '1p' | cut -d ' ' -f1)" + crypt_device="$(echo "$ANSWER_DEVICES" | sed -n '4p' | cut -d ' ' -f1)" + raw_device="$(echo "$ANSWER_DEVICES" | sed -n '5p' | cut -d ' ' -f1)" + kernel_parameters="root=$lv_device cryptdevice=$raw_device:`basename $crypt_device` ro" + elif echo "$ANSWER_DEVICES" | sed -n '1p' | grep -q 'dm_crypt$' && echo "$ANSWER_DEVICES" | sed -n '2p' | grep -q 'lvm-lv$' && echo "$ANSWER_DEVICES" | sed -n '5p' | grep -q 'raw$' + then + debug 'FS' 'get_kernel_options - Found / on dm_crypt on lvm on raw' + crypt_device="$(echo "$ANSWER_DEVICES" | sed -n '1p' | cut -d ' ' -f1)" + lv_device="$(echo "$ANSWER_DEVICES" | sed -n '2p' | cut -d ' ' -f1)" + kernel_parameters="root=$crypt_device cryptdevice=$lv_device:`basename $crypt_device` ro" + elif echo "$ANSWER_DEVICES" | sed -n '1p' | grep -q 'raw$' + then + debug 'FS' 'get_kernel_parameters - Found / on raw' + elif echo "$ANSWER_DEVICES" | sed -n '1p' | grep -q 'lvm-lv$' && echo "$ANSWER_DEVICES" | sed -n '4p' | grep -q 'raw$' + then + debug 'FS' 'get_kernel_parameters - Found / on lvm on raw' + else + debug 'FS' 'get_kernel_parameters - Could not figure this one out' + show_warning "Disk setup detection" "Are you using some really fancy dm_crypt/lvm/softraid setup?\nI could not figure it out, so the kernel line will be the default: you'll probably need to edit it.\nPlease file a bug for this and supply all files from /tmp/aif/" + fi + else + show_warning "Disk setup detection" "Could not find out where your / is. Did you setup filesystems/blockdevices? manual/autoprepare? If yes, please file a bug and tell us about this" + fi +} + # displays installation source selection menu # and sets up relevant config files -- 1.7.3.5

2 1

[arch-releng] [PATCH 1/2] Renamed mdraid_is_raid to device_is_raid
by pyther＠pyther.net 29 Jan '11

29 Jan '11

device_is_raid now uses dev major number to determine if a device is a raid device. A major ID of 8 is a /dev/sd? device whereas a major ID of 9 is a 'metadata' (raid) device. call die_error if a block device is not passed into device_is_raid() --- src/core/libs/lib-blockdevices-filesystems.sh | 21 +++++++-------------- src/core/libs/lib-ui-interactive.sh | 2 +- 2 files changed, 8 insertions(+), 15 deletions(-) diff --git a/src/core/libs/lib-blockdevices-filesystems.sh b/src/core/libs/lib-blockdevices-filesystems.sh index 8573b19..021f41b 100644 --- a/src/core/libs/lib-blockdevices-filesystems.sh +++ b/src/core/libs/lib-blockdevices-filesystems.sh @@ -919,20 +919,13 @@ get_blockdevice_size () # $1 blockdevice (ex: /dev/md0 or /dev/sda1) -# return true when blockdevice is an md raid, otherwise return a unset value -mdraid_is_raid() -{ - local israid - if [ -z $1 ]; then - # Don't call mdadm on empty blockdevice parameter! - israid="" - elif [ "$(mdadm --query $1 | cut -d':' -f2)" == " is not an md array" ]; then - israid="" - else - israid=true - fi - echo $israid -} +# All MD RAID block devices have a major id of 9 +device_is_raid() { + [[ -b "$1" ]] || die_error "device_is_raid needs a blockdevice as \$1 ($1 given)" + [[ -f /proc/mdstast ]] || return 1 + local devmajor=$(stat -c %t "$1") + (( devmajor == 9 )) +} # $1 md raid blockdevice (ex: /dev/md0) # return the array member device which is slave 0 in the given array diff --git a/src/core/libs/lib-ui-interactive.sh b/src/core/libs/lib-ui-interactive.sh index 881af90..829556a 100644 --- a/src/core/libs/lib-ui-interactive.sh +++ b/src/core/libs/lib-ui-interactive.sh @@ -878,7 +878,7 @@ interactive_grub() { bootdev=$(mount | grep $var_TARGET_DIR/boot | cut -d' ' -f 1) # check if bootdev or PART_ROOT is on a md raid array # This dialog is only shown when we detect / or /boot on a raid device. - if [ -n "$(mdraid_is_raid $bootdev)" -o -n "$(mdraid_is_raid $PART_ROOT)" ]; then + if device_is_raid $bootdev || device_is_raid $PART_ROOT; then ask_yesno "Do you have your system installed on software raid?\nAnswer 'YES' to install grub to another hard disk." no if [ $? -eq 0 ]; then onraid=true -- 1.7.3.5

2 3