[arch-projects] [mkinitcpio][PATCH 00/19] Break all the things!
Hi all, What follows this is the changes I have in store for the next release, which I'll be tagging 0.9.0. There's one change not included, but I'll explain what that is. I realize this patchset is huge, and I don't expect anyone to look through it in its entirety. However, there's two things worth pointing out here that I'd appreciate some feedback on: * Changes to install hook API I've added some new functions: - add_checked_modules - add_all_modules - add_runscript These are intended to replace all_modules and checked_modules, which currently return the list of modules that the caller should add. The new functions simply go ahead and queue the modules for addition rather than making the caller do something with them. add_runscript does some introspection to figure out which hook called it and add the appropriate script from /usr/lib/initcpio/hooks. The general idea here is that we're getting rid of using MODULES, FILES, BINARIES, and SCRIPT in the install hooks and we'll be down to a singular method of adding files to the image. With the refactored error parsing, mkinitcpio can properly detect errors from any add_* function, which should improve the experience a bit. Consider the next release the deprecation warning, though I'm not sure when I'll remove things. I've made sure that current hooks continue to work in the same way. There's no explicit callout during runtime about these things being deprecated, but maybe there should be? I'll plan on fixing all the hooks in the repos myself. Since this is new API, I'd like to get it right the first time. If anyone sees anything lacking, room for improvement, or wants to send me death threats, feel free. * Support for late running hooks I've refactored running hooks in early userspace and we now support a second run after root has been mounted. To make things simple, current hook files only need to define an extra function run_latehook() to be considered. Hooks run in reverse order from the normal run. lsinitcpio of course gains support for reading these. We already have some obvious candidates to use this feature: udev and shutdown. udev startup/shutdown no longer needs to be hardcoded in /init, and shutdown can delete modules before copying / over to /run/initramfs, saving some modicum of space in tmpfs. I'm also tempted to move mounting of /usr to a late running hook, but I'm sure that we'll see some breakage by doing this, even with proper warnings. Opinions here welcome. The final unlisted change is what I referred to earlier about udev. Since the udev hook would be handling its own startup/shutdown, we'd simply yank those if blocks out of /init and carry on. I've been testing this and it works quite well. Random feedback and tomatoes unrelated to the above points are welcome, as always. Cheers, d Dave Reisner (19): use MODULEDIR as reference for *.(order|builtin) files mkinitcpio: separate logic to resolve kernel version document special kernel cmdline parameters refactor error tracking in build hooks mmc: package as mode 644, not 755 functions: remove get_dirname and get_basename functions: move --try logic into add_module move -g option checking to parseopts loop functions: introduce add_checked_modules and add_all_modules functions: introduce add_runscript use new API for install hooks init: correct trimming of earlymodules and MODULES init_functions: move running hooks to separate func init: add support for late running hooks shutdown: package as 644, install as 755 shutdown: convert to late hook buildsys: fix clean target lsinitcpio: extract the image to a tempdir lsinitcpio: add support for listing late hooks Makefile | 2 +- functions | 125 +++++++++++++++++++++++++++++++++++++-------------- hooks/shutdown | 3 +- init | 26 +++-------- init_functions | 27 +++++++++++ install/consolefont | 6 +-- install/filesystems | 4 +- install/fsck | 4 ++ install/fw | 8 +++- install/ide | 4 +- install/keymap | 4 +- install/memdisk | 9 ++-- install/mmc | 7 ++- install/pata | 6 ++- install/resume | 2 +- install/sata | 6 ++- install/scsi | 9 ++-- install/shutdown | 7 +-- install/sleep | 2 +- install/usb | 12 +++-- install/usbinput | 5 ++- install/virtio | 2 +- lsinitcpio | 70 +++++++++++++++++++++-------- mkinitcpio | 59 ++++++++++++++---------- mkinitcpio.8.txt | 51 +++++++++++++++++++++ 25 files changed, 317 insertions(+), 143 deletions(-) mode change 100755 => 100644 install/mmc mode change 100755 => 100644 shutdown -- 1.7.10.2
--- mkinitcpio | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkinitcpio b/mkinitcpio index 9a74a8b..4fb48e4 100755 --- a/mkinitcpio +++ b/mkinitcpio @@ -400,7 +400,7 @@ if (( ${#ADDED_MODULES[*]} )); then msg "Generating module dependencies" install -m644 -t "$BUILDROOT/usr/lib/modules/$KERNELVERSION" \ - "/lib/modules/$KERNELVERSION"/modules.{builtin,order} + "$MODULEDIR"/modules.{builtin,order} depmod -b "$BUILDROOT" "$KERNELVERSION" # remove all non-binary module.* files (except devname for on-demand module loading) -- 1.7.10.2
Rename get_kernelver -> resolve_kernelver, and ensure that it only does what it's name advertises. Introduce find_moduledir to locate the corresponding module directory in /usr/lib or /lib. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- mkinitcpio | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/mkinitcpio b/mkinitcpio index 4fb48e4..74de0c2 100755 --- a/mkinitcpio +++ b/mkinitcpio @@ -73,7 +73,7 @@ cleanup() { exit ${1:0} } -get_kernver() { +resolve_kernver() { local kernver= kernel=$1 if [[ -z $kernel ]]; then @@ -87,17 +87,30 @@ get_kernver() { fi if [[ ! -e $kernel ]]; then - error "Specified kernel image does not exist: \`%s'" "$kernel" + error "specified kernel image does not exist: \`%s'" "$kernel" return 1 fi - read _ kernver < <(file -Lb "$kernel" | grep -o 'version [^ ]\+') - if [[ $kernver && -e /lib/modules/$kernver ]]; then - echo "$kernver" + if file -Lb "$kernel" | grep -oP '(?<=version )[^ ]+'; then return 0 fi - error "invalid kernel specifier: \`%s'" "$optkver" + error "invalid kernel specified: \`%s'" "$optkver" + + return 1 +} + +find_moduledir() { + local d + + for d in {/usr,}/lib/modules; do + if [[ -d $d/$1/ ]]; then + printf '%s' "$d/$1/" + return 0 + fi + done + + error "unable to locate module directory for kernel \`%s'" "$1" return 1 } @@ -219,7 +232,7 @@ readonly NC BOLD BLUE GREEN RED YELLOW [[ -e /proc/self/mountinfo ]] || die "/proc must be mounted!" [[ -e /dev/fd ]] || die "/dev must be mounted!" -KERNELVERSION=$(get_kernver "$optkver") || cleanup 1 +KERNELVERSION=$(resolve_kernver "$optkver") || cleanup 1 if [[ $TMPDIR ]]; then if [[ ! -d $TMPDIR ]]; then @@ -311,7 +324,8 @@ if (( ${#hooks[*]} == 0 )); then die "Invalid config: No hooks found" fi -MODULEDIR=/lib/modules/$KERNELVERSION/ +MODULEDIR=$(find_moduledir "$KERNELVERSION") || cleanup 1 + if [[ ! -d $MODULEDIR ]]; then die "'$MODULEDIR' is not a valid kernel module directory" fi -- 1.7.10.2
Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- install/fsck | 4 ++++ mkinitcpio.8.txt | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/install/fsck b/install/fsck index e8b5ea3..eefd451 100644 --- a/install/fsck +++ b/install/fsck @@ -41,6 +41,10 @@ operation on the root device prior to mounting. If the autodetect hook is used, only the fsck helper specific to your filesystem will be added to the image. It is highly recommended that if you include this hook that you also include any necessary modules to ensure your keyboard will work in early userspace. + +To control the behavior of fsck on bootup, fsck.mode=force can be passed on +the kernel command line to insist on running a full filesystem check. Similar,y +fsck.mode=skip can be passed to cause fsck not to run at all. HELPEOF } diff --git a/mkinitcpio.8.txt b/mkinitcpio.8.txt index f947514..070cfeb 100644 --- a/mkinitcpio.8.txt +++ b/mkinitcpio.8.txt @@ -98,6 +98,57 @@ you generate a new initial ramdisk, you define a preset and use the -p switch to generate an initial ramdisk according to your preset. Presets are located in /etc/mkinitcpio.d +Early Init Environment +---------------------- +mkinitcpio gives special treatment to certain environment variables passed on +the kernel command line: + +*break*['=<premount|postmount>']:: + If specified, mkinitcpio will start a shell during early init. The optional + parameter controls when this occurs: when 'premount' or no parameter are + specified, the shell will be launched prior to mounting root. If 'postmount' + is specified, the shell will be launched after mounting root. + +*disablehooks=*'hooklist':: + This is a comma separated list of hooks which will be skipped during early + init. + +*earlymodules=*'modulelist':: + This is a comma separated list of modules which will be loaded prior to any + others. This is generally not needed, and usually points to a configuration + or kernel problem. + +*quiet*:: + Causes mkinitcpio to output fewer messages during boot. Errors will not be + suppressed. + +*ro*:: + Specifies that root should be mounted with readonly permissions. This is the + default behavior. + +*rw*:: + Specifies that root should be mounted with readwrite permissions. This is + generally only useful if your initramfs uses the 'fsck' hook. + +*root=*'rootdevice':: + This variable describes the root partition which early init will mount + before passing control to the real init. mkinitcpio understands a variety of + formats, the most basic of which is the path to the block device, either + directly, such as '/dev/sda2', or using a udev symlink such as + '/dev/disk/by-label/CorsairF80-root'. Support for identification by LABEL or + UUID tags are also supported, such as, 'LABEL=CorsairF80-root'. As of + util-linux 2.22, PARTUUID is also supported. Identification via hex encoded + major/minor device ID is supported for legacy reasons, but should not be used. + +*rootdelay=*'seconds':: + Sets the delay, in seconds, that mkinitcpio is willing to wait for the root + device to show up, if it is not available immediately. This defaults to 5 + seconds. If an invalid integer is passed, this variable will have no effect. + +These are only the variables that the core of mkinitcpio honor. Additional +hooks may look for other environment variables and should be documented by the +help output for the hook. + Files ----- '/etc/mkinitcpio.conf':: -- 1.7.10.2
On Sun, May 13, 2012 at 1:57 PM, Dave Reisner <dreisner@archlinux.org> wrote:
Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- install/fsck | 4 ++++ mkinitcpio.8.txt | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+)
diff --git a/install/fsck b/install/fsck index e8b5ea3..eefd451 100644 --- a/install/fsck +++ b/install/fsck @@ -41,6 +41,10 @@ operation on the root device prior to mounting. If the autodetect hook is used, only the fsck helper specific to your filesystem will be added to the image. It is highly recommended that if you include this hook that you also include any necessary modules to ensure your keyboard will work in early userspace. + +To control the behavior of fsck on bootup, fsck.mode=force can be passed on +the kernel command line to insist on running a full filesystem check. Similar,y
Minor typo. Should be "Similarly".
+fsck.mode=skip can be passed to cause fsck not to run at all. HELPEOF }
diff --git a/mkinitcpio.8.txt b/mkinitcpio.8.txt index f947514..070cfeb 100644 --- a/mkinitcpio.8.txt +++ b/mkinitcpio.8.txt @@ -98,6 +98,57 @@ you generate a new initial ramdisk, you define a preset and use the -p switch to generate an initial ramdisk according to your preset. Presets are located in /etc/mkinitcpio.d
+Early Init Environment +---------------------- +mkinitcpio gives special treatment to certain environment variables passed on +the kernel command line: + +*break*['=<premount|postmount>']:: + If specified, mkinitcpio will start a shell during early init. The optional + parameter controls when this occurs: when 'premount' or no parameter are + specified, the shell will be launched prior to mounting root. If 'postmount' + is specified, the shell will be launched after mounting root. + +*disablehooks=*'hooklist':: + This is a comma separated list of hooks which will be skipped during early + init. + +*earlymodules=*'modulelist':: + This is a comma separated list of modules which will be loaded prior to any + others. This is generally not needed, and usually points to a configuration + or kernel problem. + +*quiet*:: + Causes mkinitcpio to output fewer messages during boot. Errors will not be + suppressed. + +*ro*:: + Specifies that root should be mounted with readonly permissions. This is the + default behavior. + +*rw*:: + Specifies that root should be mounted with readwrite permissions. This is + generally only useful if your initramfs uses the 'fsck' hook. + +*root=*'rootdevice':: + This variable describes the root partition which early init will mount + before passing control to the real init. mkinitcpio understands a variety of + formats, the most basic of which is the path to the block device, either + directly, such as '/dev/sda2', or using a udev symlink such as + '/dev/disk/by-label/CorsairF80-root'. Support for identification by LABEL or + UUID tags are also supported, such as, 'LABEL=CorsairF80-root'. As of + util-linux 2.22, PARTUUID is also supported. Identification via hex encoded + major/minor device ID is supported for legacy reasons, but should not be used. + +*rootdelay=*'seconds':: + Sets the delay, in seconds, that mkinitcpio is willing to wait for the root + device to show up, if it is not available immediately. This defaults to 5 + seconds. If an invalid integer is passed, this variable will have no effect. + +These are only the variables that the core of mkinitcpio honor. Additional +hooks may look for other environment variables and should be documented by the +help output for the hook. + Files ----- '/etc/mkinitcpio.conf':: -- 1.7.10.2
On Mon, May 14, 2012 at 12:18:23PM -0400, canyonknight@gmail.com wrote:
On Sun, May 13, 2012 at 1:57 PM, Dave Reisner <dreisner@archlinux.org> wrote:
Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- install/fsck | 4 ++++ mkinitcpio.8.txt | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+)
diff --git a/install/fsck b/install/fsck index e8b5ea3..eefd451 100644 --- a/install/fsck +++ b/install/fsck @@ -41,6 +41,10 @@ operation on the root device prior to mounting. If the autodetect hook is used, only the fsck helper specific to your filesystem will be added to the image. It is highly recommended that if you include this hook that you also include any necessary modules to ensure your keyboard will work in early userspace. + +To control the behavior of fsck on bootup, fsck.mode=force can be passed on +the kernel command line to insist on running a full filesystem check. Similar,y
Minor typo. Should be "Similarly".
Thanks for the catch. Fixed up on my working branch. d
Rather than catching errors solely from parse_hook via an ERR trap, implement a RETURN trap which catches all errors from the core add_* functions. In the future, this may mean that support for MODULES/FILES/BINARIES within hooks goes away, and those variables remain only in the config. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- functions | 9 +++++++-- mkinitcpio | 13 +++++++------ 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/functions b/functions index 5956dc3..bca31b4 100644 --- a/functions +++ b/functions @@ -160,6 +160,7 @@ warning() { error() { local mesg=$1; shift printf "$RED==> ERROR:$NC$BOLD $mesg$NC\n" "$@" >&2 + return 1 } die() { @@ -320,7 +321,10 @@ add_dir() { local path=$1 mode=${2:-755} - [[ -e $BUILDROOT$1 ]] && return 0 # file exists + if [[ -d $BUILDROOT$1 ]]; then + # ignore dir already exists + return 0 + fi (( QUIET )) || plain "adding dir: %s" "$path" command install -dm$mode "$BUILDROOT$path" @@ -398,7 +402,8 @@ add_binary() { # always add the binary itself add_file "$binary" "$dest" "$mode" - lddout=$(ldd "$binary" 2>/dev/null) || return 0 # not a binary! + # negate this so that the RETURN trap is not fired on non-binaries + ! lddout=$(ldd "$binary" 2>/dev/null) && return 0 # resolve sodeps regex='(/.+) \(0x[a-fA-F0-9]+\)' diff --git a/mkinitcpio b/mkinitcpio index 74de0c2..ce40b22 100755 --- a/mkinitcpio +++ b/mkinitcpio @@ -351,10 +351,10 @@ else msg "Starting build: %s" "$KERNELVERSION" fi -# set errtrace and a trap to catch errors in parse_hook +# set functrace and trap to catch errors in add_* functions declare -i builderrors=0 -set -E -trap '[[ $FUNCNAME = parse_hook ]] && (( ++builderrors ))' ERR +set -o functrace +trap '(( $? )) && [[ $FUNCNAME = add_* ]] && (( ++builderrors ))' RETURN # save vars from $CONFIG; they will be parsed last for var in MODULES BINARIES FILES; do @@ -402,7 +402,8 @@ for var in cfg_{MODULES,BINARIES,FILES}; do done parse_hook -# reset the trap to catch all errors +# switch out the error handler to catch all errors +trap -- RETURN trap '(( ++builderrors ))' ERR if (( ${#ADDED_MODULES[*]} )); then @@ -425,8 +426,8 @@ else fi # unset errtrace and trap -set +E -trap ERR +set +o functrace +trap -- ERR declare -i status=0 declare -a pipesave -- 1.7.10.2
Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- 0 files changed mode change 100755 => 100644 install/mmc diff --git a/install/mmc b/install/mmc old mode 100755 new mode 100644 -- 1.7.10.2
A lot has changed since 643e98eeb42677 when these functions were implemented, and there is exactly 1 case of get_dirname left, and no calls to get_basename. Remove these, and use a PE in place of the remaining call. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- functions | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/functions b/functions index bca31b4..5d986da 100644 --- a/functions +++ b/functions @@ -168,18 +168,6 @@ die() { cleanup 1 } -get_basename() { - local base=${1%/} - base=${base##*/} - printf '%s' "${base:-/}" -} - -get_dirname() { - local dir=${1%/} - dir=${dir%/*} - printf '%s' "${dir:-/}" -} - in_array() { # Search for an element in an array. # $1: needle @@ -337,7 +325,7 @@ add_symlink() { (( $# == 2 )) || return 1 - add_dir "$(get_dirname "$1")" + add_dir "${1%/*}" if (( ! QUIET )); then if [[ -L $BUILDROOT$1 ]]; then -- 1.7.10.2
Allow add_module to detect trailing ? characters for ignoring errors rather than passing -t or --try to add_module. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- functions | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/functions b/functions index 5d986da..052dced 100644 --- a/functions +++ b/functions @@ -232,11 +232,11 @@ add_module() { # $1: module name local module= path= dep= deps= field= value= - local -i ign_errors=0 + local ign_errors=0 - if [[ $1 = -@(t|-try) ]]; then + if [[ $1 = *\? ]]; then ign_errors=1 - shift + set -- "${1%?}" fi module=${1%.ko*} @@ -274,11 +274,18 @@ add_module() { MODPATHS+=("$path") ADDED_MODULES+=("${module//-/_}") - # explicit module depends - case "$module" in - fat) add_module --try "nls_cp437" ;; - ocfs2) add_module --try "configfs" ;; - libcrc32c) add_module --try "crc32c"; add_module --try "crc32c_intel" ;; + # handle module quirks + case $module in + fat) + add_module "nls_cp437?" + ;; + ocfs2) + add_module "configfs?" + ;; + libcrc32c) + add_module "crc32c_intel?" + add_module "crc32c?" + ;; esac } @@ -422,11 +429,7 @@ parse_hook() { local item= script= for item in $MODULES; do - if [[ ${item:(-1)} = '?' ]]; then - add_module --try "${item%\?}" - else - add_module "$item" - fi + add_module "$item" done for item in $BINARIES; do -- 1.7.10.2
Avoid any delay in checking this option, as it only serves to fragment the logic. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- mkinitcpio | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/mkinitcpio b/mkinitcpio index ce40b22..0cf17fb 100755 --- a/mkinitcpio +++ b/mkinitcpio @@ -154,7 +154,10 @@ while :; do SAVELIST=1 ;; -g|--generate) shift - GENIMG=$1 ;; + [[ -d $1 ]] && die "Invalid image path -- must not be a directory" + if ! GENIMG=$(readlink -f "$1") || [[ ! -e ${GENIMG%/*} ]]; then + die "Unable to write to path: \`%s'" "$1" + fi ;; -h|--help) usage cleanup 0 ;; @@ -304,13 +307,6 @@ if [[ $PRESET ]]; then fi fi -if [[ $GENIMG ]]; then - IMGPATH=$(readlink -f "$GENIMG") - if [[ -z $IMGPATH || ! -w ${IMGPATH%/*} ]]; then - die "Unable to write to path: \`%s'" "$GENIMG" - fi -fi - if [[ ! -f $CONFIG ]]; then die "Config file does not exist: \`%s'" "$CONFIG" fi @@ -441,7 +437,7 @@ if [[ $GENIMG ]]; then pushd "$BUILDROOT" >/dev/null find . -print0 | bsdcpio $( (( QUIET )) && echo '--quiet' ) -R 0:0 -0oH newc | - $COMPRESSION $COMPRESSION_OPTIONS > "$IMGPATH" + $COMPRESSION $COMPRESSION_OPTIONS > "$GENIMG" pipesave=("${PIPESTATUS[@]}") # save immediately popd >/dev/null -- 1.7.10.2
Instead of returning a list of modules which the caller should then add themselves or stuff into the MODULES string, call add_module for each candidate. This is basically a different wrapper around all_modules. DEPRECATION WARNING: This commit marks checked_modules as deprecated. Although it is not slated to go away, direct usage of all_modules is strongly discouraged. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- functions | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 50 insertions(+), 2 deletions(-) diff --git a/functions b/functions index 052dced..97b401c 100644 --- a/functions +++ b/functions @@ -199,12 +199,23 @@ auto_modules() { all_modules() { # Add modules to the initcpio, filtered by grep. # $@: filter arguments to grep + # -f FILTER: ERE to filter found modules local -i count=0 - local mod= + local mod= OPTIND= OPTARG= filter=() + + while getopts ':f:' flag; do + case $flag in f) filter+=("$OPTARG") ;; esac + done + shift $(( OPTIND - 1 )) while read -r -d '' mod; do (( ++count )) + + for f in "${filter[@]}"; do + [[ $mod =~ $f ]] && continue 2 + done + mod=${mod##*/} mod="${mod%.ko*}" printf '%s\n' "${mod//-/_}" @@ -213,11 +224,48 @@ all_modules() { (( count )) } -checked_modules() { +add_all_modules() { + # Add modules to the initcpio. + # $@: arguments to all_modules + + local mod mods + + mapfile -t mods < <(all_modules "$@") + + for mod in "${mods[@]}"; do + add_module "$mod" + done + + return $(( !${#mods[*]} )) +} + +add_checked_modules() { # Add modules to the initcpio, filtered by the list of autodetected # modules. # $@: arguments to all_modules + local mod mods + + if [[ -s $MODULE_FILE ]]; then + mapfile -t mods < <(all_modules "$@" | grep -xFf "$MODULE_FILE") + else + mapfile -t mods < <(all_modules "$@") + fi + + for mod in "${mods[@]}"; do + add_module "$mod" + done + + return $(( !${#mods[*]} )) +} + +checked_modules() { + # Returns a list of modules filtered by the list of autodetected modules. + # $@: arguments to all_modules + # + # DEPRECATED: Use add_checked_modules instead + # + if [[ -s $MODULE_FILE ]]; then grep -xFf "$MODULE_FILE" <(all_modules "$@") return 1 -- 1.7.10.2
This function adds a runtime script to the /hooks directory on the initramfs image. Note that this function will also install hooks with executable permissions for use by a later change to early init. With this commit, there are now methods available which can be used in place of the MODULES, FILES, BINARIES, and SCRIPT variables, as we now offer error checking on the add_* functions. Usage of the variables is deprecated, and these will no longer be read in a future version of mkinitcpio. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- functions | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/functions b/functions index 97b401c..ad24076 100644 --- a/functions +++ b/functions @@ -422,8 +422,23 @@ add_file() { command install -Dm$mode "$src" "$BUILDROOT$dest" } +add_runscript() { + # Adds a runtime script to the initcpio image. This function takes no + # parameters. The name is derived from the script which calls it and + # cannot be changed due to limitations in early userspace. + + local script hookname=${BASH_SOURCE[1]##*/} + + if ! script=$(find_in_dirs "$hookname" "${HOOKDIR[@]}"); then + error "runtime script for \`%s' not found" "$hookname" + return + fi + + add_file "$script" "/hooks/$hookname" 755 +} + add_binary() { - # add a binary file to the initcpio image. library dependencies will + # Add a binary file to the initcpio image. library dependencies will # be discovered and added. # $1: path to binary # $2: destination on initcpio (optional, defaults to same as source) @@ -490,7 +505,7 @@ parse_hook() { if [[ $SCRIPT ]]; then script=$(find_in_dirs "$SCRIPT" "${HOOKDIR[@]}") && - add_file "$script" "/hooks/$SCRIPT" + add_file "$script" "/hooks/$SCRIPT" 755 fi } -- 1.7.10.2
Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- functions | 2 +- install/consolefont | 6 +----- install/filesystems | 4 ++-- install/fw | 8 ++++++-- install/ide | 4 +--- install/keymap | 4 +--- install/memdisk | 9 ++++++--- install/mmc | 7 +++---- install/pata | 6 ++++-- install/resume | 2 +- install/sata | 6 ++++-- install/scsi | 9 +++++---- install/shutdown | 7 ++++--- install/sleep | 2 +- install/usb | 12 +++++------- install/usbinput | 5 +++-- install/virtio | 2 +- 17 files changed, 49 insertions(+), 46 deletions(-) diff --git a/functions b/functions index ad24076..b678435 100644 --- a/functions +++ b/functions @@ -219,7 +219,7 @@ all_modules() { mod=${mod##*/} mod="${mod%.ko*}" printf '%s\n' "${mod//-/_}" - done < <(find "$MODULEDIR" -name '*.ko*' -print0 2>/dev/null | grep -Zz "$@") + done < <(find "$MODULEDIR" -name '*.ko*' -print0 2>/dev/null | grep -EZz "$@") (( count )) } diff --git a/install/consolefont b/install/consolefont index cefbc45..8127502 100644 --- a/install/consolefont +++ b/install/consolefont @@ -1,8 +1,6 @@ #!/bin/bash build() { - local file ext - # subshell to avoid namespace pollution ( for cfg in /etc/{rc,vconsole}.conf; do @@ -29,9 +27,7 @@ build() { warning "consolefont: no font found in configuration" exit 1 fi - ) - - (( $? == 0 )) && SCRIPT=consolefont + ) && add_runscript } help() { diff --git a/install/filesystems b/install/filesystems index 9230b94..162768a 100644 --- a/install/filesystems +++ b/install/filesystems @@ -2,9 +2,9 @@ build() { if (( fs_autodetect_failed )); then - MODULES=$(all_modules '/kernel/fs' | grep -v "nls") + add_all_modules -f 'nls' '/kernel/fs' else - MODULES=$(checked_modules '/kernel/fs' | grep -v "nls") + add_checked_modules -f 'nls' '/kernel/fs' fi } diff --git a/install/fw b/install/fw index f624de0..98c05ab 100644 --- a/install/fw +++ b/install/fw @@ -1,9 +1,13 @@ #!/bin/bash build() { - MODULES=$(checked_modules "/drivers/firewire/") + local mod - [[ $MODULES ]] && MODULES+=" firewire-sbp2? sd_mod? sr_mod?" + if add_checked_modules '/drivers/firewire/'; then + for mod in firewire-sbp2 sd_mod sr_mod; do + add_module "$mod?" + done + fi } help() { diff --git a/install/ide b/install/ide index a8d9e97..5531132 100644 --- a/install/ide +++ b/install/ide @@ -1,9 +1,7 @@ #!/bin/bash build() { - MODULES="$(checked_modules "/ide/" | grep -v "legacy")" - - [[ $MODULES ]] && MODULES+=" ide-gd_mod?" + add_checked_modules "/ide/" && add_module "ide-gd_mod?" } help() { diff --git a/install/keymap b/install/keymap index 1dc25aa..99b816a 100644 --- a/install/keymap +++ b/install/keymap @@ -24,9 +24,7 @@ build() { warning "keymap: hook specified, but no KEYMAP found in configuration" return 1 fi - ) - - (( $? == 0 )) && SCRIPT=keymap + ) && add_runscript } help() { diff --git a/install/memdisk b/install/memdisk index b0d03f3..9b3577d 100644 --- a/install/memdisk +++ b/install/memdisk @@ -1,10 +1,13 @@ #!/bin/bash build() { - MODULES="phram mtdblock" - BINARIES="/usr/bin/memdiskfind" - SCRIPT="memdisk" + add_module 'phram' + add_module 'mtdblock' + add_file /usr/lib/initcpio/udev/01-memdisk.rules /lib/udev/rules.d/01-memdisk.rules + add_binary "memdiskfind" + + add_runscript } help() { diff --git a/install/mmc b/install/mmc index fb89239..60068df 100644 --- a/install/mmc +++ b/install/mmc @@ -1,10 +1,9 @@ #!/bin/bash build() { - MODULES=$(checked_modules "/mmc") - - if [[ $MODULES ]]; then - MODULES+=" tifm_7xx1 mmc_block" + if add_checked_modules '/mmc'; then + add_module 'tifm_7xx1' + add_module 'mmc_block' fi } diff --git a/install/pata b/install/pata index 76f268c..bcf3809 100644 --- a/install/pata +++ b/install/pata @@ -1,11 +1,13 @@ #!/bin/bash build() { + local filter added + for filter in 'ata/pata_*' 'ata/ata_generic' 'ata/ata_piix'; do - MODULES+=" $(checked_modules "$filter")" + add_checked_modules "$filter" && (( ++added )) done - [[ $MODULES ]] && MODULES+=" sd_mod?" + (( added )) && add_module "sd_mod?" } help() { diff --git a/install/resume b/install/resume index d261aad..914bca2 100644 --- a/install/resume +++ b/install/resume @@ -1,7 +1,7 @@ #!/bin/bash build() { - SCRIPT="resume" + add_runscript } help() { diff --git a/install/sata b/install/sata index c9b67c3..c6a2ade 100644 --- a/install/sata +++ b/install/sata @@ -1,12 +1,14 @@ #!/bin/bash build() { + local filter added + for filter in 'scsi/.*ata' 'block/sx8' 'ata/sata_*' \ 'ata/ahci' 'ata/pdc_adma' 'ata/ata_piix'; do - MODULES+=" $(checked_modules "$filter")" + add_checked_modules "$filter" && (( ++added )) done - [[ $MODULES ]] && MODULES+=" sd_mod?" + (( added )) && add_module "sd_mod?" } help() { diff --git a/install/scsi b/install/scsi index 4c2abc5..ab15e8c 100644 --- a/install/scsi +++ b/install/scsi @@ -1,12 +1,13 @@ #!/bin/bash build(){ + local filter added - MODULES="$(checked_modules "/scsi/" | grep -vE '(imm|ata|pcmcia)') - $(checked_modules "/block/" | grep -E '(cciss|cpqarray|DAC960)') - $(checked_modules "/fusion/")" + add_checked_modules -f '(imm|ata|pcmcia)' '/scsi/' && (( ++added )) + add_checked_modules -f '(cciss|cpqarray|DAC960)' '/block/' && (( ++added )) + add_checked_modules '/fusion/' && (( ++added )) - [[ $MODULES ]] && MODULES+=" sd_mod?" + (( added )) && add_modules 'sd_mod?' } help() { diff --git a/install/shutdown b/install/shutdown index 600209e..e9a5d93 100644 --- a/install/shutdown +++ b/install/shutdown @@ -1,10 +1,11 @@ #!/bin/bash build() { - BINARIES='cp lsblk' - SCRIPT='shutdown' - + add_binary cp + add_binary lsblk add_file "/usr/lib/initcpio/shutdown" "/shutdown" + + add_runscript } help() { diff --git a/install/sleep b/install/sleep index b81e1ca..c2bdcab 100644 --- a/install/sleep +++ b/install/sleep @@ -1,7 +1,7 @@ #!/bin/bash build() { - SCRIPT="sleep" + add_runscript } help() { diff --git a/install/usb b/install/usb index fcb2a47..2b36c35 100644 --- a/install/usb +++ b/install/usb @@ -1,13 +1,11 @@ #!/bin/bash build() { - local module= - - MODULES="$(checked_modules "/usb/host" | grep -ve "_cs" -e "sl811_hcd" -e "isp116x_hcd")" - - if [[ $MODULES ]]; then - MODULES+=" usb_storage? sd_mod? sr_mod?" - MODULES+=" $(checked_modules "drivers/usb/storage/ums-*")" + if add_checked_modules -f '(_cs|sl811_hcd|isp116x_hcd)' '/usb/host'; then + add_module 'usb_storage?' + add_module 'sd_mod?' + add_module 'sr_mod?' + add_checked_modules 'drivers/usb/storage/ums-*' fi } diff --git a/install/usbinput b/install/usbinput index f89d0bd..f427600 100644 --- a/install/usbinput +++ b/install/usbinput @@ -1,8 +1,9 @@ #!/bin/bash build() { - MODULES=" $(checked_modules "/usb/host" | grep -ve "_cs" -e "sl811_hcd" -e "isp116x_hcd")" - MODULES+=" $(all_modules "/hid/hid-") usbhid?" + add_checked_modules -f '(_cs|sl811_hcd|isp116x_hcd)' '/usb/host' + add_all_modules '/hid/hid-' + add_module 'usbhid?' } help() { diff --git a/install/virtio b/install/virtio index 0343e9f..10037ab 100644 --- a/install/virtio +++ b/install/virtio @@ -1,7 +1,7 @@ #!/bin/bash build() { - MODULES="$(checked_modules 'virtio_\(blk\|pci\)')" + add_checked_modules 'virtio_(blk|pci)' } help() { -- 1.7.10.2
This never worked properly and only "fixed" the stupid bug that found an edge case when either of these vars contains nothing more than a single space. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- init | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/init b/init index 8e9ad90..7cd812f 100644 --- a/init +++ b/init @@ -26,14 +26,11 @@ for d in ${disablehooks//,/ }; do eval "hook_${d}=disabled" done -earlymodules=${earlymodules//,/ } -if [ -n "${earlymodules## }" ]; then - modprobe -qab ${earlymodules} -fi +[ -n "${earlymodules//[[:space:]]}" ] && modprobe -qab ${earlymodules//,/ } . /config -[ -n "${MODULES## }" ] && modprobe -qab $MODULES +[ -n "${MODULES//[[:space:]]}" ] && modprobe -qab $MODULES # If rootdelay is empty or not a non-negative integer, set it to 10 if [ -z "${rootdelay}" ] || ! [ "${rootdelay}" -ge 0 ]; then -- 1.7.10.2
Abstract this out to the init_functions file, and allow this function take 2+ parameters -- the hook name to be run, a user friendly short description, and then the list of hook files to source from. While we're at it, take advantage of the fact that hooks are now installed with executable perms. If a hook isn't marked excutable, we skip it, thereby removing our eval hack. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- init | 17 ++--------------- init_functions | 16 ++++++++++++++++ 2 files changed, 18 insertions(+), 15 deletions(-) diff --git a/init b/init index 7cd812f..a8915fb 100644 --- a/init +++ b/init @@ -23,7 +23,7 @@ else fi for d in ${disablehooks//,/ }; do - eval "hook_${d}=disabled" + [ -e "/hooks/$d" ] && chmod 644 "/hooks/$d" done [ -n "${earlymodules//[[:space:]]}" ] && modprobe -qab ${earlymodules//,/ } @@ -37,20 +37,7 @@ if [ -z "${rootdelay}" ] || ! [ "${rootdelay}" -ge 0 ]; then rootdelay=10 fi -if [ -e "/hooks" ]; then - for h in ${HOOKS}; do - TST="" - eval "TST=\$hook_${h}" - if [ "${TST}" != "disabled" ]; then - run_hook () { msg "${h}: no run function defined"; } - if [ -e "/hooks/${h}" ]; then - . /hooks/${h} - msg ":: Running Hook [${h}]" - run_hook - fi - fi - done -fi +run_hookfunctions 'run_hook' "early hook" $HOOKS # honor the old behavior of break=y as a synonym for break=premount if [ "${break}" = "y" ] || [ "${break}" = "premount" ]; then diff --git a/init_functions b/init_functions index 60532d2..1bebf45 100644 --- a/init_functions +++ b/init_functions @@ -44,6 +44,22 @@ major_minor_to_device() { return 1 } +run_hookfunctions() { + local hook fn=$1 desc=$2 + + shift 2 + for hook in "$@"; do + [ -x "/hooks/$hook" ] || continue + + unset "$fn" + . "/hooks/$hook" + type "$fn" >/dev/null || continue + + msg ":: running $desc [$hook]" + "$fn" + done +} + parse_cmdline() { local w in_quotes lhs rhs in_quotes=0 -- 1.7.10.2
This adds support for a second round of hooks during early userspace. Users can define a 'run_latehook' in their hook script similar to the declaration of 'run_hook' which continues to run at the same point in execution. Scripts are executed after mounting root in reverse order of how they're clared in mkinitcpio'c config. These late running hooks should take care of things such as: - mounting other partitions - moving config to the real root (mostly for liveCDs) - shutting down previously started daemons Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- init | 2 ++ init_functions | 11 +++++++++++ 2 files changed, 13 insertions(+) diff --git a/init b/init index a8915fb..275ac68 100644 --- a/init +++ b/init @@ -71,6 +71,8 @@ elif [ ! -x "/new_root${init}" ]; then launch_interactive_shell --exec fi +run_hookfunctions 'run_latehook' 'late hook' $(reverse_list "$HOOKS") + # mount /usr if it exists realtab=/new_root/etc/fstab if [ -f "$realtab" ]; then diff --git a/init_functions b/init_functions index 1bebf45..0ba4cd5 100644 --- a/init_functions +++ b/init_functions @@ -7,6 +7,17 @@ err () { echo "ERROR: $@" } +reverse_list() { + local x rev=$1 + + shift + for x; do + rev="$x $rev" + done + + printf '%s' "$rev" +} + poll_device() { local device=$1 seconds=${2//[!0-9]} -- 1.7.10.2
There's no need to keep this mildly dangerous script executable on the filesystem. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- install/shutdown | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) mode change 100755 => 100644 shutdown diff --git a/install/shutdown b/install/shutdown index e9a5d93..96818e1 100644 --- a/install/shutdown +++ b/install/shutdown @@ -3,7 +3,7 @@ build() { add_binary cp add_binary lsblk - add_file "/usr/lib/initcpio/shutdown" "/shutdown" + add_file "/usr/lib/initcpio/shutdown" "/shutdown" 755 add_runscript } diff --git a/shutdown b/shutdown old mode 100755 new mode 100644 -- 1.7.10.2
Run this later in early init where we can delete the modules from the ramfs before copying the image over to /run/initramfs. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- hooks/shutdown | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hooks/shutdown b/hooks/shutdown index 7c5d9e8..5b7778c 100644 --- a/hooks/shutdown +++ b/hooks/shutdown @@ -1,6 +1,7 @@ #!/usr/bin/ash -run_hook() { +run_latehook() { + rm -rf /usr/lib/modules cp -ax / /run/initramfs } -- 1.7.10.2
We don't ship compressed manpages, and this never touched lsinitcpio's generated page. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 1533b3a..398be6d 100644 --- a/Makefile +++ b/Makefile @@ -67,7 +67,7 @@ lsinitcpio.1: lsinitcpio.1.txt Makefile clean: ${RM} -r build mkinitcpio-${VERSION} - ${RM} mkinitcpio-${VERSION}.tar.gz mkinitcpio.8 mkinitcpio.8.gz + ${RM} mkinitcpio-${VERSION}.tar.gz mkinitcpio.8 lsinitcpio.1 dist: doc echo ${VERSION} > VERSION -- 1.7.10.2
We read the image a number of times. Extract the image to a temp directory so we can just extract it once and use the filesystem to our advantage. This requires a temp dir which we can nuke on an EXIT trap, but the whole operation is sped is a bit, especially for larger images. The real goal of this is to make examination of the hooks easier. Since we now have multiple hook sets to contend with, dynamically tally up the available functions in the hook files so we can iterate over specific functions of interest in the output. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- lsinitcpio | 59 +++++++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 18 deletions(-) diff --git a/lsinitcpio b/lsinitcpio index 5abe37f..f33d9d3 100755 --- a/lsinitcpio +++ b/lsinitcpio @@ -59,7 +59,6 @@ size_to_human() { }' } - OPT_SHORT='ahnvx' OPT_LONG=('analyze' 'help' 'nocolor' 'verbose' 'extract') @@ -127,6 +126,9 @@ if (( analyze )); then declare -a binaries explicitmod modules foundhooks hooks declare kernver ratio columns=$(tput cols) + workdir=$(mktemp -d --tmpdir="$TMPDIR" lsinitcpio.XXXXXX) + trap 'rm -rf "$workdir"' EXIT + # fallback in case tput failed us columns=${columns:-80} @@ -139,27 +141,48 @@ if (( analyze )); then ratio=.$(( zsize * 1000 / fullsize % 1000 )) fi - # read contents of image - while read -r line; do - if [[ $line = *.ko?(.?z) ]]; then - line=${line##*/} - modules+=("${line%.ko?(.?z)}") - elif [[ -z $kernver && $line =~ /lib/modules/([^/]+)/ ]]; then - kernver=${BASH_REMATCH[1]} - elif [[ $line = ./hooks/* ]]; then - foundhooks+=("${line##*/}") - elif [[ $line = *@(/?(s)bin/)* ]]; then - binaries+=("${line##*/}") - fi - done < <(decomp "$image" | bsdtar tf -) - - read -r version < <(decomp "$image" | bsdtar xOf - VERSION 2>/dev/null) + # decompress the image since we need to read from it multiple times + decomp "$image" | bsdtar -C "$workdir" -xf - + + # collect stats + kernver=("$workdir"/usr/lib/modules/*/) + kernver=${kernver%/} + kernver=${kernver##*/} + + modules=("$workdir/usr/lib/modules/$kernver"/kernel/*.ko*) + modules=("${modules[@]##*/}") + modules=("${modules[@]%.ko*}") + + foundhooks=("$workdir"/hooks/*) + foundhooks=("${foundhooks[@]##*/}") + + binaries=("$workdir"/usr/bin/*) + binaries=("${binaries[@]##*/}") + + read -r version < "$workdir/VERSION" # source and read config - . <(decomp "$image" | bsdtar xOf - config) + . "$workdir/config" + explicitmod=($MODULES) + for hook in $HOOKS; do - in_array "$hook" "${foundhooks[@]}" && hooks+=("$hook") + [[ -e $workdir/hooks/$hook ]] || continue + + mapfile -t funcs < \ + <(awk ' + /^[[:space:]]*[[:alnum:]_]+/ && /\([[:space:]]*\)/ { + match($1, /[[:alnum:]_]+/) + print substr($1, RSTART, RLENGTH) + }' "$workdir/hooks/$hook") + + for fn in "${funcs[@]}"; do + case $fn in + run_hook) + hooks+=("$hook") + ;; + esac + done done # print results -- 1.7.10.2
On Sun, May 13, 2012 at 12:57 PM, Dave Reisner <dreisner@archlinux.org> wrote:
We read the image a number of times. Extract the image to a temp directory so we can just extract it once and use the filesystem to our advantage. This requires a temp dir which we can nuke on an EXIT trap, but the whole operation is sped is a bit, especially for larger images.
The real goal of this is to make examination of the hooks easier. Since we now have multiple hook sets to contend with, dynamically tally up the available functions in the hook files so we can iterate over specific functions of interest in the output.
Signed-off-by: Dave Reisner <dreisner@archlinux.org> ---
You missed one: TIMEFORMAT=%R decomptime=$({ time decomp "$image" >/dev/null; } 2>&1 ) This is guarded by a 'if compressed' block now, but it is probably useful even if you are using an uncompressed tar image- there is still work being done in reading and breaking the individual files out of the tarfile.
lsinitcpio | 59 +++++++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 18 deletions(-)
diff --git a/lsinitcpio b/lsinitcpio index 5abe37f..f33d9d3 100755 --- a/lsinitcpio +++ b/lsinitcpio @@ -59,7 +59,6 @@ size_to_human() { }' }
- OPT_SHORT='ahnvx' OPT_LONG=('analyze' 'help' 'nocolor' 'verbose' 'extract')
@@ -127,6 +126,9 @@ if (( analyze )); then declare -a binaries explicitmod modules foundhooks hooks declare kernver ratio columns=$(tput cols)
+ workdir=$(mktemp -d --tmpdir="$TMPDIR" lsinitcpio.XXXXXX) + trap 'rm -rf "$workdir"' EXIT + # fallback in case tput failed us columns=${columns:-80}
@@ -139,27 +141,48 @@ if (( analyze )); then ratio=.$(( zsize * 1000 / fullsize % 1000 )) fi
- # read contents of image - while read -r line; do - if [[ $line = *.ko?(.?z) ]]; then - line=${line##*/} - modules+=("${line%.ko?(.?z)}") - elif [[ -z $kernver && $line =~ /lib/modules/([^/]+)/ ]]; then - kernver=${BASH_REMATCH[1]} - elif [[ $line = ./hooks/* ]]; then - foundhooks+=("${line##*/}") - elif [[ $line = *@(/?(s)bin/)* ]]; then - binaries+=("${line##*/}") - fi - done < <(decomp "$image" | bsdtar tf -) - - read -r version < <(decomp "$image" | bsdtar xOf - VERSION 2>/dev/null) + # decompress the image since we need to read from it multiple times + decomp "$image" | bsdtar -C "$workdir" -xf - + + # collect stats + kernver=("$workdir"/usr/lib/modules/*/) + kernver=${kernver%/} + kernver=${kernver##*/} + + modules=("$workdir/usr/lib/modules/$kernver"/kernel/*.ko*) + modules=("${modules[@]##*/}") + modules=("${modules[@]%.ko*}") + + foundhooks=("$workdir"/hooks/*) + foundhooks=("${foundhooks[@]##*/}") + + binaries=("$workdir"/usr/bin/*) + binaries=("${binaries[@]##*/}") + + read -r version < "$workdir/VERSION"
# source and read config - . <(decomp "$image" | bsdtar xOf - config) + . "$workdir/config" + explicitmod=($MODULES) + for hook in $HOOKS; do - in_array "$hook" "${foundhooks[@]}" && hooks+=("$hook") + [[ -e $workdir/hooks/$hook ]] || continue + + mapfile -t funcs < \ + <(awk ' + /^[[:space:]]*[[:alnum:]_]+/ && /\([[:space:]]*\)/ { + match($1, /[[:alnum:]_]+/) + print substr($1, RSTART, RLENGTH) + }' "$workdir/hooks/$hook") + + for fn in "${funcs[@]}"; do + case $fn in + run_hook) + hooks+=("$hook") + ;; + esac + done done
# print results -- 1.7.10.2
On Sun, May 13, 2012 at 01:39:35PM -0500, Dan McGee wrote:
On Sun, May 13, 2012 at 12:57 PM, Dave Reisner <dreisner@archlinux.org> wrote:
We read the image a number of times. Extract the image to a temp directory so we can just extract it once and use the filesystem to our advantage. This requires a temp dir which we can nuke on an EXIT trap, but the whole operation is sped is a bit, especially for larger images.
The real goal of this is to make examination of the hooks easier. Since we now have multiple hook sets to contend with, dynamically tally up the available functions in the hook files so we can iterate over specific functions of interest in the output.
Signed-off-by: Dave Reisner <dreisner@archlinux.org> ---
You missed one: TIMEFORMAT=%R decomptime=$({ time decomp "$image" >/dev/null; } 2>&1 )
This is guarded by a 'if compressed' block now, but it is probably useful even if you are using an uncompressed tar image- there is still work being done in reading and breaking the individual files out of the tarfile.
Sure, we can placate the few weirdos who don't compress their images at all. This patch was more interested in getting rid of the archive reads for: scanning binaries/modules/hooks + extracting the config + extracting the version file. Added locally as a separate patch.
lsinitcpio | 59 +++++++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 18 deletions(-)
diff --git a/lsinitcpio b/lsinitcpio index 5abe37f..f33d9d3 100755 --- a/lsinitcpio +++ b/lsinitcpio @@ -59,7 +59,6 @@ size_to_human() { }' }
- OPT_SHORT='ahnvx' OPT_LONG=('analyze' 'help' 'nocolor' 'verbose' 'extract')
@@ -127,6 +126,9 @@ if (( analyze )); then declare -a binaries explicitmod modules foundhooks hooks declare kernver ratio columns=$(tput cols)
+ workdir=$(mktemp -d --tmpdir="$TMPDIR" lsinitcpio.XXXXXX) + trap 'rm -rf "$workdir"' EXIT + # fallback in case tput failed us columns=${columns:-80}
@@ -139,27 +141,48 @@ if (( analyze )); then ratio=.$(( zsize * 1000 / fullsize % 1000 )) fi
- # read contents of image - while read -r line; do - if [[ $line = *.ko?(.?z) ]]; then - line=${line##*/} - modules+=("${line%.ko?(.?z)}") - elif [[ -z $kernver && $line =~ /lib/modules/([^/]+)/ ]]; then - kernver=${BASH_REMATCH[1]} - elif [[ $line = ./hooks/* ]]; then - foundhooks+=("${line##*/}") - elif [[ $line = *@(/?(s)bin/)* ]]; then - binaries+=("${line##*/}") - fi - done < <(decomp "$image" | bsdtar tf -) - - read -r version < <(decomp "$image" | bsdtar xOf - VERSION 2>/dev/null) + # decompress the image since we need to read from it multiple times + decomp "$image" | bsdtar -C "$workdir" -xf - + + # collect stats + kernver=("$workdir"/usr/lib/modules/*/) + kernver=${kernver%/} + kernver=${kernver##*/} + + modules=("$workdir/usr/lib/modules/$kernver"/kernel/*.ko*) + modules=("${modules[@]##*/}") + modules=("${modules[@]%.ko*}") + + foundhooks=("$workdir"/hooks/*) + foundhooks=("${foundhooks[@]##*/}") + + binaries=("$workdir"/usr/bin/*) + binaries=("${binaries[@]##*/}") + + read -r version < "$workdir/VERSION"
# source and read config - . <(decomp "$image" | bsdtar xOf - config) + . "$workdir/config" + explicitmod=($MODULES) + for hook in $HOOKS; do - in_array "$hook" "${foundhooks[@]}" && hooks+=("$hook") + [[ -e $workdir/hooks/$hook ]] || continue + + mapfile -t funcs < \ + <(awk ' + /^[[:space:]]*[[:alnum:]_]+/ && /\([[:space:]]*\)/ { + match($1, /[[:alnum:]_]+/) + print substr($1, RSTART, RLENGTH) + }' "$workdir/hooks/$hook") + + for fn in "${funcs[@]}"; do + case $fn in + run_hook) + hooks+=("$hook") + ;; + esac + done done
# print results -- 1.7.10.2
The previous patch makes this quite simple, and we can easily catch instances of run_latehook() for printing in the output. Signed-off-by: Dave Reisner <dreisner@archlinux.org> --- lsinitcpio | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/lsinitcpio b/lsinitcpio index f33d9d3..fcb5199 100755 --- a/lsinitcpio +++ b/lsinitcpio @@ -181,6 +181,9 @@ if (( analyze )); then run_hook) hooks+=("$hook") ;; + run_latehook) + latehooks=("$hook" "${latehooks[@]}") + ;; esac done done @@ -215,10 +218,16 @@ if (( analyze )); then printf '\n' if (( ${#hooks[*]} )); then - msg 'Hook run order:' + msg 'Early hook run order:' printf ' %s\n' "${hooks[@]}" printf '\n' fi + + if (( ${#latehooks[*]} )); then + msg 'Late hook run order:' + printf ' %s\n' "${latehooks[@]}" + printf '\n' + fi else decomp "$image" | bsdcpio -i --quiet $verbose $list fi -- 1.7.10.2
Am 13.05.2012 19:57, schrieb Dave Reisner:
The final unlisted change is what I referred to earlier about udev. Since the udev hook would be handling its own startup/shutdown, we'd simply yank those if blocks out of /init and carry on. I've been testing this and it works quite well.
I haven't looked through your patchset yet, but here is the bad part about udev: In the past, udev needed to run before the modules listed in MODULES= from mkinitcpio.conf were loaded. Otherwise, firmware loading would fail, and almost everyone with MODULES="radeon" would be very unhappy! I know that some things have changed regarding firmware loading since then, but this is likely still an issue in some form - and even if it isn't, it is a good idea to have a firmware helper present before doing anything.
On Mon, May 14, 2012 at 10:37:38AM +0200, Thomas Bächler wrote:
Am 13.05.2012 19:57, schrieb Dave Reisner:
The final unlisted change is what I referred to earlier about udev. Since the udev hook would be handling its own startup/shutdown, we'd simply yank those if blocks out of /init and carry on. I've been testing this and it works quite well.
I haven't looked through your patchset yet, but here is the bad part about udev:
In the past, udev needed to run before the modules listed in MODULES= from mkinitcpio.conf were loaded. Otherwise, firmware loading would fail, and almost everyone with MODULES="radeon" would be very unhappy! I know that some things have changed regarding firmware loading since then, but this is likely still an issue in some form - and even if it isn't, it is a good idea to have a firmware helper present before doing anything.
Excellent point -- was curious about this myself. I asked Kay about it: 08:50 kay » it will be catched with the coldplug run, yes 08:50 kay » but if udev runs in intramfs, and there is no firmware file, the request will be canceled 08:51 falconindy » yeah, this is initramfs 08:51 falconindy » so settle will play catchup and things will just work (hopefully) 08:57 kay » settle? 08:57 kay » trigger will 08:58 falconindy » even better 08:58 kay » but if udev from initramfs still runs and there is no firmware file, all is discarded So it seems to me that we're covered here since we assume that the firmware is present (or else its a bug in mkinitcpio). I'd like to get someone with perhaps a radeon to confirm this via my working branch and a modified udev hook (I can provide this), but I guess the worst case scenario is that we file a bug. d
Am 14.05.2012 15:50, schrieb Dave Reisner:
In the past, udev needed to run before the modules listed in MODULES= from mkinitcpio.conf were loaded. Otherwise, firmware loading would fail, and almost everyone with MODULES="radeon" would be very unhappy! I know that some things have changed regarding firmware loading since then, but this is likely still an issue in some form - and even if it isn't, it is a good idea to have a firmware helper present before doing anything.
Excellent point -- was curious about this myself. I asked Kay about it:
08:50 kay » it will be catched with the coldplug run, yes 08:50 kay » but if udev runs in intramfs, and there is no firmware file, the request will be canceled
That should be taken care of.
08:51 falconindy » yeah, this is initramfs 08:51 falconindy » so settle will play catchup and things will just work (hopefully) 08:57 kay » settle? 08:57 kay » trigger will 08:58 falconindy » even better 08:58 kay » but if udev from initramfs still runs and there is no firmware file, all is discarded
So it seems to me that we're covered here since we assume that the firmware is present (or else its a bug in mkinitcpio). I'd like to get someone with perhaps a radeon to confirm this via my working branch and a modified udev hook (I can provide this), but I guess the worst case scenario is that we file a bug.
Back in the day (haha), udev still worked with modules waiting for firmware in module_init. But running 'modprobe foobar; udevd --daemon' would wait for the timeout and start udev afterwards. Starting udev before module loading is safe and also works with old versions. If these bugs are all fixed, then Kay is of course correct.
On Mon, May 14, 2012 at 04:04:24PM +0200, Thomas Bächler wrote:
Am 14.05.2012 15:50, schrieb Dave Reisner:
In the past, udev needed to run before the modules listed in MODULES= from mkinitcpio.conf were loaded. Otherwise, firmware loading would fail, and almost everyone with MODULES="radeon" would be very unhappy! I know that some things have changed regarding firmware loading since then, but this is likely still an issue in some form - and even if it isn't, it is a good idea to have a firmware helper present before doing anything.
Excellent point -- was curious about this myself. I asked Kay about it:
08:50 kay » it will be catched with the coldplug run, yes 08:50 kay » but if udev runs in intramfs, and there is no firmware file, the request will be canceled
That should be taken care of.
08:51 falconindy » yeah, this is initramfs 08:51 falconindy » so settle will play catchup and things will just work (hopefully) 08:57 kay » settle? 08:57 kay » trigger will 08:58 falconindy » even better 08:58 kay » but if udev from initramfs still runs and there is no firmware file, all is discarded
So it seems to me that we're covered here since we assume that the firmware is present (or else its a bug in mkinitcpio). I'd like to get someone with perhaps a radeon to confirm this via my working branch and a modified udev hook (I can provide this), but I guess the worst case scenario is that we file a bug.
Back in the day (haha), udev still worked with modules waiting for firmware in module_init. But running 'modprobe foobar; udevd --daemon' would wait for the timeout and start udev afterwards.
Hmmm, so this problem would be visible via the same broken modules that cropped up when udev removed the timeout handling (mostly wifi drivers). I've got a user testing this for me and reporting that radeon doesn't work properly. Even though the driver looks like it does an async firmware request, it blocks for 60 seconds before timing out and saying failed to load firmware (even though the provided path exists).
Starting udev before module loading is safe and also works with old versions. If these bugs are all fixed, then Kay is of course correct.
I'm more inclined to think that this is module specific than udev specific. Either way, probably best to leave the udev startup where it is and just move out the cleanup. Boooo. d
Am 14.05.2012 17:59, schrieb Dave Reisner:
I'm more inclined to think that this is module specific than udev specific. Either way, probably best to leave the udev startup where it is and just move out the cleanup. Boooo.
Have it your way and introduce early hooks.
On Mon, May 14, 2012 at 06:19:11PM +0200, Thomas Bächler wrote:
Am 14.05.2012 17:59, schrieb Dave Reisner:
I'm more inclined to think that this is module specific than udev specific. Either way, probably best to leave the udev startup where it is and just move out the cleanup. Boooo.
Have it your way and introduce early hooks.
I thought about that briefly, and can't see a use for it beyond starting udev. It's low overhead, but I'm not convinced there's enough need for it. Maybe someone can prove me wrong. d
On Mon, May 14, 2012 at 12:27:11PM -0400, Dave Reisner wrote:
On Mon, May 14, 2012 at 06:19:11PM +0200, Thomas Bächler wrote:
Am 14.05.2012 17:59, schrieb Dave Reisner:
I'm more inclined to think that this is module specific than udev specific. Either way, probably best to leave the udev startup where it is and just move out the cleanup. Boooo.
Have it your way and introduce early hooks.
I thought about that briefly, and can't see a use for it beyond starting udev. It's low overhead, but I'm not convinced there's enough need for it.
Maybe someone can prove me wrong.
d
So, I'm going back on this and implementing early hooks anyways. I'm going to take some time to see if I can optimize this at all so that we can avoid rereading the same N files 4 times. Same user who reported the radeon fail confirmed that udev as an early hook works well with loading radeon in early userspace, but I guess we knew already that it would work. d
participants (5)
-
canyonknight＠gmail.com
-
Dan McGee
-
Dave Reisner
-
Dave Reisner
-
Thomas Bächler