[pacman-dev] [PATCH 1/3] bacman: allow for parallel packaging
* move the actual assembly process into its own function * allow for packaging multiple packages with one command * handle SIGHUP SIGINT SIGTERM and remove working dirs accordingly * add some comments Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 460 ++++++++++++++++++++++++++------------------------- 1 file changed, 238 insertions(+), 222 deletions(-) mode change 100644 => 100755 contrib/bacman.sh.in diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in old mode 100644 new mode 100755 index a611c1a..4cd78e4 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -33,9 +33,18 @@ ARGS=("$@") m4_include(../scripts/library/output_format.sh) -# -# User Friendliness -# +# Lazy recursive clean up of temporary dirs +work_dir_root="${TMPDIR:-/tmp}/bacman" +clean_up() { + rm -r "${work_dir_root}".* + echo + exit +} + +# Trap termination signals +trap clean_up SIGHUP SIGINT SIGTERM + +# Print usage information usage() { echo "${myname} (pacman) v${myver}" echo @@ -46,12 +55,14 @@ usage() { echo "Example: ${myname} linux-headers" } +# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } +# Check for specified arguments while [[ ! -z $1 ]]; do if [[ $1 == "--nocolor" ]]; then USE_COLOR='n' @@ -64,13 +75,16 @@ while [[ ! -z $1 ]]; do fi done +# Configure colored output m4_include(../scripts/library/term_colors.sh) -if (( $# != 1 )); then +# Break if no argument was given +if (( $# < 1 )); then usage exit 1 fi +# Print usage or version if requested if [[ $1 = -@(h|-help) ]]; then usage exit 0 @@ -79,9 +93,7 @@ elif [[ $1 = -@(V|-version) ]]; then exit 0 fi -# -# Fakeroot support -# +# Run in fakeroot if EUID is not root if (( EUID )); then if [[ -f /usr/bin/fakeroot ]]; then msg "Entering fakeroot environment" @@ -94,264 +106,268 @@ if (( EUID )); then fi fi -# -# Setting environmental variables -# +# Source environmental variables and specify fallbacks if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 fi - eval $(awk '/DBPath/ {print $1$2$3}' @sysconfdir@/pacman.conf) pac_db="${DBPath:-@localstatedir@/lib/pacman/}/local" - if [[ ! -r @sysconfdir@/makepkg.conf ]]; then error "unable to read @sysconfdir@/makepkg.conf" exit 1 fi - source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi - pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -pkg_name="$1" -pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) -pkg_namver=("${pkg_dir[@]##*/}") - -# -# Checks everything is in place -# +# Check for an existing database if [[ ! -d $pac_db ]]; then error "pacman database directory ${pac_db} not found" exit 1 fi -if (( ${#pkg_dir[@]} != 1 )); then - error "%d entries for package %s found in pacman database" \ - ${#pkg_dir[@]} "${pkg_name}" - msg2 "%s" "${pkg_dir[@]}" - exit 1 -fi - -if [[ ! -d $pkg_dir ]]; then - error "package %s is found in pacman database," "${pkg_name}" - plain " but '%s' is not a directory" "${pkg_dir}" - exit 1 -fi - -# -# Begin -# -msg "Package: ${pkg_namver}" -work_dir=$(mktemp -d "${TMPDIR:-/tmp}/bacman.XXXXXXXXXX") -cd "$work_dir" || exit 1 - -# -# File copying -# -msg2 "Copying package files..." - -while read i; do - if [[ -z $i ]]; then - continue +# Assemble a single package: $1 = pkgname +fakebuild() { + pkg_name="$1" + pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) + pkg_namver=("${pkg_dir[@]##*/}") + + # Checks database for specified package + if (( ${#pkg_dir[@]} != 1 )); then + error "%d entries for package %s found in pacman database" \ + ${#pkg_dir[@]} "${pkg_name}" + msg2 "%s" "${pkg_dir[@]}" + exit 1 fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + if [[ ! -d $pkg_dir ]]; then + error "package %s is found in pacman database," "${pkg_name}" + plain " but '%s' is not a directory" "${pkg_dir}" + exit 1 fi - case "$current" in - %FILES%) - local_file="/$i" - package_file="$work_dir/$i" + # Create working directory + msg "Package: ${pkg_namver}" + work_dir=$(mktemp -d "${work_dir_root}.XXXXXXXXXX") + cd "$work_dir" || exit 1 - if [[ ! -e $local_file ]]; then - warning "package file $local_file is missing" - continue - fi - ;; - - %BACKUP%) - # Get the MD5 checksum. - original_md5="${i##*$'\t'}" - # Strip the md5sum after the tab. - i="${i%$'\t'*}" - local_file="/$i.pacnew" - package_file="$work_dir/$i" - - # Include unmodified .pacnew files. - local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" - if [[ $INCLUDE_PACNEW == 'n' ]] \ - || [[ ! -e $local_file ]] \ - || [[ $local_md5 != $original_md5 ]]; then - # Warn about modified files. - local_md5="$(md5sum "/$i" | cut -d' ' -f1)" - if [[ $local_md5 != $original_md5 ]]; then - warning "package file /$i has been modified" - fi - # Let the normal file be included in the %FILES% list. - continue - fi - ;; + # Assemble list of files which belong to the package and tar them + msg2 "Copying package files..." + while read i; do + if [[ -z $i ]]; then + continue + fi - *) + if [[ $i == %+([A-Z])% ]]; then + current=$i continue - ;; - esac + fi - ret=0 - bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + case "$current" in + %FILES%) + local_file="/$i" + package_file="$work_dir/$i" + + if [[ ! -e $local_file ]]; then + warning "package file $local_file is missing" + continue + fi + ;; + + %BACKUP%) + # Get the MD5 checksum. + original_md5="${i##*$'\t'}" + # Strip the md5sum after the tab. + i="${i%$'\t'*}" + local_file="/$i.pacnew" + package_file="$work_dir/$i" + + # Include unmodified .pacnew files. + local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" + if [[ $INCLUDE_PACNEW == 'n' ]] \ + || [[ ! -e $local_file ]] \ + || [[ $local_md5 != $original_md5 ]]; then + # Warn about modified files. + local_md5="$(md5sum "/$i" | cut -d' ' -f1)" + if [[ $local_md5 != $original_md5 ]]; then + warning "package file /$i has been modified" + fi + # Let the normal file be included in the %FILES% list. + continue + fi + ;; - # Workaround to bsdtar not reporting a missing file as an error - if ! [[ -e $package_file || -L $package_file ]]; then - error "unable to add $local_file to the package" - plain " If your user does not have permission to read this file, then" - plain " you will need to run $myname as root." + *) + continue + ;; + esac + + # Tar files + ret=0 + bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + # Workaround to bsdtar not reporting a missing file as an error + if ! [[ -e $package_file || -L $package_file ]]; then + error "unable to add $local_file to the package" + plain " If your user does not have permission to read this file, then" + plain " you will need to run $myname as root." + rm -rf "$work_dir" + exit 1 + fi + done < "$pkg_dir"/files + + ret=$? + if (( ret )); then rm -rf "$work_dir" exit 1 fi -done < "$pkg_dir"/files -ret=$? -if (( ret )); then - rm -rf "$work_dir" - exit 1 -fi + # Calculate package size + pkg_size=$(du -sk | awk '{print $1 * 1024}') -pkg_size=$(du -sk | awk '{print $1 * 1024}') - -# -# .PKGINFO stuff -# TODO adopt makepkg's write_pkginfo() into this or scripts/library -# -msg2 "Generating .PKGINFO metadata..." -echo "# Generated by $myname $myver" > .PKGINFO -if [[ $INFAKEROOT == "1" ]]; then - echo "# Using $(fakeroot -v)" >> .PKGINFO -fi -echo "# $(LC_ALL=C date)" >> .PKGINFO -echo "#" >> .PKGINFO - -while read i; do - if [[ -z $i ]]; then - continue; + # Reconstruct .PKGINFO from database + # TODO adopt makepkg's write_pkginfo() into this or scripts/library + msg2 "Generating .PKGINFO metadata..." + echo "# Generated by $myname $myver" > .PKGINFO + if [[ $INFAKEROOT == "1" ]]; then + echo "# Using $(fakeroot -v)" >> .PKGINFO fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + echo "# $(LC_ALL=C date)" >> .PKGINFO + echo "#" >> .PKGINFO + while read i; do + if [[ -z $i ]]; then + continue; + fi + if [[ $i == %+([A-Z])% ]]; then + current=$i + continue + fi + + case "$current" in + # desc + %NAME%) + echo "pkgname = $i" >> .PKGINFO + ;; + %VERSION%) + echo "pkgver = $i" >> .PKGINFO + ;; + %DESC%) + echo "pkgdesc = $i" >> .PKGINFO + ;; + %URL%) + echo "url = $i" >> .PKGINFO + ;; + %LICENSE%) + echo "license = $i" >> .PKGINFO + ;; + %ARCH%) + echo "arch = $i" >> .PKGINFO + pkg_arch="$i" + ;; + %BUILDDATE%) + echo "builddate = $(date -u "+%s")" >> .PKGINFO + ;; + %PACKAGER%) + echo "packager = $pkg_pkger" >> .PKGINFO + ;; + %SIZE%) + echo "size = $pkg_size" >> .PKGINFO + ;; + %GROUPS%) + echo "group = $i" >> .PKGINFO + ;; + %REPLACES%) + echo "replaces = $i" >> .PKGINFO + ;; + %DEPENDS%) + echo "depend = $i" >> .PKGINFO + ;; + %OPTDEPENDS%) + echo "optdepend = $i" >> .PKGINFO + ;; + %CONFLICTS%) + echo "conflict = $i" >> .PKGINFO + ;; + %PROVIDES%) + echo "provides = $i" >> .PKGINFO + ;; + %BACKUP%) + # Strip the md5sum after the tab + echo "backup = ${i%%$'\t'*}" >> .PKGINFO + ;; + esac + done < <(cat "$pkg_dir"/{desc,files}) + + comp_files=".PKGINFO" + + # Add instal file if present + if [[ -f $pkg_dir/install ]]; then + cp "$pkg_dir/install" "$work_dir/.INSTALL" + comp_files+=" .INSTALL" + fi + if [[ -f $pkg_dir/changelog ]]; then + cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" + comp_files+=" .CHANGELOG" fi - case "$current" in - # desc - %NAME%) - echo "pkgname = $i" >> .PKGINFO - ;; - %VERSION%) - echo "pkgver = $i" >> .PKGINFO - ;; - %DESC%) - echo "pkgdesc = $i" >> .PKGINFO - ;; - %URL%) - echo "url = $i" >> .PKGINFO - ;; - %LICENSE%) - echo "license = $i" >> .PKGINFO - ;; - %ARCH%) - echo "arch = $i" >> .PKGINFO - pkg_arch="$i" - ;; - %BUILDDATE%) - echo "builddate = $(date -u "+%s")" >> .PKGINFO - ;; - %PACKAGER%) - echo "packager = $pkg_pkger" >> .PKGINFO - ;; - %SIZE%) - echo "size = $pkg_size" >> .PKGINFO - ;; - %GROUPS%) - echo "group = $i" >> .PKGINFO - ;; - %REPLACES%) - echo "replaces = $i" >> .PKGINFO - ;; - %DEPENDS%) - echo "depend = $i" >> .PKGINFO - ;; - %OPTDEPENDS%) - echo "optdepend = $i" >> .PKGINFO - ;; - %CONFLICTS%) - echo "conflict = $i" >> .PKGINFO - ;; - %PROVIDES%) - echo "provides = $i" >> .PKGINFO - ;; - - # files - %BACKUP%) - # Strip the md5sum after the tab - echo "backup = ${i%%$'\t'*}" >> .PKGINFO - ;; - esac -done < <(cat "$pkg_dir"/{desc,files}) - -comp_files=".PKGINFO" - -if [[ -f $pkg_dir/install ]]; then - cp "$pkg_dir/install" "$work_dir/.INSTALL" - comp_files+=" .INSTALL" -fi -if [[ -f $pkg_dir/changelog ]]; then - cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" - comp_files+=" .CHANGELOG" -fi + # Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL + chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -# -# Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL -# -chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + # Generate the package + msg2 "Generating the package..." -# -# Generate the package -# -msg2 "Generating the package..." - -pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" -ret=0 - -# TODO: Maybe this can be set globally for robustness -shopt -s -o pipefail -bsdtar -cf - $comp_files * | -case "$PKGEXT" in - *tar.gz) gzip -c -f -n ;; - *tar.bz2) bzip2 -c -f ;; - *tar.xz) xz -c -z - ;; - *tar.Z) compress -c -f ;; - *tar) cat ;; - *) warning "'%s' is not a valid archive extension." \ - "$PKGEXT"; cat ;; -esac > "${pkg_file}"; ret=$? - -if (( ret )); then - error "Unable to write package to $pkg_dest" - plain " Maybe the disk is full or you do not have write access" + pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" + ret=0 + + # TODO: Maybe this can be set globally for robustness + shopt -s -o pipefail + bsdtar -cf - $comp_files * | + case "$PKGEXT" in + *tar.gz) gzip -c -f -n ;; + *tar.bz2) bzip2 -c -f ;; + *tar.xz) xz -c -z - ;; + *tar.Z) compress -c -f ;; + *tar) cat ;; + *) warning "'%s' is not a valid archive extension." \ + "$PKGEXT"; cat ;; + esac > "${pkg_file}"; ret=$? + + # Move compressed package to destination + if (( ret )); then + error "Unable to write package to $pkg_dest" + plain " Maybe the disk is full or you do not have write access" + rm -rf "$work_dir" + exit 1 + fi + + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi + msg "Done." +} -rm -rf "$work_dir" +# Run fakebuild in parralel with at maximum $MAX_JOBS jobs +# By default only run one job +MAX_JOBS=${MAX_JOBS:-1} +parallelize() { + while [[ $# -gt 0 ]] ; do + job_count=($(jobs -p)) + if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then + fakebuild "$1" & + shift + fi + done + wait +} -msg "Done." +# Initiate assembly function +if [[ $MAX_JOBS -gt "1" ]]; then + parallelize "$@" +else + for PKG in $@; do fakebuild $PKG; done +fi exit 0 -- 2.9.2
* Add option for printing fewer status updates * Add option for controlling the output directory * Add option for specyfying the the maximum number of jobs * Adjust output to the number of jobs * Rewrite usage page * Alter version information Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 154 +++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 113 insertions(+), 41 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 4cd78e4..1df84db 100755 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -28,6 +28,7 @@ declare -r myname='bacman' declare -r myver='@PACKAGE_VERSION@' USE_COLOR='y' INCLUDE_PACNEW='n' +QUIET='n' # Required for fakeroot because options are shifted off the array. ARGS=("$@") @@ -40,57 +41,111 @@ clean_up() { echo exit } - # Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM # Print usage information usage() { - echo "${myname} (pacman) v${myver}" - echo - echo "Recreate a package using pacman's database and system files" - echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" - echo - echo "Example: ${myname} linux-headers" + cat <<-EOF + $myname (pacman) v$myver + This script was design to reassemble installed packages from its deliverd files. + It comes in handy if you have no access to an up-to-date package cache. + + Usage: $myname [-v] [-p] [-n] [-j <jobs>] [-o <dir>] <package(s)> + -h, --help Display this help message and exit + -q, --quiet Silence most of the status reporting + -n, --nocolor Disable colored output + -p, --pacnew Package .pacnew files + -j, --jobs <jobs> Build in parallel - you may want to set XZ_OPT + -o, --out <dir> Write output to <dir> + + Examples: # $myname linux-headers + # $myname gzip munge binutils -o ~/Downloads + # $myname -pno /tmp -j 5 gzip munge binutils + # $myname \$(pacman -Qsq) + EOF } # Print version information version() { - printf "%s %s\n" "$myname" "$myver" - echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' - echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' -} + cat <<-EOF + $myname (pacman) v$myver -# Check for specified arguments -while [[ ! -z $1 ]]; do - if [[ $1 == "--nocolor" ]]; then - USE_COLOR='n' - shift - elif [[ $1 == "--pacnew" ]]; then - INCLUDE_PACNEW='y' - shift - else - break - fi -done + Copyright (C) 2008 locci <carlocci_at_gmail_dot_com> + Copyright (C) 2016 Gordian Edenhofer <gordian.edenhofer@gmail.com> + Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org> + EOF +} # Configure colored output m4_include(../scripts/library/term_colors.sh) # Break if no argument was given -if (( $# < 1 )); then +if (( $# == 0 )); then usage exit 1 fi -# Print usage or version if requested -if [[ $1 = -@(h|-help) ]]; then +# Printing the usage information takes precedence over every other parameter +for option in "$@"; do + [[ $option == "-h" || $option == "--help" ]] && usage && exit 0 +done + +# Parse arguments and create a list of packages to be assembled +params=$(getopt -o o:j:qnpv --long out:,jobs:,quiets,nocolor,pacnew,version -n "$myname" -- "$@") +[[ $? -ne 0 ]] && echo "Try '$myname --help' for more information." && exit 1 +eval set -- "$params" +while true; do + case "$1" in + -o|--out) + pkg_dest="$(readlink -e $2)" + [[ $? -ne 0 ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 + shift + ;; + + -j|--jobs) + if [[ $2 =~ ^-?[0-9]+$ ]]; then + MAX_JOBS=$2 + else + echo -e "\e[39;1m$2\e[0m is not a valid integer!" + exit -1 + fi + shift + ;; + + -q|--quiet) + QUIET='y' + ;; + + -n|--nocolor) + USE_COLOR='n' + ;; + + -p|--pacnew) + INCLUDE_PACNEW='y' + ;; + + -v|--version) + version + exit 0 + ;; + + --) + shift + break + ;; + + *) + usage + exit 1 + ;; + esac + shift +done +pkg_list=($*) +if [[ ${#pkg_list[@]} == 0 ]]; then usage - exit 0 -elif [[ $1 = -@(V|-version) ]]; then - version - exit 0 + exit 1 fi # Run in fakeroot if EUID is not root @@ -106,7 +161,7 @@ if (( EUID )); then fi fi -# Source environmental variables and specify fallbacks +# Read in environmental variables if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 @@ -121,8 +176,9 @@ source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi -pkg_dest="${PKGDEST:-$PWD}" -pkg_pkger=${PACKAGER:-'Unknown Packager'} +PKGDEST="${PKGDEST:-$PWD}" +pkg_dest="${pkg_dest:-$PKGDEST}" +pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database if [[ ! -d $pac_db ]]; then @@ -155,7 +211,12 @@ fakebuild() { cd "$work_dir" || exit 1 # Assemble list of files which belong to the package and tar them - msg2 "Copying package files..." + if [[ $MAX_JOBS -gt 1 && "$QUIET" != "y" ]]; then + msg2 "${pkg_name}: Copying package files..." + elif [[ "$QUIET" != "y" ]]; then + msg2 "Copying package files..." + fi + while read i; do if [[ -z $i ]]; then continue @@ -229,7 +290,11 @@ fakebuild() { # Reconstruct .PKGINFO from database # TODO adopt makepkg's write_pkginfo() into this or scripts/library - msg2 "Generating .PKGINFO metadata..." + if [[ $MAX_JOBS -gt 1 && "$QUIET" != "y" ]]; then + msg2 "${pkg_name}: Generating .PKGINFO metadata..." + elif [[ "$QUIET" != "y" ]]; then + msg2 "Generating .PKGINFO metadata..." + fi echo "# Generated by $myname $myver" > .PKGINFO if [[ $INFAKEROOT == "1" ]]; then echo "# Using $(fakeroot -v)" >> .PKGINFO @@ -317,11 +382,16 @@ fakebuild() { chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null # Generate the package - msg2 "Generating the package..." + if [[ $MAX_JOBS -gt 1 && "$QUIET" != "y" ]]; then + msg2 "${pkg_name}: Generating the package..." + elif [[ "$QUIET" != "y" ]]; then + msg2 "Generating the package..." + fi pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 + # Move compressed package to destination # TODO: Maybe this can be set globally for robustness shopt -s -o pipefail bsdtar -cf - $comp_files * | @@ -335,7 +405,7 @@ fakebuild() { "$PKGEXT"; cat ;; esac > "${pkg_file}"; ret=$? - # Move compressed package to destination + # Evaluate return code if (( ret )); then error "Unable to write package to $pkg_dest" plain " Maybe the disk is full or you do not have write access" @@ -345,7 +415,6 @@ fakebuild() { # Clean up working directory rm -rf "$work_dir" - msg "Done." } # Run fakebuild in parralel with at maximum $MAX_JOBS jobs @@ -364,11 +433,14 @@ parallelize() { # Initiate assembly function if [[ $MAX_JOBS -gt "1" ]]; then - parallelize "$@" + parallelize "${pkg_list[@]}" else - for PKG in $@; do fakebuild $PKG; done + for PKG in ${pkg_list[@]}; do + fakebuild $PKG + done fi +msg "Done." exit 0 # vim: set noet: -- 2.9.2
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- doc/.gitignore | 1 + doc/Makefile.am | 4 +++ doc/bacman.8.txt | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 79 insertions(+) create mode 100644 doc/bacman.8.txt diff --git a/doc/.gitignore b/doc/.gitignore index 2eae9e4..80d0026 100644 --- a/doc/.gitignore +++ b/doc/.gitignore @@ -7,6 +7,7 @@ makepkg.conf.5 pacman.8 pacman-key.8 pacman.conf.5 +bacman.8 pactree.8 pkgdelta.8 repo-add.8 diff --git a/doc/Makefile.am b/doc/Makefile.am index 5d7c975..88f23cc 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -13,6 +13,7 @@ ASCIIDOC_MANS = \ pkgdelta.8 \ pacman-key.8 \ pactree.8 \ + bacman.8 \ PKGBUILD.5 \ makepkg.conf.5 \ pacman.conf.5 \ @@ -30,6 +31,7 @@ HTML_MANPAGES = \ pkgdelta.8.html \ pacman-key.8.html \ pactree.8.html \ + bacman.8.html \ PKGBUILD.5.html \ makepkg.conf.5.html \ pacman.conf.5.html \ @@ -57,6 +59,7 @@ EXTRA_DIST = \ pkgdelta.8.txt \ pacman-key.8.txt \ pactree.8.txt \ + bacman.8.txt \ PKGBUILD.5.txt \ PKGBUILD-example.txt \ makepkg.conf.5.txt \ @@ -160,6 +163,7 @@ vercmp.8 vercmp.8.html: vercmp.8.txt pkgdelta.8 pkgdelta.8.html: pkgdelta.8.txt pacman-key.8 pacman-key.8.html: pacman-key.8.txt pactree.8 pactree.8.html: pactree.8.txt +bacman.8 bacman.8.html: bacman.8.txt PKGBUILD.5 PKGBUILD.5.html: PKGBUILD.5.txt PKGBUILD-example.txt makepkg.conf.5 makepkg.conf.5.html: makepkg.conf.5.txt pacman.conf.5 pacman.conf.5.html: pacman.conf.5.txt diff --git a/doc/bacman.8.txt b/doc/bacman.8.txt new file mode 100644 index 0000000..b78bbbe --- /dev/null +++ b/doc/bacman.8.txt @@ -0,0 +1,74 @@ +///// +vim:set ts=4 sw=4 syntax=asciidoc noet spell spelllang=en_us: +///// +bacman(8) +========== + +Name +---- +bacman - reassemble installed packages + + +Synopsis +-------- +'bacman' [options] <package(s)> + + +Description +----------- +'bacman' was designed to reassemble installed packages from its deliverd files. +It comes in handy if there is no internet connection available and you have no +access to a up-to-date package cache. Otherwise packages from the Arch Linux +Archive should always be preferred. + +'bacman' fully honors linkman:makepkg.conf[8] and all compression environment +variables: namely ``XZ_OPT''. + + +Options +------- +*-h, \--help*:: + Display usage information. + +*-q, \--quiet*:: + Silence most of the status reporting. + +*-n, \--nocolor*:: + Disable colored output. + +*-p, \--pacnew*:: + Package .pacnew files instead of the concerning files currently in place. + +*-p, \--pacnew*:: + Package .pacnew files instead of the concerning files currently in place. + +*-j, \--jobs <jobs>*:: + Assemble multiple packages in parallel. + +*-o, \--out <dir>*:: + Write the assembled package(s) to the specified directory. + + +Examples +-------- +'bacman' linux-headers + +Recreate the package ``linux-headers''. + +'bacman' gzip munge binutils -o ~/Downloads + +Assemble the packages gzip munge binutils and place the packages at +``~/Downloads''. + +$myname -pno ~/backup -j 5 \$(pacman -Qsq) + +Assemble all currently installed packages using 5 concurrent jobs, use +``.pacnew'' whenever available, suppress colored output and place the desired +packages in ``~/backup''. + + +See Also +-------- +linkman:makepkg[8], linkman:pacman[8] + +include::footer.txt[] -- 2.9.2
On Sun, Aug 14, 2016 at 08:10:52PM +0200, Gordian Edenhofer wrote:
* move the actual assembly process into its own function * allow for packaging multiple packages with one command * handle SIGHUP SIGINT SIGTERM and remove working dirs accordingly * add some comments
Can't this be done already by running multiple instances of bacman? In other words, what's gained by adding this complexity to bacman itself? Bonus, if you're on a system that has bash 4.3, you can do better than the parallelize function you wrote and avoiding a tight loop and the CPU it'll eat up: pbacman() { local max_jobs=$1; shift for arg; do while $(( $(jobs -p | wc -l) > max_jobs )); do wait -n done bacman "$arg" & done wait } We can't use this in bacman without bumping the bash version requirement which probably isn't something the pacman maintainers are willing to do.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 460 ++++++++++++++++++++++++++------------------------- 1 file changed, 238 insertions(+), 222 deletions(-) mode change 100644 => 100755 contrib/bacman.sh.in
diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in old mode 100644 new mode 100755 index a611c1a..4cd78e4 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -33,9 +33,18 @@ ARGS=("$@")
m4_include(../scripts/library/output_format.sh)
-# -# User Friendliness -# +# Lazy recursive clean up of temporary dirs +work_dir_root="${TMPDIR:-/tmp}/bacman" +clean_up() { + rm -r "${work_dir_root}".* + echo + exit +} + +# Trap termination signals +trap clean_up SIGHUP SIGINT SIGTERM + +# Print usage information usage() { echo "${myname} (pacman) v${myver}" echo @@ -46,12 +55,14 @@ usage() { echo "Example: ${myname} linux-headers" }
+# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' }
+# Check for specified arguments while [[ ! -z $1 ]]; do if [[ $1 == "--nocolor" ]]; then USE_COLOR='n' @@ -64,13 +75,16 @@ while [[ ! -z $1 ]]; do fi done
+# Configure colored output m4_include(../scripts/library/term_colors.sh)
-if (( $# != 1 )); then +# Break if no argument was given +if (( $# < 1 )); then usage exit 1 fi
+# Print usage or version if requested if [[ $1 = -@(h|-help) ]]; then usage exit 0 @@ -79,9 +93,7 @@ elif [[ $1 = -@(V|-version) ]]; then exit 0 fi
-# -# Fakeroot support -# +# Run in fakeroot if EUID is not root if (( EUID )); then if [[ -f /usr/bin/fakeroot ]]; then msg "Entering fakeroot environment" @@ -94,264 +106,268 @@ if (( EUID )); then fi fi
-# -# Setting environmental variables -# +# Source environmental variables and specify fallbacks if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 fi - eval $(awk '/DBPath/ {print $1$2$3}' @sysconfdir@/pacman.conf) pac_db="${DBPath:-@localstatedir@/lib/pacman/}/local" - if [[ ! -r @sysconfdir@/makepkg.conf ]]; then error "unable to read @sysconfdir@/makepkg.conf" exit 1 fi - source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi - pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'}
-pkg_name="$1" -pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) -pkg_namver=("${pkg_dir[@]##*/}") - -# -# Checks everything is in place -# +# Check for an existing database if [[ ! -d $pac_db ]]; then error "pacman database directory ${pac_db} not found" exit 1 fi
-if (( ${#pkg_dir[@]} != 1 )); then - error "%d entries for package %s found in pacman database" \ - ${#pkg_dir[@]} "${pkg_name}" - msg2 "%s" "${pkg_dir[@]}" - exit 1 -fi - -if [[ ! -d $pkg_dir ]]; then - error "package %s is found in pacman database," "${pkg_name}" - plain " but '%s' is not a directory" "${pkg_dir}" - exit 1 -fi - -# -# Begin -# -msg "Package: ${pkg_namver}" -work_dir=$(mktemp -d "${TMPDIR:-/tmp}/bacman.XXXXXXXXXX") -cd "$work_dir" || exit 1 - -# -# File copying -# -msg2 "Copying package files..." - -while read i; do - if [[ -z $i ]]; then - continue +# Assemble a single package: $1 = pkgname +fakebuild() { + pkg_name="$1" + pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) + pkg_namver=("${pkg_dir[@]##*/}") + + # Checks database for specified package + if (( ${#pkg_dir[@]} != 1 )); then + error "%d entries for package %s found in pacman database" \ + ${#pkg_dir[@]} "${pkg_name}" + msg2 "%s" "${pkg_dir[@]}" + exit 1 fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + if [[ ! -d $pkg_dir ]]; then + error "package %s is found in pacman database," "${pkg_name}" + plain " but '%s' is not a directory" "${pkg_dir}" + exit 1 fi
- case "$current" in - %FILES%) - local_file="/$i" - package_file="$work_dir/$i" + # Create working directory + msg "Package: ${pkg_namver}" + work_dir=$(mktemp -d "${work_dir_root}.XXXXXXXXXX") + cd "$work_dir" || exit 1
- if [[ ! -e $local_file ]]; then - warning "package file $local_file is missing" - continue - fi - ;; - - %BACKUP%) - # Get the MD5 checksum. - original_md5="${i##*$'\t'}" - # Strip the md5sum after the tab. - i="${i%$'\t'*}" - local_file="/$i.pacnew" - package_file="$work_dir/$i" - - # Include unmodified .pacnew files. - local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" - if [[ $INCLUDE_PACNEW == 'n' ]] \ - || [[ ! -e $local_file ]] \ - || [[ $local_md5 != $original_md5 ]]; then - # Warn about modified files. - local_md5="$(md5sum "/$i" | cut -d' ' -f1)" - if [[ $local_md5 != $original_md5 ]]; then - warning "package file /$i has been modified" - fi - # Let the normal file be included in the %FILES% list. - continue - fi - ;; + # Assemble list of files which belong to the package and tar them + msg2 "Copying package files..." + while read i; do + if [[ -z $i ]]; then + continue + fi
- *) + if [[ $i == %+([A-Z])% ]]; then + current=$i continue - ;; - esac + fi
- ret=0 - bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + case "$current" in + %FILES%) + local_file="/$i" + package_file="$work_dir/$i" + + if [[ ! -e $local_file ]]; then + warning "package file $local_file is missing" + continue + fi + ;; + + %BACKUP%) + # Get the MD5 checksum. + original_md5="${i##*$'\t'}" + # Strip the md5sum after the tab. + i="${i%$'\t'*}" + local_file="/$i.pacnew" + package_file="$work_dir/$i" + + # Include unmodified .pacnew files. + local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" + if [[ $INCLUDE_PACNEW == 'n' ]] \ + || [[ ! -e $local_file ]] \ + || [[ $local_md5 != $original_md5 ]]; then + # Warn about modified files. + local_md5="$(md5sum "/$i" | cut -d' ' -f1)" + if [[ $local_md5 != $original_md5 ]]; then + warning "package file /$i has been modified" + fi + # Let the normal file be included in the %FILES% list. + continue + fi + ;;
- # Workaround to bsdtar not reporting a missing file as an error - if ! [[ -e $package_file || -L $package_file ]]; then - error "unable to add $local_file to the package" - plain " If your user does not have permission to read this file, then" - plain " you will need to run $myname as root." + *) + continue + ;; + esac + + # Tar files + ret=0 + bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + # Workaround to bsdtar not reporting a missing file as an error + if ! [[ -e $package_file || -L $package_file ]]; then + error "unable to add $local_file to the package" + plain " If your user does not have permission to read this file, then" + plain " you will need to run $myname as root." + rm -rf "$work_dir" + exit 1 + fi + done < "$pkg_dir"/files + + ret=$? + if (( ret )); then rm -rf "$work_dir" exit 1 fi -done < "$pkg_dir"/files
-ret=$? -if (( ret )); then - rm -rf "$work_dir" - exit 1 -fi + # Calculate package size + pkg_size=$(du -sk | awk '{print $1 * 1024}')
-pkg_size=$(du -sk | awk '{print $1 * 1024}') - -# -# .PKGINFO stuff -# TODO adopt makepkg's write_pkginfo() into this or scripts/library -# -msg2 "Generating .PKGINFO metadata..." -echo "# Generated by $myname $myver" > .PKGINFO -if [[ $INFAKEROOT == "1" ]]; then - echo "# Using $(fakeroot -v)" >> .PKGINFO -fi -echo "# $(LC_ALL=C date)" >> .PKGINFO -echo "#" >> .PKGINFO - -while read i; do - if [[ -z $i ]]; then - continue; + # Reconstruct .PKGINFO from database + # TODO adopt makepkg's write_pkginfo() into this or scripts/library + msg2 "Generating .PKGINFO metadata..." + echo "# Generated by $myname $myver" > .PKGINFO + if [[ $INFAKEROOT == "1" ]]; then + echo "# Using $(fakeroot -v)" >> .PKGINFO fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + echo "# $(LC_ALL=C date)" >> .PKGINFO + echo "#" >> .PKGINFO + while read i; do + if [[ -z $i ]]; then + continue; + fi + if [[ $i == %+([A-Z])% ]]; then + current=$i + continue + fi + + case "$current" in + # desc + %NAME%) + echo "pkgname = $i" >> .PKGINFO + ;; + %VERSION%) + echo "pkgver = $i" >> .PKGINFO + ;; + %DESC%) + echo "pkgdesc = $i" >> .PKGINFO + ;; + %URL%) + echo "url = $i" >> .PKGINFO + ;; + %LICENSE%) + echo "license = $i" >> .PKGINFO + ;; + %ARCH%) + echo "arch = $i" >> .PKGINFO + pkg_arch="$i" + ;; + %BUILDDATE%) + echo "builddate = $(date -u "+%s")" >> .PKGINFO + ;; + %PACKAGER%) + echo "packager = $pkg_pkger" >> .PKGINFO + ;; + %SIZE%) + echo "size = $pkg_size" >> .PKGINFO + ;; + %GROUPS%) + echo "group = $i" >> .PKGINFO + ;; + %REPLACES%) + echo "replaces = $i" >> .PKGINFO + ;; + %DEPENDS%) + echo "depend = $i" >> .PKGINFO + ;; + %OPTDEPENDS%) + echo "optdepend = $i" >> .PKGINFO + ;; + %CONFLICTS%) + echo "conflict = $i" >> .PKGINFO + ;; + %PROVIDES%) + echo "provides = $i" >> .PKGINFO + ;; + %BACKUP%) + # Strip the md5sum after the tab + echo "backup = ${i%%$'\t'*}" >> .PKGINFO + ;; + esac + done < <(cat "$pkg_dir"/{desc,files}) + + comp_files=".PKGINFO" + + # Add instal file if present + if [[ -f $pkg_dir/install ]]; then + cp "$pkg_dir/install" "$work_dir/.INSTALL" + comp_files+=" .INSTALL" + fi + if [[ -f $pkg_dir/changelog ]]; then + cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" + comp_files+=" .CHANGELOG" fi
- case "$current" in - # desc - %NAME%) - echo "pkgname = $i" >> .PKGINFO - ;; - %VERSION%) - echo "pkgver = $i" >> .PKGINFO - ;; - %DESC%) - echo "pkgdesc = $i" >> .PKGINFO - ;; - %URL%) - echo "url = $i" >> .PKGINFO - ;; - %LICENSE%) - echo "license = $i" >> .PKGINFO - ;; - %ARCH%) - echo "arch = $i" >> .PKGINFO - pkg_arch="$i" - ;; - %BUILDDATE%) - echo "builddate = $(date -u "+%s")" >> .PKGINFO - ;; - %PACKAGER%) - echo "packager = $pkg_pkger" >> .PKGINFO - ;; - %SIZE%) - echo "size = $pkg_size" >> .PKGINFO - ;; - %GROUPS%) - echo "group = $i" >> .PKGINFO - ;; - %REPLACES%) - echo "replaces = $i" >> .PKGINFO - ;; - %DEPENDS%) - echo "depend = $i" >> .PKGINFO - ;; - %OPTDEPENDS%) - echo "optdepend = $i" >> .PKGINFO - ;; - %CONFLICTS%) - echo "conflict = $i" >> .PKGINFO - ;; - %PROVIDES%) - echo "provides = $i" >> .PKGINFO - ;; - - # files - %BACKUP%) - # Strip the md5sum after the tab - echo "backup = ${i%%$'\t'*}" >> .PKGINFO - ;; - esac -done < <(cat "$pkg_dir"/{desc,files}) - -comp_files=".PKGINFO" - -if [[ -f $pkg_dir/install ]]; then - cp "$pkg_dir/install" "$work_dir/.INSTALL" - comp_files+=" .INSTALL" -fi -if [[ -f $pkg_dir/changelog ]]; then - cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" - comp_files+=" .CHANGELOG" -fi + # Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL + chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null
-# -# Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL -# -chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + # Generate the package + msg2 "Generating the package..."
-# -# Generate the package -# -msg2 "Generating the package..." - -pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" -ret=0 - -# TODO: Maybe this can be set globally for robustness -shopt -s -o pipefail -bsdtar -cf - $comp_files * | -case "$PKGEXT" in - *tar.gz) gzip -c -f -n ;; - *tar.bz2) bzip2 -c -f ;; - *tar.xz) xz -c -z - ;; - *tar.Z) compress -c -f ;; - *tar) cat ;; - *) warning "'%s' is not a valid archive extension." \ - "$PKGEXT"; cat ;; -esac > "${pkg_file}"; ret=$? - -if (( ret )); then - error "Unable to write package to $pkg_dest" - plain " Maybe the disk is full or you do not have write access" + pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" + ret=0 + + # TODO: Maybe this can be set globally for robustness + shopt -s -o pipefail + bsdtar -cf - $comp_files * | + case "$PKGEXT" in + *tar.gz) gzip -c -f -n ;; + *tar.bz2) bzip2 -c -f ;; + *tar.xz) xz -c -z - ;; + *tar.Z) compress -c -f ;; + *tar) cat ;; + *) warning "'%s' is not a valid archive extension." \ + "$PKGEXT"; cat ;; + esac > "${pkg_file}"; ret=$? + + # Move compressed package to destination + if (( ret )); then + error "Unable to write package to $pkg_dest" + plain " Maybe the disk is full or you do not have write access" + rm -rf "$work_dir" + exit 1 + fi + + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi + msg "Done." +}
-rm -rf "$work_dir" +# Run fakebuild in parralel with at maximum $MAX_JOBS jobs +# By default only run one job +MAX_JOBS=${MAX_JOBS:-1} +parallelize() { + while [[ $# -gt 0 ]] ; do + job_count=($(jobs -p)) + if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then + fakebuild "$1" & + shift + fi + done + wait +}
-msg "Done." +# Initiate assembly function +if [[ $MAX_JOBS -gt "1" ]]; then + parallelize "$@" +else + for PKG in $@; do fakebuild $PKG; done +fi
exit 0
-- 2.9.2
On Sun, 2016-08-14 at 14:23 -0400, Dave Reisner wrote:
On Sun, Aug 14, 2016 at 08:10:52PM +0200, Gordian Edenhofer wrote:
* move the actual assembly process into its own function * allow for packaging multiple packages with one command * handle SIGHUP SIGINT SIGTERM and remove working dirs accordingly * add some comments
Can't this be done already by running multiple instances of bacman? In other words, what's gained by adding this complexity to bacman itself?
Yes, sure this basically achieves the same purpose. Though I would argue the added complexity is negligible since it merely puts the assembly process in its own function, nothing more. The git-diff may suggest something different but this is mainly due to the changed indentation.
Bonus, if you're on a system that has bash 4.3, you can do better than the parallelize function you wrote and avoiding a tight loop and the CPU it'll eat up:
pbacman() { local max_jobs=$1; shift
for arg; do while $(( $(jobs -p | wc -l) > max_jobs )); do wait -n done
bacman "$arg" & done wait }
We can't use this in bacman without bumping the bash version requirement which probably isn't something the pacman maintainers are willing to do.
I am aware this probably isn't an elegant way to do things but it is simple and gets the job done. The CPU usage can be drastically decreased by adding a short sleep in between the checks. But I guess there is still room for improvement.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 460 ++++++++++++++++++++++++++------------- ------------ 1 file changed, 238 insertions(+), 222 deletions(-) mode change 100644 => 100755 contrib/bacman.sh.in
diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in old mode 100644 new mode 100755 index a611c1a..4cd78e4 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -33,9 +33,18 @@ ARGS=("$@") m4_include(../scripts/library/output_format.sh) -# -# User Friendliness -# +# Lazy recursive clean up of temporary dirs +work_dir_root="${TMPDIR:-/tmp}/bacman" +clean_up() { + rm -r "${work_dir_root}".* + echo + exit +} + +# Trap termination signals +trap clean_up SIGHUP SIGINT SIGTERM + +# Print usage information usage() { echo "${myname} (pacman) v${myver}" echo @@ -46,12 +55,14 @@ usage() { echo "Example: ${myname} linux-headers" } +# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pac man-dev@archlinux.org>' } +# Check for specified arguments while [[ ! -z $1 ]]; do if [[ $1 == "--nocolor" ]]; then USE_COLOR='n' @@ -64,13 +75,16 @@ while [[ ! -z $1 ]]; do fi done +# Configure colored output m4_include(../scripts/library/term_colors.sh) -if (( $# != 1 )); then +# Break if no argument was given +if (( $# < 1 )); then usage exit 1 fi +# Print usage or version if requested if [[ $1 = -@(h|-help) ]]; then usage exit 0 @@ -79,9 +93,7 @@ elif [[ $1 = -@(V|-version) ]]; then exit 0 fi -# -# Fakeroot support -# +# Run in fakeroot if EUID is not root if (( EUID )); then if [[ -f /usr/bin/fakeroot ]]; then msg "Entering fakeroot environment" @@ -94,264 +106,268 @@ if (( EUID )); then fi fi -# -# Setting environmental variables -# +# Source environmental variables and specify fallbacks if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 fi - eval $(awk '/DBPath/ {print $1$2$3}' @sysconfdir@/pacman.conf) pac_db="${DBPath:-@localstatedir@/lib/pacman/}/local" - if [[ ! -r @sysconfdir@/makepkg.conf ]]; then error "unable to read @sysconfdir@/makepkg.conf" exit 1 fi - source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi - pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -pkg_name="$1" -pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) -pkg_namver=("${pkg_dir[@]##*/}") - -# -# Checks everything is in place -# +# Check for an existing database if [[ ! -d $pac_db ]]; then error "pacman database directory ${pac_db} not found" exit 1 fi -if (( ${#pkg_dir[@]} != 1 )); then - error "%d entries for package %s found in pacman database" \ - ${#pkg_dir[@]} "${pkg_name}" - msg2 "%s" "${pkg_dir[@]}" - exit 1 -fi - -if [[ ! -d $pkg_dir ]]; then - error "package %s is found in pacman database," "${pkg_name}" - plain " but '%s' is not a directory" "${pkg_dir}" - exit 1 -fi - -# -# Begin -# -msg "Package: ${pkg_namver}" -work_dir=$(mktemp -d "${TMPDIR:-/tmp}/bacman.XXXXXXXXXX") -cd "$work_dir" || exit 1 - -# -# File copying -# -msg2 "Copying package files..." - -while read i; do - if [[ -z $i ]]; then - continue +# Assemble a single package: $1 = pkgname +fakebuild() { + pkg_name="$1" + pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) + pkg_namver=("${pkg_dir[@]##*/}") + + # Checks database for specified package + if (( ${#pkg_dir[@]} != 1 )); then + error "%d entries for package %s found in pacman database" \ + ${#pkg_dir[@]} "${pkg_name}" + msg2 "%s" "${pkg_dir[@]}" + exit 1 fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + if [[ ! -d $pkg_dir ]]; then + error "package %s is found in pacman database," "${pkg_name}" + plain " but '%s' is not a directory" "${pkg_dir}" + exit 1 fi - case "$current" in - %FILES%) - local_file="/$i" - package_file="$work_dir/$i" + # Create working directory + msg "Package: ${pkg_namver}" + work_dir=$(mktemp -d "${work_dir_root}.XXXXXXXXXX") + cd "$work_dir" || exit 1 - if [[ ! -e $local_file ]]; then - warning "package file $local_file is missing" - continue - fi - ;; - - %BACKUP%) - # Get the MD5 checksum. - original_md5="${i##*$'\t'}" - # Strip the md5sum after the tab. - i="${i%$'\t'*}" - local_file="/$i.pacnew" - package_file="$work_dir/$i" - - # Include unmodified .pacnew files. - local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" - if [[ $INCLUDE_PACNEW == 'n' ]] \ - || [[ ! -e $local_file ]] \ - || [[ $local_md5 != $original_md5 ]]; then - # Warn about modified files. - local_md5="$(md5sum "/$i" | cut -d' ' -f1)" - if [[ $local_md5 != $original_md5 ]]; then - warning "package file /$i has been modified" - fi - # Let the normal file be included in the %FILES% list. - continue - fi - ;; + # Assemble list of files which belong to the package and tar them + msg2 "Copying package files..." + while read i; do + if [[ -z $i ]]; then + continue + fi - *) + if [[ $i == %+([A-Z])% ]]; then + current=$i continue - ;; - esac + fi - ret=0 - bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + case "$current" in + %FILES%) + local_file="/$i" + package_file="$work_dir/$i" + + if [[ ! -e $local_file ]]; then + warning "package file $local_file is missing" + continue + fi + ;; + + %BACKUP%) + # Get the MD5 checksum. + original_md5="${i##*$'\t'}" + # Strip the md5sum after the tab. + i="${i%$'\t'*}" + local_file="/$i.pacnew" + package_file="$work_dir/$i" + + # Include unmodified .pacnew files. + local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" + if [[ $INCLUDE_PACNEW == 'n' ]] \ + || [[ ! -e $local_file ]] \ + || [[ $local_md5 != $original_md5 ]]; then + # Warn about modified files. + local_md5="$(md5sum "/$i" | cut -d' ' -f1)" + if [[ $local_md5 != $original_md5 ]]; then + warning "package file /$i has been modified" + fi + # Let the normal file be included in the %FILES% list. + continue + fi + ;; - # Workaround to bsdtar not reporting a missing file as an error - if ! [[ -e $package_file || -L $package_file ]]; then - error "unable to add $local_file to the package" - plain " If your user does not have permission to read this file, then" - plain " you will need to run $myname as root." + *) + continue + ;; + esac + + # Tar files + ret=0 + bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + # Workaround to bsdtar not reporting a missing file as an error + if ! [[ -e $package_file || -L $package_file ]]; then + error "unable to add $local_file to the package" + plain " If your user does not have permission to read this file, then" + plain " you will need to run $myname as root." + rm -rf "$work_dir" + exit 1 + fi + done < "$pkg_dir"/files + + ret=$? + if (( ret )); then rm -rf "$work_dir" exit 1 fi -done < "$pkg_dir"/files -ret=$? -if (( ret )); then - rm -rf "$work_dir" - exit 1 -fi + # Calculate package size + pkg_size=$(du -sk | awk '{print $1 * 1024}') -pkg_size=$(du -sk | awk '{print $1 * 1024}') - -# -# .PKGINFO stuff -# TODO adopt makepkg's write_pkginfo() into this or scripts/library -# -msg2 "Generating .PKGINFO metadata..." -echo "# Generated by $myname $myver" > .PKGINFO -if [[ $INFAKEROOT == "1" ]]; then - echo "# Using $(fakeroot -v)" >> .PKGINFO -fi -echo "# $(LC_ALL=C date)" >> .PKGINFO -echo "#" >> .PKGINFO - -while read i; do - if [[ -z $i ]]; then - continue; + # Reconstruct .PKGINFO from database + # TODO adopt makepkg's write_pkginfo() into this or scripts/library + msg2 "Generating .PKGINFO metadata..." + echo "# Generated by $myname $myver" > .PKGINFO + if [[ $INFAKEROOT == "1" ]]; then + echo "# Using $(fakeroot -v)" >> .PKGINFO fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + echo "# $(LC_ALL=C date)" >> .PKGINFO + echo "#" >> .PKGINFO + while read i; do + if [[ -z $i ]]; then + continue; + fi + if [[ $i == %+([A-Z])% ]]; then + current=$i + continue + fi + + case "$current" in + # desc + %NAME%) + echo "pkgname = $i" >> .PKGINFO + ;; + %VERSION%) + echo "pkgver = $i" >> .PKGINFO + ;; + %DESC%) + echo "pkgdesc = $i" >> .PKGINFO + ;; + %URL%) + echo "url = $i" >> .PKGINFO + ;; + %LICENSE%) + echo "license = $i" >> .PKGINFO + ;; + %ARCH%) + echo "arch = $i" >> .PKGINFO + pkg_arch="$i" + ;; + %BUILDDATE%) + echo "builddate = $(date -u "+%s")" >> .PKGINFO + ;; + %PACKAGER%) + echo "packager = $pkg_pkger" >> .PKGINFO + ;; + %SIZE%) + echo "size = $pkg_size" >> .PKGINFO + ;; + %GROUPS%) + echo "group = $i" >> .PKGINFO + ;; + %REPLACES%) + echo "replaces = $i" >> .PKGINFO + ;; + %DEPENDS%) + echo "depend = $i" >> .PKGINFO + ;; + %OPTDEPENDS%) + echo "optdepend = $i" >> .PKGINFO + ;; + %CONFLICTS%) + echo "conflict = $i" >> .PKGINFO + ;; + %PROVIDES%) + echo "provides = $i" >> .PKGINFO + ;; + %BACKUP%) + # Strip the md5sum after the tab + echo "backup = ${i%%$'\t'*}" >> .PKGINFO + ;; + esac + done < <(cat "$pkg_dir"/{desc,files}) + + comp_files=".PKGINFO" + + # Add instal file if present + if [[ -f $pkg_dir/install ]]; then + cp "$pkg_dir/install" "$work_dir/.INSTALL" + comp_files+=" .INSTALL" + fi + if [[ -f $pkg_dir/changelog ]]; then + cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" + comp_files+=" .CHANGELOG" fi - case "$current" in - # desc - %NAME%) - echo "pkgname = $i" >> .PKGINFO - ;; - %VERSION%) - echo "pkgver = $i" >> .PKGINFO - ;; - %DESC%) - echo "pkgdesc = $i" >> .PKGINFO - ;; - %URL%) - echo "url = $i" >> .PKGINFO - ;; - %LICENSE%) - echo "license = $i" >> .PKGINFO - ;; - %ARCH%) - echo "arch = $i" >> .PKGINFO - pkg_arch="$i" - ;; - %BUILDDATE%) - echo "builddate = $(date -u "+%s")" >> .PKGINFO - ;; - %PACKAGER%) - echo "packager = $pkg_pkger" >> .PKGINFO - ;; - %SIZE%) - echo "size = $pkg_size" >> .PKGINFO - ;; - %GROUPS%) - echo "group = $i" >> .PKGINFO - ;; - %REPLACES%) - echo "replaces = $i" >> .PKGINFO - ;; - %DEPENDS%) - echo "depend = $i" >> .PKGINFO - ;; - %OPTDEPENDS%) - echo "optdepend = $i" >> .PKGINFO - ;; - %CONFLICTS%) - echo "conflict = $i" >> .PKGINFO - ;; - %PROVIDES%) - echo "provides = $i" >> .PKGINFO - ;; - - # files - %BACKUP%) - # Strip the md5sum after the tab - echo "backup = ${i%%$'\t'*}" >> .PKGINFO - ;; - esac -done < <(cat "$pkg_dir"/{desc,files}) - -comp_files=".PKGINFO" - -if [[ -f $pkg_dir/install ]]; then - cp "$pkg_dir/install" "$work_dir/.INSTALL" - comp_files+=" .INSTALL" -fi -if [[ -f $pkg_dir/changelog ]]; then - cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" - comp_files+=" .CHANGELOG" -fi + # Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL + chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -# -# Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL -# -chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + # Generate the package + msg2 "Generating the package..." -# -# Generate the package -# -msg2 "Generating the package..." - -pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" -ret=0 - -# TODO: Maybe this can be set globally for robustness -shopt -s -o pipefail -bsdtar -cf - $comp_files * | -case "$PKGEXT" in - *tar.gz) gzip -c -f -n ;; - *tar.bz2) bzip2 -c -f ;; - *tar.xz) xz -c -z - ;; - *tar.Z) compress -c -f ;; - *tar) cat ;; - *) warning "'%s' is not a valid archive extension." \ - "$PKGEXT"; cat ;; -esac > "${pkg_file}"; ret=$? - -if (( ret )); then - error "Unable to write package to $pkg_dest" - plain " Maybe the disk is full or you do not have write access" + pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" + ret=0 + + # TODO: Maybe this can be set globally for robustness + shopt -s -o pipefail + bsdtar -cf - $comp_files * | + case "$PKGEXT" in + *tar.gz) gzip -c -f -n ;; + *tar.bz2) bzip2 -c -f ;; + *tar.xz) xz -c -z - ;; + *tar.Z) compress -c -f ;; + *tar) cat ;; + *) warning "'%s' is not a valid archive extension." \ + "$PKGEXT"; cat ;; + esac > "${pkg_file}"; ret=$? + + # Move compressed package to destination + if (( ret )); then + error "Unable to write package to $pkg_dest" + plain " Maybe the disk is full or you do not have write access" + rm -rf "$work_dir" + exit 1 + fi + + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi + msg "Done." +} -rm -rf "$work_dir" +# Run fakebuild in parralel with at maximum $MAX_JOBS jobs +# By default only run one job +MAX_JOBS=${MAX_JOBS:-1} +parallelize() { + while [[ $# -gt 0 ]] ; do + job_count=($(jobs -p)) + if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then + fakebuild "$1" & + shift + fi + done + wait +} -msg "Done." +# Initiate assembly function +if [[ $MAX_JOBS -gt "1" ]]; then + parallelize "$@" +else + for PKG in $@; do fakebuild $PKG; done +fi exit 0 -- 2.9.2
* move the actual assembly process into its own function * allow for packaging multiple packages with one command * handle SIGHUP SIGINT SIGTERM and remove working dirs accordingly * add some comments Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- * Clean up working directories with force on abort * Sleep for 100ms in between parallelization work contrib/bacman.sh.in | 461 ++++++++++++++++++++++++++------------------------- 1 file changed, 239 insertions(+), 222 deletions(-) mode change 100644 => 100755 contrib/bacman.sh.in diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in old mode 100644 new mode 100755 index a611c1a..6c9b1e5 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -33,9 +33,18 @@ ARGS=("$@") m4_include(../scripts/library/output_format.sh) -# -# User Friendliness -# +# Lazy recursive clean up of temporary dirs +work_dir_root="${TMPDIR:-/tmp}/bacman" +clean_up() { + rm -rf "$work_dir_root".* + echo + exit +} + +# Trap termination signals +trap clean_up SIGHUP SIGINT SIGTERM + +# Print usage information usage() { echo "${myname} (pacman) v${myver}" echo @@ -46,12 +55,14 @@ usage() { echo "Example: ${myname} linux-headers" } +# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } +# Check for specified arguments while [[ ! -z $1 ]]; do if [[ $1 == "--nocolor" ]]; then USE_COLOR='n' @@ -64,13 +75,16 @@ while [[ ! -z $1 ]]; do fi done +# Configure colored output m4_include(../scripts/library/term_colors.sh) -if (( $# != 1 )); then +# Break if no argument was given +if (( $# < 1 )); then usage exit 1 fi +# Print usage or version if requested if [[ $1 = -@(h|-help) ]]; then usage exit 0 @@ -79,9 +93,7 @@ elif [[ $1 = -@(V|-version) ]]; then exit 0 fi -# -# Fakeroot support -# +# Run with fake root privileges if EUID is not root if (( EUID )); then if [[ -f /usr/bin/fakeroot ]]; then msg "Entering fakeroot environment" @@ -94,264 +106,269 @@ if (( EUID )); then fi fi -# -# Setting environmental variables -# +# Source environmental variables and specify fallbacks if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 fi - eval $(awk '/DBPath/ {print $1$2$3}' @sysconfdir@/pacman.conf) pac_db="${DBPath:-@localstatedir@/lib/pacman/}/local" - if [[ ! -r @sysconfdir@/makepkg.conf ]]; then error "unable to read @sysconfdir@/makepkg.conf" exit 1 fi - source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi - pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -pkg_name="$1" -pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) -pkg_namver=("${pkg_dir[@]##*/}") - -# -# Checks everything is in place -# +# Check for an existing database if [[ ! -d $pac_db ]]; then error "pacman database directory ${pac_db} not found" exit 1 fi -if (( ${#pkg_dir[@]} != 1 )); then - error "%d entries for package %s found in pacman database" \ - ${#pkg_dir[@]} "${pkg_name}" - msg2 "%s" "${pkg_dir[@]}" - exit 1 -fi - -if [[ ! -d $pkg_dir ]]; then - error "package %s is found in pacman database," "${pkg_name}" - plain " but '%s' is not a directory" "${pkg_dir}" - exit 1 -fi - -# -# Begin -# -msg "Package: ${pkg_namver}" -work_dir=$(mktemp -d "${TMPDIR:-/tmp}/bacman.XXXXXXXXXX") -cd "$work_dir" || exit 1 - -# -# File copying -# -msg2 "Copying package files..." - -while read i; do - if [[ -z $i ]]; then - continue +# Assemble a single package: $1 = pkgname +fakebuild() { + pkg_name="$1" + pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) + pkg_namver=("${pkg_dir[@]##*/}") + + # Checks database for specified package + if (( ${#pkg_dir[@]} != 1 )); then + error "%d entries for package %s found in pacman database" \ + ${#pkg_dir[@]} "${pkg_name}" + msg2 "%s" "${pkg_dir[@]}" + exit 1 fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + if [[ ! -d $pkg_dir ]]; then + error "package %s is found in pacman database," "${pkg_name}" + plain " but '%s' is not a directory" "${pkg_dir}" + exit 1 fi - case "$current" in - %FILES%) - local_file="/$i" - package_file="$work_dir/$i" + # Create working directory + msg "Package: ${pkg_namver}" + work_dir=$(mktemp -d "${work_dir_root}.XXXXXXXXXX") + cd "$work_dir" || exit 1 - if [[ ! -e $local_file ]]; then - warning "package file $local_file is missing" - continue - fi - ;; - - %BACKUP%) - # Get the MD5 checksum. - original_md5="${i##*$'\t'}" - # Strip the md5sum after the tab. - i="${i%$'\t'*}" - local_file="/$i.pacnew" - package_file="$work_dir/$i" - - # Include unmodified .pacnew files. - local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" - if [[ $INCLUDE_PACNEW == 'n' ]] \ - || [[ ! -e $local_file ]] \ - || [[ $local_md5 != $original_md5 ]]; then - # Warn about modified files. - local_md5="$(md5sum "/$i" | cut -d' ' -f1)" - if [[ $local_md5 != $original_md5 ]]; then - warning "package file /$i has been modified" - fi - # Let the normal file be included in the %FILES% list. - continue - fi - ;; + # Assemble list of files which belong to the package and tar them + msg2 "Copying package files..." + while read i; do + if [[ -z $i ]]; then + continue + fi - *) + if [[ $i == %+([A-Z])% ]]; then + current=$i continue - ;; - esac + fi - ret=0 - bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + case "$current" in + %FILES%) + local_file="/$i" + package_file="$work_dir/$i" + + if [[ ! -e $local_file ]]; then + warning "package file $local_file is missing" + continue + fi + ;; + + %BACKUP%) + # Get the MD5 checksum. + original_md5="${i##*$'\t'}" + # Strip the md5sum after the tab. + i="${i%$'\t'*}" + local_file="/$i.pacnew" + package_file="$work_dir/$i" + + # Include unmodified .pacnew files. + local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" + if [[ $INCLUDE_PACNEW == 'n' ]] \ + || [[ ! -e $local_file ]] \ + || [[ $local_md5 != $original_md5 ]]; then + # Warn about modified files. + local_md5="$(md5sum "/$i" | cut -d' ' -f1)" + if [[ $local_md5 != $original_md5 ]]; then + warning "package file /$i has been modified" + fi + # Let the normal file be included in the %FILES% list. + continue + fi + ;; - # Workaround to bsdtar not reporting a missing file as an error - if ! [[ -e $package_file || -L $package_file ]]; then - error "unable to add $local_file to the package" - plain " If your user does not have permission to read this file, then" - plain " you will need to run $myname as root." + *) + continue + ;; + esac + + # Tar files + ret=0 + bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + # Workaround to bsdtar not reporting a missing file as an error + if ! [[ -e $package_file || -L $package_file ]]; then + error "unable to add $local_file to the package" + plain " If your user does not have permission to read this file, then" + plain " you will need to run $myname as root." + rm -rf "$work_dir" + exit 1 + fi + done < "$pkg_dir"/files + + ret=$? + if (( ret )); then rm -rf "$work_dir" exit 1 fi -done < "$pkg_dir"/files -ret=$? -if (( ret )); then - rm -rf "$work_dir" - exit 1 -fi + # Calculate package size + pkg_size=$(du -sk | awk '{print $1 * 1024}') -pkg_size=$(du -sk | awk '{print $1 * 1024}') - -# -# .PKGINFO stuff -# TODO adopt makepkg's write_pkginfo() into this or scripts/library -# -msg2 "Generating .PKGINFO metadata..." -echo "# Generated by $myname $myver" > .PKGINFO -if [[ $INFAKEROOT == "1" ]]; then - echo "# Using $(fakeroot -v)" >> .PKGINFO -fi -echo "# $(LC_ALL=C date)" >> .PKGINFO -echo "#" >> .PKGINFO - -while read i; do - if [[ -z $i ]]; then - continue; + # Reconstruct .PKGINFO from database + # TODO adopt makepkg's write_pkginfo() into this or scripts/library + msg2 "Generating .PKGINFO metadata..." + echo "# Generated by $myname $myver" > .PKGINFO + if [[ $INFAKEROOT == "1" ]]; then + echo "# Using $(fakeroot -v)" >> .PKGINFO fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + echo "# $(LC_ALL=C date)" >> .PKGINFO + echo "#" >> .PKGINFO + while read i; do + if [[ -z $i ]]; then + continue; + fi + if [[ $i == %+([A-Z])% ]]; then + current=$i + continue + fi + + case "$current" in + # desc + %NAME%) + echo "pkgname = $i" >> .PKGINFO + ;; + %VERSION%) + echo "pkgver = $i" >> .PKGINFO + ;; + %DESC%) + echo "pkgdesc = $i" >> .PKGINFO + ;; + %URL%) + echo "url = $i" >> .PKGINFO + ;; + %LICENSE%) + echo "license = $i" >> .PKGINFO + ;; + %ARCH%) + echo "arch = $i" >> .PKGINFO + pkg_arch="$i" + ;; + %BUILDDATE%) + echo "builddate = $(date -u "+%s")" >> .PKGINFO + ;; + %PACKAGER%) + echo "packager = $pkg_pkger" >> .PKGINFO + ;; + %SIZE%) + echo "size = $pkg_size" >> .PKGINFO + ;; + %GROUPS%) + echo "group = $i" >> .PKGINFO + ;; + %REPLACES%) + echo "replaces = $i" >> .PKGINFO + ;; + %DEPENDS%) + echo "depend = $i" >> .PKGINFO + ;; + %OPTDEPENDS%) + echo "optdepend = $i" >> .PKGINFO + ;; + %CONFLICTS%) + echo "conflict = $i" >> .PKGINFO + ;; + %PROVIDES%) + echo "provides = $i" >> .PKGINFO + ;; + %BACKUP%) + # Strip the md5sum after the tab + echo "backup = ${i%%$'\t'*}" >> .PKGINFO + ;; + esac + done < <(cat "$pkg_dir"/{desc,files}) + + comp_files=".PKGINFO" + + # Add instal file if present + if [[ -f $pkg_dir/install ]]; then + cp "$pkg_dir/install" "$work_dir/.INSTALL" + comp_files+=" .INSTALL" + fi + if [[ -f $pkg_dir/changelog ]]; then + cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" + comp_files+=" .CHANGELOG" fi - case "$current" in - # desc - %NAME%) - echo "pkgname = $i" >> .PKGINFO - ;; - %VERSION%) - echo "pkgver = $i" >> .PKGINFO - ;; - %DESC%) - echo "pkgdesc = $i" >> .PKGINFO - ;; - %URL%) - echo "url = $i" >> .PKGINFO - ;; - %LICENSE%) - echo "license = $i" >> .PKGINFO - ;; - %ARCH%) - echo "arch = $i" >> .PKGINFO - pkg_arch="$i" - ;; - %BUILDDATE%) - echo "builddate = $(date -u "+%s")" >> .PKGINFO - ;; - %PACKAGER%) - echo "packager = $pkg_pkger" >> .PKGINFO - ;; - %SIZE%) - echo "size = $pkg_size" >> .PKGINFO - ;; - %GROUPS%) - echo "group = $i" >> .PKGINFO - ;; - %REPLACES%) - echo "replaces = $i" >> .PKGINFO - ;; - %DEPENDS%) - echo "depend = $i" >> .PKGINFO - ;; - %OPTDEPENDS%) - echo "optdepend = $i" >> .PKGINFO - ;; - %CONFLICTS%) - echo "conflict = $i" >> .PKGINFO - ;; - %PROVIDES%) - echo "provides = $i" >> .PKGINFO - ;; - - # files - %BACKUP%) - # Strip the md5sum after the tab - echo "backup = ${i%%$'\t'*}" >> .PKGINFO - ;; - esac -done < <(cat "$pkg_dir"/{desc,files}) - -comp_files=".PKGINFO" - -if [[ -f $pkg_dir/install ]]; then - cp "$pkg_dir/install" "$work_dir/.INSTALL" - comp_files+=" .INSTALL" -fi -if [[ -f $pkg_dir/changelog ]]; then - cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" - comp_files+=" .CHANGELOG" -fi + # Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL + chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -# -# Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL -# -chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + # Generate the package + msg2 "Generating the package..." -# -# Generate the package -# -msg2 "Generating the package..." - -pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" -ret=0 - -# TODO: Maybe this can be set globally for robustness -shopt -s -o pipefail -bsdtar -cf - $comp_files * | -case "$PKGEXT" in - *tar.gz) gzip -c -f -n ;; - *tar.bz2) bzip2 -c -f ;; - *tar.xz) xz -c -z - ;; - *tar.Z) compress -c -f ;; - *tar) cat ;; - *) warning "'%s' is not a valid archive extension." \ - "$PKGEXT"; cat ;; -esac > "${pkg_file}"; ret=$? - -if (( ret )); then - error "Unable to write package to $pkg_dest" - plain " Maybe the disk is full or you do not have write access" + pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" + ret=0 + + # TODO: Maybe this can be set globally for robustness + shopt -s -o pipefail + bsdtar -cf - $comp_files * | + case "$PKGEXT" in + *tar.gz) gzip -c -f -n ;; + *tar.bz2) bzip2 -c -f ;; + *tar.xz) xz -c -z - ;; + *tar.Z) compress -c -f ;; + *tar) cat ;; + *) warning "'%s' is not a valid archive extension." \ + "$PKGEXT"; cat ;; + esac > "${pkg_file}"; ret=$? + + # Move compressed package to destination + if (( ret )); then + error "Unable to write package to $pkg_dest" + plain " Maybe the disk is full or you do not have write access" + rm -rf "$work_dir" + exit 1 + fi + + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi + msg "Done." +} -rm -rf "$work_dir" +# Run fakebuild in parralel with at maximum $MAX_JOBS jobs +# By default only run one job +MAX_JOBS=${MAX_JOBS:-1} +parallelize() { + while [[ $# -gt 0 ]]; do + job_count=($(jobs -p)) + if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then + fakebuild "$1" & + shift + fi + sleep 0.1 + done + wait +} -msg "Done." +# Initiate assembly function +if [[ $MAX_JOBS -gt "1" ]]; then + parallelize "$@" +else + for PKG in $@; do fakebuild $PKG; done +fi exit 0 -- 2.9.2
* Add option for printing fewer status updates * Add option for controlling the output directory * Add option for specyfying the the maximum number of jobs * Adjust output to the number of jobs * Rewrite usage page * Alter version information Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- * Streamline option syntax and version function with other scripts (mainly paccache) * Use the provided parseopts instead of getopt contrib/bacman.sh.in | 136 +++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 98 insertions(+), 38 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 6c9b1e5..ba9177e 100755 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -28,10 +28,12 @@ declare -r myname='bacman' declare -r myver='@PACKAGE_VERSION@' USE_COLOR='y' INCLUDE_PACNEW='n' +QUIET='n' # Required for fakeroot because options are shifted off the array. ARGS=("$@") m4_include(../scripts/library/output_format.sh) +m4_include(../scripts/library/parseopts.sh) # Lazy recursive clean up of temporary dirs work_dir_root="${TMPDIR:-/tmp}/bacman" @@ -40,57 +42,98 @@ clean_up() { echo exit } - # Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM # Print usage information usage() { - echo "${myname} (pacman) v${myver}" - echo - echo "Recreate a package using pacman's database and system files" - echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" - echo - echo "Example: ${myname} linux-headers" + cat <<-EOF + $myname (pacman) v$myver + This script was design to reassemble installed packages from its deliverd files. + It comes in handy if you have no access to an up-to-date package cache. + + Usage: $myname [options] <package(s)> + -h, --help display this help message and exit + -q, --quiet silence most of the status reporting + --nocolor turn off color in output + --pacnew package .pacnew files + -j, --jobs <jobs> build in parallel - you may want to set XZ_OPT + -o, --out <dir> write output to <dir> + + Examples: # $myname linux-headers + # $myname gzip munge binutils -o ~/Downloads + # $myname --nocolor --pacnew -o /tmp -j 5 gzip munge binutils + # $myname \$(pacman -Qsq) + EOF } # Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' + echo 'Copyright (C) 2016 Gordian Edenhofer <gordian.edenhofer@gmail.com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } -# Check for specified arguments -while [[ ! -z $1 ]]; do - if [[ $1 == "--nocolor" ]]; then - USE_COLOR='n' - shift - elif [[ $1 == "--pacnew" ]]; then - INCLUDE_PACNEW='y' - shift - else - break - fi -done - # Configure colored output m4_include(../scripts/library/term_colors.sh) # Break if no argument was given -if (( $# < 1 )); then +if (( $# == 0 )); then + usage + exit 1 +fi + +# Printing the usage information takes precedence over every other parameter +for option in "$@"; do + [[ $option == "-h" || $option == "--help" ]] && usage && exit 0 +done + +# Parse arguments +OPT_SHORT=':o:j:qv' +OPT_LONG=('out:' 'jobs:' 'quiet' 'nocolor' 'pacnew' 'version') +if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 fi +set -- "${OPTRET[@]}" +unset OPT_SHORT OPT_LONG OPTRET + +while :; do + case "$1" in + -o|--out) + pkg_dest="$(readlink -e $2)" + [[ $? -ne 0 ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 + shift ;; + -j|--jobs) + if [[ $2 =~ ^-?[0-9]+$ ]]; then + MAX_JOBS=$2 + else + echo -e "\e[39;1m$2\e[0m is not a valid integer!" + exit -1 + fi + shift ;; + -q|--quiet) + QUIET='y' ;; + --nocolor) + USE_COLOR='n' ;; + --pacnew) + INCLUDE_PACNEW='y' ;; + -v|--version) + version + exit 0 ;; + --) + shift + break 2 ;; + esac + shift +done -# Print usage or version if requested -if [[ $1 = -@(h|-help) ]]; then +# Retrieve the list of packages to be assembled and break if none was specified +pkg_list=($*) +if [[ ${#pkg_list[@]} == 0 ]]; then usage - exit 0 -elif [[ $1 = -@(V|-version) ]]; then - version - exit 0 + exit 1 fi # Run with fake root privileges if EUID is not root @@ -106,7 +149,7 @@ if (( EUID )); then fi fi -# Source environmental variables and specify fallbacks +# Read in environmental variables if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 @@ -121,8 +164,9 @@ source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi -pkg_dest="${PKGDEST:-$PWD}" -pkg_pkger=${PACKAGER:-'Unknown Packager'} +PKGDEST="${PKGDEST:-$PWD}" +pkg_dest="${pkg_dest:-$PKGDEST}" +pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database if [[ ! -d $pac_db ]]; then @@ -155,7 +199,12 @@ fakebuild() { cd "$work_dir" || exit 1 # Assemble list of files which belong to the package and tar them - msg2 "Copying package files..." + if [[ $MAX_JOBS -gt 1 && "$QUIET" != "y" ]]; then + msg2 "${pkg_name}: Copying package files..." + elif [[ "$QUIET" != "y" ]]; then + msg2 "Copying package files..." + fi + while read i; do if [[ -z $i ]]; then continue @@ -229,7 +278,11 @@ fakebuild() { # Reconstruct .PKGINFO from database # TODO adopt makepkg's write_pkginfo() into this or scripts/library - msg2 "Generating .PKGINFO metadata..." + if [[ $MAX_JOBS -gt 1 && "$QUIET" != "y" ]]; then + msg2 "${pkg_name}: Generating .PKGINFO metadata..." + elif [[ "$QUIET" != "y" ]]; then + msg2 "Generating .PKGINFO metadata..." + fi echo "# Generated by $myname $myver" > .PKGINFO if [[ $INFAKEROOT == "1" ]]; then echo "# Using $(fakeroot -v)" >> .PKGINFO @@ -317,11 +370,16 @@ fakebuild() { chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null # Generate the package - msg2 "Generating the package..." + if [[ $MAX_JOBS -gt 1 && "$QUIET" != "y" ]]; then + msg2 "${pkg_name}: Generating the package..." + elif [[ "$QUIET" != "y" ]]; then + msg2 "Generating the package..." + fi pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 + # Move compressed package to destination # TODO: Maybe this can be set globally for robustness shopt -s -o pipefail bsdtar -cf - $comp_files * | @@ -335,7 +393,7 @@ fakebuild() { "$PKGEXT"; cat ;; esac > "${pkg_file}"; ret=$? - # Move compressed package to destination + # Evaluate return code if (( ret )); then error "Unable to write package to $pkg_dest" plain " Maybe the disk is full or you do not have write access" @@ -345,7 +403,6 @@ fakebuild() { # Clean up working directory rm -rf "$work_dir" - msg "Done." } # Run fakebuild in parralel with at maximum $MAX_JOBS jobs @@ -365,11 +422,14 @@ parallelize() { # Initiate assembly function if [[ $MAX_JOBS -gt "1" ]]; then - parallelize "$@" + parallelize "${pkg_list[@]}" else - for PKG in $@; do fakebuild $PKG; done + for PKG in ${pkg_list[@]}; do + fakebuild $PKG + done fi +msg "Done." exit 0 # vim: set noet: -- 2.9.2
On 15/08/16 06:39, Gordian Edenhofer wrote:
* Add option for printing fewer status updates * Add option for controlling the output directory * Add option for specyfying the the maximum number of jobs * Adjust output to the number of jobs * Rewrite usage page * Alter version information
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- * Streamline option syntax and version function with other scripts (mainly paccache) * Use the provided parseopts instead of getopt
Multiple patches please. Each patch should contain a single change. E.g. (1) switch to parsepots, (2) rewrite usage, ... A
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- * Fix some typos * Adjust man page according to v2 of this patch series doc/.gitignore | 1 + doc/Makefile.am | 4 ++++ doc/bacman.8.txt | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+) create mode 100644 doc/bacman.8.txt diff --git a/doc/.gitignore b/doc/.gitignore index 2eae9e4..80d0026 100644 --- a/doc/.gitignore +++ b/doc/.gitignore @@ -7,6 +7,7 @@ makepkg.conf.5 pacman.8 pacman-key.8 pacman.conf.5 +bacman.8 pactree.8 pkgdelta.8 repo-add.8 diff --git a/doc/Makefile.am b/doc/Makefile.am index 5d7c975..88f23cc 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -13,6 +13,7 @@ ASCIIDOC_MANS = \ pkgdelta.8 \ pacman-key.8 \ pactree.8 \ + bacman.8 \ PKGBUILD.5 \ makepkg.conf.5 \ pacman.conf.5 \ @@ -30,6 +31,7 @@ HTML_MANPAGES = \ pkgdelta.8.html \ pacman-key.8.html \ pactree.8.html \ + bacman.8.html \ PKGBUILD.5.html \ makepkg.conf.5.html \ pacman.conf.5.html \ @@ -57,6 +59,7 @@ EXTRA_DIST = \ pkgdelta.8.txt \ pacman-key.8.txt \ pactree.8.txt \ + bacman.8.txt \ PKGBUILD.5.txt \ PKGBUILD-example.txt \ makepkg.conf.5.txt \ @@ -160,6 +163,7 @@ vercmp.8 vercmp.8.html: vercmp.8.txt pkgdelta.8 pkgdelta.8.html: pkgdelta.8.txt pacman-key.8 pacman-key.8.html: pacman-key.8.txt pactree.8 pactree.8.html: pactree.8.txt +bacman.8 bacman.8.html: bacman.8.txt PKGBUILD.5 PKGBUILD.5.html: PKGBUILD.5.txt PKGBUILD-example.txt makepkg.conf.5 makepkg.conf.5.html: makepkg.conf.5.txt pacman.conf.5 pacman.conf.5.html: pacman.conf.5.txt diff --git a/doc/bacman.8.txt b/doc/bacman.8.txt new file mode 100644 index 0000000..f66e9c2 --- /dev/null +++ b/doc/bacman.8.txt @@ -0,0 +1,71 @@ +///// +vim:set ts=4 sw=4 syntax=asciidoc noet spell spelllang=en_us: +///// +bacman(8) +========== + +Name +---- +bacman - reassemble installed packages + + +Synopsis +-------- +'bacman' [options] <package(s)> + + +Description +----------- +'bacman' was designed to reassemble installed packages from its deliverd files. +It comes in handy if there is no internet connection available and you have no +access to a up-to-date package cache. Otherwise packages from the Arch Linux +Archive should always be preferred. + +'bacman' fully honors linkman:makepkg.conf[8] and all compression environment +variables, notably ``XZ_OPT''. + + +Options +------- +*-h, \--help*:: + Display usage information. + +*-q, \--quiet*:: + Silence most of the status reporting. + +*\--nocolor*:: + Disable colored output. + +*\--pacnew*:: + Package .pacnew files instead of the concerning files currently in place. + +*-j, \--jobs <jobs>*:: + Assemble multiple packages in parallel. + +*-o, \--out <dir>*:: + Write the assembled package(s) to the specified directory. + + +Examples +-------- +'bacman' linux-headers + +Recreate the package ``linux-headers''. + +'bacman' gzip munge binutils -o ~/Downloads + +Assemble the packages gzip munge binutils and place the packages at +``~/Downloads''. + +$myname --nocolor --pacnew -o ~/backup -j 5 \$(pacman -Qsq) + +Assemble all currently installed packages using 5 concurrent jobs, use +``.pacnew'' whenever available, suppress colored output and place the desired +packages in ``~/backup''. + + +See Also +-------- +linkman:makepkg[8], linkman:pacman[8] + +include::footer.txt[] -- 2.9.2
On 15/08/16 06:39, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- * Fix some typos * Adjust man page according to v2 of this patch series
doc/.gitignore | 1 + doc/Makefile.am | 4 ++++ doc/bacman.8.txt | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+) create mode 100644 doc/bacman.8.txt
Contrib man pages now go to contrib/doc. Allan
On 15/08/16 06:39, Gordian Edenhofer wrote:
* move the actual assembly process into its own function * allow for packaging multiple packages with one command * handle SIGHUP SIGINT SIGTERM and remove working dirs accordingly * add some comments
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- * Clean up working directories with force on abort * Sleep for 100ms in between parallelization work
Why sleep? Two patches are needed: 1) allow bacman to take multiple package arguments 2) parallelize... The second probably would not be accepted... Allan
On Tue, 2016-08-30 at 20:30 +1000, Allan McRae wrote:
On 15/08/16 06:39, Gordian Edenhofer wrote:
* move the actual assembly process into its own function * allow for packaging multiple packages with one command * handle SIGHUP SIGINT SIGTERM and remove working dirs accordingly * add some comments
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- * Clean up working directories with force on abort * Sleep for 100ms in between parallelization work
Why sleep?
Sleep is invoked to reduce the CPU usage caused by busy-waiting. The approach for parallelization is straight forward and simple. It checks every n seconds whether enough jobs are running and in case not it starts a new one. Since this is not an elegant way to do it, I added a note which incorporates Dave Reisner hint to use wait -n at some point in the future when bash 4.3 is available at every system.
Two patches are needed:
1) allow bacman to take multiple package arguments 2) parallelize...
I will submit a second patch very soon which will address your complaints concerning my commit division.
The second probably would not be accepted...
I urge you to reconsider. Parallelization increases the speed of this tool immensely. To proof my point I made some test on an Intel NUC with an i5-4250U dual-core CPU. The results [1] clearly show that when using all available four threads the time needed to assemble various chunks of packages is sliced in half. The graphs show the time required for a group of packages to be recreated with bacman. To show the potential of parallelization I chose: base + base-devel, base + base-devel + gnome and finally I assembled all packages installed on the system. Packages contained in multiple groups were only assembled once. All chosen package groups were recreated using 1-6 jobs. The respective measured values and code can be seen here [2]. Best Regards, Gordian Edenhofer [1] http://edh.ddns.net/pacman_ml_bacman_benchmarks/bacman:%20simple%20 benchmark.svg [2] http://edh.ddns.net/pacman_ml_bacman_benchmarks/bacman:%20simple%20 benchmark.R.txt
On Wed, Aug 31, 2016 at 10:35:37PM +0200, Gordian Edenhofer wrote:
On Tue, 2016-08-30 at 20:30 +1000, Allan McRae wrote:
On 15/08/16 06:39, Gordian Edenhofer wrote:
* move the actual assembly process into its own function * allow for packaging multiple packages with one command * handle SIGHUP SIGINT SIGTERM and remove working dirs accordingly * add some comments
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- * Clean up working directories with force on abort * Sleep for 100ms in between parallelization work
Why sleep?
Sleep is invoked to reduce the CPU usage caused by busy-waiting. The approach for parallelization is straight forward and simple. It checks every n seconds whether enough jobs are running and in case not it starts a new one. Since this is not an elegant way to do it, I added a note which incorporates Dave Reisner hint to use wait -n at some point in the future when bash 4.3 is available at every system.
Two patches are needed:
1) allow bacman to take multiple package arguments 2) parallelize...
I will submit a second patch very soon which will address your complaints concerning my commit division.
The second probably would not be accepted...
I urge you to reconsider. Parallelization increases the speed of this
I don't think anyone is suggesting that packaging multiple things in parallel isn't useful. I already suggested that nothing needs to be implemented in bacman proper in order for you to parallelize the work. You can write your own "pbacman" as simply as: for arg; do bacman "$arg" & done; wait
tool immensely. To proof my point I made some test on an Intel NUC with an i5-4250U dual-core CPU. The results [1] clearly show that when using all available four threads the time needed to assemble various chunks of packages is sliced in half. The graphs show the time required for a group of packages to be recreated with bacman. To show the potential of parallelization I chose: base + base-devel, base + base-devel + gnome and finally I assembled all packages installed on the system. Packages contained in multiple groups were only assembled once. All chosen package groups were recreated using 1-6 jobs. The respective measured values and code can be seen here [2].
Best Regards, Gordian Edenhofer
[1] http://edh.ddns.net/pacman_ml_bacman_benchmarks/bacman:%20simple%20 benchmark.svg [2] http://edh.ddns.net/pacman_ml_bacman_benchmarks/bacman:%20simple%20 benchmark.R.txt
The second probably would not be accepted...
I urge you to reconsider. Parallelization increases the speed of this
I don't think anyone is suggesting that packaging multiple things in parallel isn't useful. I already suggested that nothing needs to be implemented in bacman proper in order for you to parallelize the work. You can write your own "pbacman" as simply as:
for arg; do bacman "$arg" & done; wait
There is a huge difference between flooding your system with ~1000 jobs and tightly controlling the maximum number. Adjusting the precise number of jobs enables you to organize your resources which itself is desirable. Even if there would be a perfect wrapper for bacman - which there is none - it would still make sense to implement easily understandable options into bacman for everyone to use simply to spare others the time of coming up with one. Best Regards, Gordian Edenhofer
On Wed, Aug 31, 2016 at 11:18:32PM +0200, Gordian Edenhofer wrote:
The second probably would not be accepted...
I urge you to reconsider. Parallelization increases the speed of this
I don't think anyone is suggesting that packaging multiple things in parallel isn't useful. I already suggested that nothing needs to be implemented in bacman proper in order for you to parallelize the work. You can write your own "pbacman" as simply as:
for arg; do bacman "$arg" & done; wait
There is a huge difference between flooding your system with ~1000 jobs and tightly controlling the maximum number. Adjusting the precise number of jobs enables you to organize your resources which itself is desirable.
Then use a program like 'parallel' which has this sort of knob. I really wonder what it is you're doing that requires running bacman with a large number of packages with any regularity.
Even if there would be a perfect wrapper for bacman - which there is none - it would still make sense to implement easily understandable options into bacman for everyone to use simply to spare others the time of coming up with one.
Best Regards, Gordian Edenhofer
On 01/09/16 08:08, Dave Reisner wrote:
On Wed, Aug 31, 2016 at 11:18:32PM +0200, Gordian Edenhofer wrote:
The second probably would not be accepted...
I urge you to reconsider. Parallelization increases the speed of this
I don't think anyone is suggesting that packaging multiple things in parallel isn't useful. I already suggested that nothing needs to be implemented in bacman proper in order for you to parallelize the work. You can write your own "pbacman" as simply as:
for arg; do bacman "$arg" & done; wait
There is a huge difference between flooding your system with ~1000 jobs and tightly controlling the maximum number. Adjusting the precise number of jobs enables you to organize your resources which itself is desirable.
Then use a program like 'parallel' which has this sort of knob. I really wonder what it is you're doing that requires running bacman with a large number of packages with any regularity.
Gathering the files etc takes no time. It is really the compression that is being made parallel. If only there was a way to set compression to use mutlithreading... As I already stated, I'm not looking to accept the parallel patch. A
On Thu, 2016-09-01 at 08:28 +1000, Allan McRae wrote:
On 01/09/16 08:08, Dave Reisner wrote:
On Wed, Aug 31, 2016 at 11:18:32PM +0200, Gordian Edenhofer wrote:
The second probably would not be accepted...
I urge you to reconsider. Parallelization increases the speed of this
I don't think anyone is suggesting that packaging multiple things in parallel isn't useful. I already suggested that nothing needs to be implemented in bacman proper in order for you to parallelize the work. You can write your own "pbacman" as simply as:
for arg; do bacman "$arg" & done; wait
There is a huge difference between flooding your system with ~1000 jobs and tightly controlling the maximum number. Adjusting the precise number of jobs enables you to organize your resources which itself is desirable.
Then use a program like 'parallel' which has this sort of knob. I really wonder what it is you're doing that requires running bacman with a large number of packages with any regularity.
Gathering the files etc takes no time. It is really the compression that is being made parallel. If only there was a way to set compression to use mutlithreading...
The actual compression using xz (default) is not necessary the most time intensive part. The linux-headers package for example is compressed within a few seconds but the whole process before xz is run takes way longer. This can be seen with top as an illustration or simply by running bacman one time without compression and the other with. Moreover using bacman to parallelize makes it completely independent from the archive format used and still brings gains when recreating multiple packages. At the very least it would fill the gap in between the compression of multiple packages. Therefore it would be beneficial even if compression would take the longest which is doesn't always do. Best Regards, Gordian Edenhofer
On 01/09/16 09:44, Gordian Edenhofer wrote:
On Thu, 2016-09-01 at 08:28 +1000, Allan McRae wrote:
On 01/09/16 08:08, Dave Reisner wrote:
On Wed, Aug 31, 2016 at 11:18:32PM +0200, Gordian Edenhofer wrote:
> > The second probably would not be accepted...
I urge you to reconsider. Parallelization increases the speed of this
I don't think anyone is suggesting that packaging multiple things in parallel isn't useful. I already suggested that nothing needs to be implemented in bacman proper in order for you to parallelize the work. You can write your own "pbacman" as simply as:
for arg; do bacman "$arg" & done; wait
There is a huge difference between flooding your system with ~1000 jobs and tightly controlling the maximum number. Adjusting the precise number of jobs enables you to organize your resources which itself is desirable.
Then use a program like 'parallel' which has this sort of knob. I really wonder what it is you're doing that requires running bacman with a large number of packages with any regularity.
Gathering the files etc takes no time. It is really the compression that is being made parallel. If only there was a way to set compression to use mutlithreading...
The actual compression using xz (default) is not necessary the most time intensive part. The linux-headers package for example is compressed within a few seconds but the whole process before xz is run takes way longer. This can be seen with top as an illustration or simply by running bacman one time without compression and the other with. Moreover using bacman to parallelize makes it completely independent from the archive format used and still brings gains when recreating multiple packages. At the very least it would fill the gap in between the compression of multiple packages. Therefore it would be beneficial even if compression would take the longest which is doesn't always do.
So read speed is the slow part? And trying to read more files at the same time helps? Allan
On Thu, 2016-09-01 at 13:14 +1000, Allan McRae wrote:
On 01/09/16 09:44, Gordian Edenhofer wrote:
On Thu, 2016-09-01 at 08:28 +1000, Allan McRae wrote:
On 01/09/16 08:08, Dave Reisner wrote:
On Wed, Aug 31, 2016 at 11:18:32PM +0200, Gordian Edenhofer wrote:
> > > > > > > > The second probably would not be accepted... > > I urge you to reconsider. Parallelization increases the > speed > of > this
I don't think anyone is suggesting that packaging multiple things in parallel isn't useful. I already suggested that nothing needs to be implemented in bacman proper in order for you to parallelize the work. You can write your own "pbacman" as simply as:
for arg; do bacman "$arg" & done; wait
There is a huge difference between flooding your system with ~1000 jobs and tightly controlling the maximum number. Adjusting the precise number of jobs enables you to organize your resources which itself is desirable.
Then use a program like 'parallel' which has this sort of knob. I really wonder what it is you're doing that requires running bacman with a large number of packages with any regularity.
Gathering the files etc takes no time. It is really the compression that is being made parallel. If only there was a way to set compression to use mutlithreading...
The actual compression using xz (default) is not necessary the most time intensive part. The linux-headers package for example is compressed within a few seconds but the whole process before xz is run takes way longer. This can be seen with top as an illustration or simply by running bacman one time without compression and the other with. Moreover using bacman to parallelize makes it completely independent from the archive format used and still brings gains when recreating multiple packages. At the very least it would fill the gap in between the compression of multiple packages. Therefore it would be beneficial even if compression would take the longest which is doesn't always do.
So read speed is the slow part? And trying to read more files at the same time helps?
Obviously read speed is not the limitation here. In this case bacman would not speed up by increasing the job count - no matter the implementation - but it obviously does. To have a good comparison I ran the tests again with xz set to use multiple threads. The results can be seen here [1] and the code is available here [2]. Surely tuning xz helps especially for single packages but using multiple jobs bring the real speed boost when recreating more than one package. The fact that xz can be tuned as well is no secret and was stated in the man page and mentioned in the usage section from the beginning on. Furthermore the implementation is only a few additional lines of code, must be explicitly invoked and should in no case slow someone down. Best Regards, Gordian Edenhofer [1] http://edh.ddns.net/pacman_ml_bacman_benchmarks/bacman:%20simple%20 benchmark.svg [2] http://edh.ddns.net/pacman_ml_bacman_benchmarks/bacman:%20simple%20 benchmark.R.txt
On Wed, 2016-08-31 at 18:08 -0400, Dave Reisner wrote:
On Wed, Aug 31, 2016 at 11:18:32PM +0200, Gordian Edenhofer wrote:
The second probably would not be accepted...
I urge you to reconsider. Parallelization increases the speed of this
I don't think anyone is suggesting that packaging multiple things in parallel isn't useful. I already suggested that nothing needs to be implemented in bacman proper in order for you to parallelize the work. You can write your own "pbacman" as simply as:
for arg; do bacman "$arg" & done; wait
There is a huge difference between flooding your system with ~1000 jobs and tightly controlling the maximum number. Adjusting the precise number of jobs enables you to organize your resources which itself is desirable.
Then use a program like 'parallel' which has this sort of knob. I really wonder what it is you're doing that requires running bacman with a large number of packages with any regularity.
Sure, parallel is an awesome program. However I already explained why I chose to write this patch and added more functionality to bacman: Save time for other users. The added code is modest in size and enables anyone to significantly speed up their bacman process by having a look at the man page and without downloading an additional ~5000 line perl program. Please also consider that this patch is about more than just parallelization: It adds a manual page, some useful command line options and let bacman handle abort signals.
Even if there would be a perfect wrapper for bacman - which there is none - it would still make sense to implement easily understandable options into bacman for everyone to use simply to spare others the time of coming up with one.
To enable the creation of multiple packages with one command move the assembly process into its own function. Handle SIGHUP, SIGINT, SIGTERM and remove working directories accordingly. Add some comments. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 442 +++++++++++++++++++++++++-------------------------- 1 file changed, 220 insertions(+), 222 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a611c1a..a13f074 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -33,9 +33,18 @@ ARGS=("$@") m4_include(../scripts/library/output_format.sh) -# -# User Friendliness -# +# Lazy recursive clean up of temporary dirs +work_dir_root="${TMPDIR:-/tmp}/bacman" +clean_up() { + rm -rf "$work_dir_root".* + echo + exit +} + +# Trap termination signals +trap clean_up SIGHUP SIGINT SIGTERM + +# Print usage information usage() { echo "${myname} (pacman) v${myver}" echo @@ -46,12 +55,14 @@ usage() { echo "Example: ${myname} linux-headers" } +# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } +# Check for specified arguments while [[ ! -z $1 ]]; do if [[ $1 == "--nocolor" ]]; then USE_COLOR='n' @@ -64,13 +75,16 @@ while [[ ! -z $1 ]]; do fi done +# Configure colored output m4_include(../scripts/library/term_colors.sh) -if (( $# != 1 )); then +# Break if no argument was given +if (( $# < 1 )); then usage exit 1 fi +# Print usage or version if requested if [[ $1 = -@(h|-help) ]]; then usage exit 0 @@ -79,9 +93,7 @@ elif [[ $1 = -@(V|-version) ]]; then exit 0 fi -# -# Fakeroot support -# +# Run with fake root privileges if EUID is not root if (( EUID )); then if [[ -f /usr/bin/fakeroot ]]; then msg "Entering fakeroot environment" @@ -94,264 +106,250 @@ if (( EUID )); then fi fi -# -# Setting environmental variables -# +# Source environmental variables and specify fallbacks if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 fi - eval $(awk '/DBPath/ {print $1$2$3}' @sysconfdir@/pacman.conf) pac_db="${DBPath:-@localstatedir@/lib/pacman/}/local" - if [[ ! -r @sysconfdir@/makepkg.conf ]]; then error "unable to read @sysconfdir@/makepkg.conf" exit 1 fi - source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi - pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -pkg_name="$1" -pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) -pkg_namver=("${pkg_dir[@]##*/}") - -# -# Checks everything is in place -# +# Check for an existing database if [[ ! -d $pac_db ]]; then error "pacman database directory ${pac_db} not found" exit 1 fi -if (( ${#pkg_dir[@]} != 1 )); then - error "%d entries for package %s found in pacman database" \ - ${#pkg_dir[@]} "${pkg_name}" - msg2 "%s" "${pkg_dir[@]}" - exit 1 -fi - -if [[ ! -d $pkg_dir ]]; then - error "package %s is found in pacman database," "${pkg_name}" - plain " but '%s' is not a directory" "${pkg_dir}" - exit 1 -fi - -# -# Begin -# -msg "Package: ${pkg_namver}" -work_dir=$(mktemp -d "${TMPDIR:-/tmp}/bacman.XXXXXXXXXX") -cd "$work_dir" || exit 1 - -# -# File copying -# -msg2 "Copying package files..." - -while read i; do - if [[ -z $i ]]; then - continue +# Assemble a single package: $1 = pkgname +fakebuild() { + pkg_name="$1" + pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) + pkg_namver=("${pkg_dir[@]##*/}") + + # Checks database for specified package + if (( ${#pkg_dir[@]} != 1 )); then + error "%d entries for package %s found in pacman database" \ + ${#pkg_dir[@]} "${pkg_name}" + msg2 "%s" "${pkg_dir[@]}" + exit 1 fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + if [[ ! -d $pkg_dir ]]; then + error "package %s is found in pacman database," "${pkg_name}" + plain " but '%s' is not a directory" "${pkg_dir}" + exit 1 fi - case "$current" in - %FILES%) - local_file="/$i" - package_file="$work_dir/$i" + # Create working directory + msg "Package: ${pkg_namver}" + work_dir=$(mktemp -d "${work_dir_root}.XXXXXXXXXX") + cd "$work_dir" || exit 1 - if [[ ! -e $local_file ]]; then - warning "package file $local_file is missing" - continue - fi - ;; - - %BACKUP%) - # Get the MD5 checksum. - original_md5="${i##*$'\t'}" - # Strip the md5sum after the tab. - i="${i%$'\t'*}" - local_file="/$i.pacnew" - package_file="$work_dir/$i" - - # Include unmodified .pacnew files. - local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" - if [[ $INCLUDE_PACNEW == 'n' ]] \ - || [[ ! -e $local_file ]] \ - || [[ $local_md5 != $original_md5 ]]; then - # Warn about modified files. - local_md5="$(md5sum "/$i" | cut -d' ' -f1)" - if [[ $local_md5 != $original_md5 ]]; then - warning "package file /$i has been modified" - fi - # Let the normal file be included in the %FILES% list. - continue - fi - ;; + # Assemble list of files which belong to the package and tar them + msg2 "Copying package files..." + while read i; do + if [[ -z $i ]]; then + continue + fi - *) + if [[ $i == %+([A-Z])% ]]; then + current=$i continue - ;; - esac + fi - ret=0 - bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + case "$current" in + %FILES%) + local_file="/$i" + package_file="$work_dir/$i" - # Workaround to bsdtar not reporting a missing file as an error - if ! [[ -e $package_file || -L $package_file ]]; then - error "unable to add $local_file to the package" - plain " If your user does not have permission to read this file, then" - plain " you will need to run $myname as root." + if [[ ! -e $local_file ]]; then + warning "package file $local_file is missing" + continue + fi + ;; + + %BACKUP%) + # Get the MD5 checksum. + original_md5="${i##*$'\t'}" + # Strip the md5sum after the tab. + i="${i%$'\t'*}" + local_file="/$i.pacnew" + package_file="$work_dir/$i" + + # Include unmodified .pacnew files. + local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" + if [[ $INCLUDE_PACNEW == 'n' ]] \ + || [[ ! -e $local_file ]] \ + || [[ $local_md5 != $original_md5 ]]; then + # Warn about modified files. + local_md5="$(md5sum "/$i" | cut -d' ' -f1)" + if [[ $local_md5 != $original_md5 ]]; then + warning "package file /$i has been modified" + fi + # Let the normal file be included in the %FILES% list. + continue + fi + ;; + + *) + continue + ;; + esac + + # Tar files + ret=0 + bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + # Workaround to bsdtar not reporting a missing file as an error + if ! [[ -e $package_file || -L $package_file ]]; then + error "unable to add $local_file to the package" + plain " If your user does not have permission to read this file, then" + plain " you will need to run $myname as root." + rm -rf "$work_dir" + exit 1 + fi + done < "$pkg_dir"/files + + ret=$? + if (( ret )); then rm -rf "$work_dir" exit 1 fi -done < "$pkg_dir"/files -ret=$? -if (( ret )); then - rm -rf "$work_dir" - exit 1 -fi - -pkg_size=$(du -sk | awk '{print $1 * 1024}') - -# -# .PKGINFO stuff -# TODO adopt makepkg's write_pkginfo() into this or scripts/library -# -msg2 "Generating .PKGINFO metadata..." -echo "# Generated by $myname $myver" > .PKGINFO -if [[ $INFAKEROOT == "1" ]]; then - echo "# Using $(fakeroot -v)" >> .PKGINFO -fi -echo "# $(LC_ALL=C date)" >> .PKGINFO -echo "#" >> .PKGINFO + # Calculate package size + pkg_size=$(du -sk | awk '{print $1 * 1024}') -while read i; do - if [[ -z $i ]]; then - continue; + # Reconstruct .PKGINFO from database + # TODO adopt makepkg's write_pkginfo() into this or scripts/library + msg2 "Generating .PKGINFO metadata..." + echo "# Generated by $myname $myver" > .PKGINFO + if [[ $INFAKEROOT == "1" ]]; then + echo "# Using $(fakeroot -v)" >> .PKGINFO fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + echo "# $(LC_ALL=C date)" >> .PKGINFO + echo "#" >> .PKGINFO + while read i; do + if [[ -z $i ]]; then + continue; + fi + if [[ $i == %+([A-Z])% ]]; then + current=$i + continue + fi + + case "$current" in + # desc + %NAME%) + echo "pkgname = $i" >> .PKGINFO + ;; + %VERSION%) + echo "pkgver = $i" >> .PKGINFO + ;; + %DESC%) + echo "pkgdesc = $i" >> .PKGINFO + ;; + %URL%) + echo "url = $i" >> .PKGINFO + ;; + %LICENSE%) + echo "license = $i" >> .PKGINFO + ;; + %ARCH%) + echo "arch = $i" >> .PKGINFO + pkg_arch="$i" + ;; + %BUILDDATE%) + echo "builddate = $(date -u "+%s")" >> .PKGINFO + ;; + %PACKAGER%) + echo "packager = $pkg_pkger" >> .PKGINFO + ;; + %SIZE%) + echo "size = $pkg_size" >> .PKGINFO + ;; + %GROUPS%) + echo "group = $i" >> .PKGINFO + ;; + %REPLACES%) + echo "replaces = $i" >> .PKGINFO + ;; + %DEPENDS%) + echo "depend = $i" >> .PKGINFO + ;; + %OPTDEPENDS%) + echo "optdepend = $i" >> .PKGINFO + ;; + %CONFLICTS%) + echo "conflict = $i" >> .PKGINFO + ;; + %PROVIDES%) + echo "provides = $i" >> .PKGINFO + ;; + %BACKUP%) + # Strip the md5sum after the tab + echo "backup = ${i%%$'\t'*}" >> .PKGINFO + ;; + esac + done < <(cat "$pkg_dir"/{desc,files}) + + comp_files=".PKGINFO" + + # Add instal file if present + if [[ -f $pkg_dir/install ]]; then + cp "$pkg_dir/install" "$work_dir/.INSTALL" + comp_files+=" .INSTALL" + fi + if [[ -f $pkg_dir/changelog ]]; then + cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" + comp_files+=" .CHANGELOG" fi - case "$current" in - # desc - %NAME%) - echo "pkgname = $i" >> .PKGINFO - ;; - %VERSION%) - echo "pkgver = $i" >> .PKGINFO - ;; - %DESC%) - echo "pkgdesc = $i" >> .PKGINFO - ;; - %URL%) - echo "url = $i" >> .PKGINFO - ;; - %LICENSE%) - echo "license = $i" >> .PKGINFO - ;; - %ARCH%) - echo "arch = $i" >> .PKGINFO - pkg_arch="$i" - ;; - %BUILDDATE%) - echo "builddate = $(date -u "+%s")" >> .PKGINFO - ;; - %PACKAGER%) - echo "packager = $pkg_pkger" >> .PKGINFO - ;; - %SIZE%) - echo "size = $pkg_size" >> .PKGINFO - ;; - %GROUPS%) - echo "group = $i" >> .PKGINFO - ;; - %REPLACES%) - echo "replaces = $i" >> .PKGINFO - ;; - %DEPENDS%) - echo "depend = $i" >> .PKGINFO - ;; - %OPTDEPENDS%) - echo "optdepend = $i" >> .PKGINFO - ;; - %CONFLICTS%) - echo "conflict = $i" >> .PKGINFO - ;; - %PROVIDES%) - echo "provides = $i" >> .PKGINFO - ;; - - # files - %BACKUP%) - # Strip the md5sum after the tab - echo "backup = ${i%%$'\t'*}" >> .PKGINFO - ;; - esac -done < <(cat "$pkg_dir"/{desc,files}) - -comp_files=".PKGINFO" - -if [[ -f $pkg_dir/install ]]; then - cp "$pkg_dir/install" "$work_dir/.INSTALL" - comp_files+=" .INSTALL" -fi -if [[ -f $pkg_dir/changelog ]]; then - cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" - comp_files+=" .CHANGELOG" -fi + # Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL + chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -# -# Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL -# -chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + # Generate the package + msg2 "Generating the package..." -# -# Generate the package -# -msg2 "Generating the package..." - -pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" -ret=0 - -# TODO: Maybe this can be set globally for robustness -shopt -s -o pipefail -bsdtar -cf - $comp_files * | -case "$PKGEXT" in - *tar.gz) gzip -c -f -n ;; - *tar.bz2) bzip2 -c -f ;; - *tar.xz) xz -c -z - ;; - *tar.Z) compress -c -f ;; - *tar) cat ;; - *) warning "'%s' is not a valid archive extension." \ - "$PKGEXT"; cat ;; -esac > "${pkg_file}"; ret=$? - -if (( ret )); then - error "Unable to write package to $pkg_dest" - plain " Maybe the disk is full or you do not have write access" + pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" + ret=0 + + # TODO: Maybe this can be set globally for robustness + shopt -s -o pipefail + bsdtar -cf - $comp_files * | + case "$PKGEXT" in + *tar.gz) gzip -c -f -n ;; + *tar.bz2) bzip2 -c -f ;; + *tar.xz) xz -c -z - ;; + *tar.Z) compress -c -f ;; + *tar) cat ;; + *) warning "'%s' is not a valid archive extension." \ + "$PKGEXT"; cat ;; + esac > "${pkg_file}"; ret=$? + + # Move compressed package to destination + if (( ret )); then + error "Unable to write package to $pkg_dest" + plain " Maybe the disk is full or you do not have write access" + rm -rf "$work_dir" + exit 1 + fi + + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi + msg "Done." +} -rm -rf "$work_dir" -msg "Done." +for PKG in $@; do fakebuild $PKG; done exit 0 -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a13f074..1241718 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -348,8 +348,29 @@ fakebuild() { msg "Done." } +# Run fakebuild in parralel with at maximum $MAX_JOBS jobs +# By default only run one job +MAX_JOBS=${MAX_JOBS:-1} +# TODO: Use wait -n instead of this hack as soon as it is ok +# to bump the version of the bash dependency +parallelize() { + while [[ $# -gt 0 ]]; do + job_count=($(jobs -p)) + if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then + fakebuild "$1" & + shift + fi + sleep 0.1 + done + wait +} -for PKG in $@; do fakebuild $PKG; done +# Initiate assembly function +if [[ $MAX_JOBS -gt "1" ]]; then + parallelize "$@" +else + for PKG in $@; do fakebuild $PKG; done +fi exit 0 -- 2.9.3
Switch to parseopts instead of merely checking the first argument. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 63 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 22 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 1241718..a0bfcf1 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -32,6 +32,7 @@ INCLUDE_PACNEW='n' ARGS=("$@") m4_include(../scripts/library/output_format.sh) +m4_include(../scripts/library/parseopts.sh) # Lazy recursive clean up of temporary dirs work_dir_root="${TMPDIR:-/tmp}/bacman" @@ -62,35 +63,51 @@ version() { echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } -# Check for specified arguments -while [[ ! -z $1 ]]; do - if [[ $1 == "--nocolor" ]]; then - USE_COLOR='n' - shift - elif [[ $1 == "--pacnew" ]]; then - INCLUDE_PACNEW='y' - shift - else - break - fi -done - # Configure colored output m4_include(../scripts/library/term_colors.sh) # Break if no argument was given -if (( $# < 1 )); then +if (( $# == 0 )); then + usage + exit 1 +fi + +# Printing the usage information takes precedence over every other parameter +for option in "$@"; do + [[ $option == "-h" || $option == "--help" ]] && usage && exit 0 +done + +# Parse arguments +OPT_SHORT=':v' +OPT_LONG=('nocolor' 'pacnew' 'version') +if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 fi +set -- "${OPTRET[@]}" +unset OPT_SHORT OPT_LONG OPTRET + +while :; do + case "$1" in + --nocolor) + USE_COLOR='n' ;; + --pacnew) + INCLUDE_PACNEW='y' ;; + -v|--version) + version + exit 0 ;; + --) + shift + break 2 ;; + esac + shift +done -# Print usage or version if requested -if [[ $1 = -@(h|-help) ]]; then +# Retrieve the list of packages to be assembled and break if none was specified +pkg_list=($*) +if [[ ${#pkg_list[@]} == 0 ]]; then usage - exit 0 -elif [[ $1 = -@(V|-version) ]]; then - version - exit 0 + exit 1 fi # Run with fake root privileges if EUID is not root @@ -367,9 +384,11 @@ parallelize() { # Initiate assembly function if [[ $MAX_JOBS -gt "1" ]]; then - parallelize "$@" + parallelize "${pkg_list[@]}" else - for PKG in $@; do fakebuild $PKG; done + for PKG in ${pkg_list[@]}; do + fakebuild $PKG + done fi exit 0 -- 2.9.3
On 01/09/16 07:05, Gordian Edenhofer wrote:
Switch to parseopts instead of merely checking the first argument.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 63 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 22 deletions(-)
<snip>
+# Parse arguments +OPT_SHORT=':v'
Why the colon?
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a0bfcf1..ca2945f 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -41,25 +41,33 @@ clean_up() { echo exit } - # Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM # Print usage information usage() { - echo "${myname} (pacman) v${myver}" - echo - echo "Recreate a package using pacman's database and system files" - echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" - echo - echo "Example: ${myname} linux-headers" + cat <<-EOF + $myname (pacman) v$myver + This script was design to reassemble installed packages from its deliverd files. + It comes in handy if you have no access to an up-to-date package cache. + + Usage: $myname [options] <package(s)> + -h, --help display this help message and exit + --nocolor turn off color in output + --pacnew package .pacnew files + + Examples: # $myname linux-headers + # $myname gzip munge binutils -o ~/Downloads + # $myname --nocolor --pacnew gzip munge binutils + # $myname \$(pacman -Qsq) + EOF } # Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' + echo 'Copyright (C) 2016 Gordian Edenhofer <gordian.edenhofer@gmail.com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } @@ -123,7 +131,7 @@ if (( EUID )); then fi fi -# Source environmental variables and specify fallbacks +# Read in environmental variables if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 @@ -139,7 +147,7 @@ if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi pkg_dest="${PKGDEST:-$PWD}" -pkg_pkger=${PACKAGER:-'Unknown Packager'} +pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database if [[ ! -d $pac_db ]]; then @@ -339,6 +347,7 @@ fakebuild() { pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 + # Move compressed package to destination # TODO: Maybe this can be set globally for robustness shopt -s -o pipefail bsdtar -cf - $comp_files * | @@ -352,7 +361,7 @@ fakebuild() { "$PKGEXT"; cat ;; esac > "${pkg_file}"; ret=$? - # Move compressed package to destination + # Evaluate return code if (( ret )); then error "Unable to write package to $pkg_dest" plain " Maybe the disk is full or you do not have write access" @@ -362,7 +371,6 @@ fakebuild() { # Clean up working directory rm -rf "$work_dir" - msg "Done." } # Run fakebuild in parralel with at maximum $MAX_JOBS jobs @@ -391,6 +399,7 @@ else done fi +msg "Done." exit 0 # vim: set noet: -- 2.9.3
On 01/09/16 07:05, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-)
The format for the header is the same for all scripts in the pacman codebase. Do not change the format. You may add examples onto the end.
diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a0bfcf1..ca2945f 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -41,25 +41,33 @@ clean_up() { echo exit } - # Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM
# Print usage information usage() { - echo "${myname} (pacman) v${myver}" - echo - echo "Recreate a package using pacman's database and system files" - echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" - echo - echo "Example: ${myname} linux-headers" + cat <<-EOF + $myname (pacman) v$myver + This script was design to reassemble installed packages from its deliverd files.
Typo
+ It comes in handy if you have no access to an up-to-date package cache. + + Usage: $myname [options] <package(s)> + -h, --help display this help message and exit + --nocolor turn off color in output + --pacnew package .pacnew files + + Examples: # $myname linux-headers + # $myname gzip munge binutils -o ~/Downloads + # $myname --nocolor --pacnew gzip munge binutils + # $myname \$(pacman -Qsq) + EOF }
# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' + echo 'Copyright (C) 2016 Gordian Edenhofer <gordian.edenhofer@gmail.com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' }
@@ -123,7 +131,7 @@ if (( EUID )); then fi fi
-# Source environmental variables and specify fallbacks +# Read in environmental variables if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 @@ -139,7 +147,7 @@ if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi pkg_dest="${PKGDEST:-$PWD}" -pkg_pkger=${PACKAGER:-'Unknown Packager'} +pkg_pkger="${PACKAGER:-'Unknown Packager'}"
# Check for an existing database if [[ ! -d $pac_db ]]; then @@ -339,6 +347,7 @@ fakebuild() { pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0
+ # Move compressed package to destination # TODO: Maybe this can be set globally for robustness shopt -s -o pipefail bsdtar -cf - $comp_files * | @@ -352,7 +361,7 @@ fakebuild() { "$PKGEXT"; cat ;; esac > "${pkg_file}"; ret=$?
- # Move compressed package to destination + # Evaluate return code if (( ret )); then error "Unable to write package to $pkg_dest" plain " Maybe the disk is full or you do not have write access" @@ -362,7 +371,6 @@ fakebuild() {
# Clean up working directory rm -rf "$work_dir" - msg "Done." }
# Run fakebuild in parralel with at maximum $MAX_JOBS jobs @@ -391,6 +399,7 @@ else done fi
+msg "Done." exit 0
# vim: set noet:
On 01/09/16 13:59, Allan McRae wrote:
On 01/09/16 07:05, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-)
The format for the header is the same for all scripts in the pacman codebase. Do not change the format. You may add examples onto the end.
diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a0bfcf1..ca2945f 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -41,25 +41,33 @@ clean_up() { echo exit } - # Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM
# Print usage information usage() { - echo "${myname} (pacman) v${myver}" - echo - echo "Recreate a package using pacman's database and system files" - echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" - echo - echo "Example: ${myname} linux-headers" + cat <<-EOF + $myname (pacman) v$myver + This script was design to reassemble installed packages from its deliverd files.
Typo
+ It comes in handy if you have no access to an up-to-date package cache. + + Usage: $myname [options] <package(s)> + -h, --help display this help message and exit + --nocolor turn off color in output + --pacnew package .pacnew files + + Examples: # $myname linux-headers + # $myname gzip munge binutils -o ~/Downloads
Also, the -o option does not exist at this point.
On Thu, 2016-09-01 at 13:59 +1000, Allan McRae wrote:
On 01/09/16 07:05, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-)
The format for the header is the same for all scripts in the pacman codebase. Do not change the format. You may add examples onto the end.
The currently used format is consistent with the one used by paccache. However I have no problem reverting back to the old format, just say the word.
On 01/09/16 23:36, Gordian Edenhofer wrote:
On Thu, 2016-09-01 at 13:59 +1000, Allan McRae wrote:
On 01/09/16 07:05, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-)
The format for the header is the same for all scripts in the pacman codebase. Do not change the format. You may add examples onto the end.
The currently used format is consistent with the one used by paccache. However I have no problem reverting back to the old format, just say the word.
The format of the official tools should be followed. makepkg, pacman-key, repo-add, pacman-db-upgrade, etc. Allan
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index ca2945f..322bb66 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -28,6 +28,7 @@ declare -r myname='bacman' declare -r myver='@PACKAGE_VERSION@' USE_COLOR='y' INCLUDE_PACNEW='n' +QUIET='n' # Required for fakeroot because options are shifted off the array. ARGS=("$@") @@ -53,6 +54,7 @@ usage() { Usage: $myname [options] <package(s)> -h, --help display this help message and exit + -q, --quiet silence most of the status reporting --nocolor turn off color in output --pacnew package .pacnew files @@ -86,8 +88,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT=':v' -OPT_LONG=('nocolor' 'pacnew' 'version') +OPT_SHORT=':qv' +OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -97,6 +99,8 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -q|--quiet) + QUIET='y' ;; --nocolor) USE_COLOR='n' ;; --pacnew) @@ -180,7 +184,10 @@ fakebuild() { cd "$work_dir" || exit 1 # Assemble list of files which belong to the package and tar them - msg2 "Copying package files..." + if [[ "$QUIET" != "y" ]]; then + msg2 "Copying package files..." + fi + while read i; do if [[ -z $i ]]; then continue @@ -254,7 +261,9 @@ fakebuild() { # Reconstruct .PKGINFO from database # TODO adopt makepkg's write_pkginfo() into this or scripts/library - msg2 "Generating .PKGINFO metadata..." + if [[ "$QUIET" != "y" ]]; then + msg2 "Generating .PKGINFO metadata..." + fi echo "# Generated by $myname $myver" > .PKGINFO if [[ $INFAKEROOT == "1" ]]; then echo "# Using $(fakeroot -v)" >> .PKGINFO @@ -342,7 +351,9 @@ fakebuild() { chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null # Generate the package - msg2 "Generating the package..." + if [[ "$QUIET" != "y" ]]; then + msg2 "Generating the package..." + fi pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 -- 2.9.3
On 01/09/16 07:05, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)
Look at the definition of msg2() etc: scripts/library/output_format.sh
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 322bb66..706c6fb 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,10 +57,11 @@ usage() { -q, --quiet silence most of the status reporting --nocolor turn off color in output --pacnew package .pacnew files + -o, --out <dir> write output to <dir> Examples: # $myname linux-headers # $myname gzip munge binutils -o ~/Downloads - # $myname --nocolor --pacnew gzip munge binutils + # $myname --nocolor --pacnew -o /tmp gzip munge binutils # $myname \$(pacman -Qsq) EOF } @@ -88,8 +89,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT=':qv' -OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT=':o:qv' +OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -99,6 +100,10 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -o|--out) + pkg_dest="$(readlink -e $2)" + [[ $? -ne 0 ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 + shift ;; -q|--quiet) QUIET='y' ;; --nocolor) @@ -150,7 +155,8 @@ source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi -pkg_dest="${PKGDEST:-$PWD}" +PKGDEST="${PKGDEST:-$PWD}" +pkg_dest="${pkg_dest:-$PKGDEST}" pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database -- 2.9.3
On 01/09/16 07:05, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 322bb66..706c6fb 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,10 +57,11 @@ usage() { -q, --quiet silence most of the status reporting --nocolor turn off color in output --pacnew package .pacnew files + -o, --out <dir> write output to <dir>
Examples: # $myname linux-headers # $myname gzip munge binutils -o ~/Downloads - # $myname --nocolor --pacnew gzip munge binutils + # $myname --nocolor --pacnew -o /tmp gzip munge binutils # $myname \$(pacman -Qsq) EOF } @@ -88,8 +89,8 @@ for option in "$@"; do done
# Parse arguments -OPT_SHORT=':qv' -OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT=':o:qv' +OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -99,6 +100,10 @@ unset OPT_SHORT OPT_LONG OPTRET
while :; do case "$1" in + -o|--out) + pkg_dest="$(readlink -e $2)"
readlink -e is not portable.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 706c6fb..e8e53ea 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,11 +57,12 @@ usage() { -q, --quiet silence most of the status reporting --nocolor turn off color in output --pacnew package .pacnew files + -j, --jobs <jobs> build in parallel - you may want to set XZ_OPT -o, --out <dir> write output to <dir> Examples: # $myname linux-headers # $myname gzip munge binutils -o ~/Downloads - # $myname --nocolor --pacnew -o /tmp gzip munge binutils + # $myname --nocolor --pacnew -o /tmp -j 5 gzip munge binutils # $myname \$(pacman -Qsq) EOF } @@ -89,8 +90,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT=':o:qv' -OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT=':o:j:qv' +OPT_LONG=('out:' 'jobs:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -104,6 +105,14 @@ while :; do pkg_dest="$(readlink -e $2)" [[ $? -ne 0 ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 shift ;; + -j|--jobs) + if [[ $2 =~ ^-?[0-9]+$ ]]; then + MAX_JOBS=$2 + else + echo -e "\e[39;1m$2\e[0m is not a valid integer!" + exit -1 + fi + shift ;; -q|--quiet) QUIET='y' ;; --nocolor) @@ -190,7 +199,9 @@ fakebuild() { cd "$work_dir" || exit 1 # Assemble list of files which belong to the package and tar them - if [[ "$QUIET" != "y" ]]; then + if [[ $MAX_JOBS -gt 1 && "$QUIET" != "y" ]]; then + msg2 "${pkg_name}: Copying package files..." + elif [[ "$QUIET" != "y" ]]; then msg2 "Copying package files..." fi @@ -267,7 +278,9 @@ fakebuild() { # Reconstruct .PKGINFO from database # TODO adopt makepkg's write_pkginfo() into this or scripts/library - if [[ "$QUIET" != "y" ]]; then + if [[ $MAX_JOBS -gt 1 && "$QUIET" != "y" ]]; then + msg2 "${pkg_name}: Generating .PKGINFO metadata..." + elif [[ "$QUIET" != "y" ]]; then msg2 "Generating .PKGINFO metadata..." fi echo "# Generated by $myname $myver" > .PKGINFO @@ -357,7 +370,9 @@ fakebuild() { chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null # Generate the package - if [[ "$QUIET" != "y" ]]; then + if [[ $MAX_JOBS -gt 1 && "$QUIET" != "y" ]]; then + msg2 "${pkg_name}: Generating the package..." + elif [[ "$QUIET" != "y" ]]; then msg2 "Generating the package..." fi -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/doc/.gitignore | 1 + contrib/doc/Makefile.am | 5 +++- contrib/doc/bacman.8.txt | 71 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 1 deletion(-) create mode 100644 contrib/doc/bacman.8.txt diff --git a/contrib/doc/.gitignore b/contrib/doc/.gitignore index c5612bc..3ab2035 100644 --- a/contrib/doc/.gitignore +++ b/contrib/doc/.gitignore @@ -1 +1,2 @@ verify-pacman-repo-db.1 +bacman.8 diff --git a/contrib/doc/Makefile.am b/contrib/doc/Makefile.am index 4c316bb..d5725b1 100644 --- a/contrib/doc/Makefile.am +++ b/contrib/doc/Makefile.am @@ -4,12 +4,14 @@ # man_MANS if --enable-asciidoc and/or --enable-doxygen are used. ASCIIDOC_MANS = \ - verify-pacman-repo-db.1 + verify-pacman-repo-db.1 \ + bacman.8 EXTRA_DIST = \ asciidoc.conf \ footer.txt \ verify-pacman-repo-db.1.txt \ + bacman.8.txt \ $(ASCIIDOC_MANS) # Files that should be removed, but which Automake does not know. @@ -53,5 +55,6 @@ $(ASCIIDOC_MANS): asciidoc.conf footer.txt Makefile.am # Dependency rules verify-pacman-repo-db.1: verify-pacman-repo-db.1.txt +bacman.8: bacman.8.txt # vim:set noet: diff --git a/contrib/doc/bacman.8.txt b/contrib/doc/bacman.8.txt new file mode 100644 index 0000000..2c3112f --- /dev/null +++ b/contrib/doc/bacman.8.txt @@ -0,0 +1,71 @@ +///// +vim:set ts=4 sw=4 syntax=asciidoc noet spell spelllang=en_us: +///// +bacman(8) +========== + +Name +---- +bacman - reassemble installed packages + + +Synopsis +-------- +'bacman' [options] <package(s)> + + +Description +----------- +'bacman' was designed to reassemble installed packages from its deliverd files. +It comes in handy if there is no internet connection available and you have no +access to a up-to-date package cache. Otherwise packages from the Arch Linux +Archive should always be preferred. + +'bacman' fully honors linkman:makepkg.conf[8] and all compression environment +variables, notably ``XZ_OPT''. + + +Options +------- +*-h, \--help*:: + Display usage information. + +*-q, \--quiet*:: + Silence most of the status reporting. + +*\--nocolor*:: + Disable colored output. + +*\--pacnew*:: + Package .pacnew files instead of the concerning files currently in place. + +*-j, \--jobs <jobs>*:: + Assemble multiple packages in parallel. + +*-o, \--out <dir>*:: + Write the assembled package(s) to the specified directory. + + +Examples +-------- +'bacman' linux-headers + +Recreate the package ``linux-headers''. + +'bacman' gzip munge binutils -o ~/Downloads + +Assemble the packages gzip munge binutils and place the packages at +``~/Downloads''. + +'bacman' --nocolor --pacnew -o ~/backup -j 5 $(pacman -Qsq) + +Assemble all currently installed packages using 5 concurrent jobs, use +``.pacnew'' whenever available, suppress colored output and place the desired +packages in ``~/backup''. + + +See Also +-------- +linkman:makepkg[8], linkman:pacman[8] + +include::footer.txt[] -- 2.9.3
On 01/09/16 07:05, Gordian Edenhofer wrote:
To enable the creation of multiple packages with one command move the assembly process into its own function. Handle SIGHUP, SIGINT, SIGTERM and remove working directories accordingly. Add some comments.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com>
OK. I included one minor change below:
+ + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi + msg "Done." +}
-rm -rf "$work_dir"
-msg "Done." +for PKG in $@; do fakebuild $PKG; done
Moved the "Done" message to be displayed once at the end.
To enable the creation of multiple packages with one command move the assembly process into its own function. Handle SIGHUP, SIGINT, SIGTERM and remove working directories accordingly. Add some comments. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Display the done message only once at the end. contrib/bacman.sh.in | 440 +++++++++++++++++++++++++-------------------------- 1 file changed, 219 insertions(+), 221 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a611c1a..512973a 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -33,9 +33,18 @@ ARGS=("$@") m4_include(../scripts/library/output_format.sh) -# -# User Friendliness -# +# Lazy recursive clean up of temporary dirs +work_dir_root="${TMPDIR:-/tmp}/bacman" +clean_up() { + rm -rf "$work_dir_root".* + echo + exit +} + +# Trap termination signals +trap clean_up SIGHUP SIGINT SIGTERM + +# Print usage information usage() { echo "${myname} (pacman) v${myver}" echo @@ -46,12 +55,14 @@ usage() { echo "Example: ${myname} linux-headers" } +# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } +# Check for specified arguments while [[ ! -z $1 ]]; do if [[ $1 == "--nocolor" ]]; then USE_COLOR='n' @@ -64,13 +75,16 @@ while [[ ! -z $1 ]]; do fi done +# Configure colored output m4_include(../scripts/library/term_colors.sh) -if (( $# != 1 )); then +# Break if no argument was given +if (( $# < 1 )); then usage exit 1 fi +# Print usage or version if requested if [[ $1 = -@(h|-help) ]]; then usage exit 0 @@ -79,9 +93,7 @@ elif [[ $1 = -@(V|-version) ]]; then exit 0 fi -# -# Fakeroot support -# +# Run with fake root privileges if EUID is not root if (( EUID )); then if [[ -f /usr/bin/fakeroot ]]; then msg "Entering fakeroot environment" @@ -94,263 +106,249 @@ if (( EUID )); then fi fi -# -# Setting environmental variables -# +# Source environmental variables and specify fallbacks if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 fi - eval $(awk '/DBPath/ {print $1$2$3}' @sysconfdir@/pacman.conf) pac_db="${DBPath:-@localstatedir@/lib/pacman/}/local" - if [[ ! -r @sysconfdir@/makepkg.conf ]]; then error "unable to read @sysconfdir@/makepkg.conf" exit 1 fi - source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi - pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -pkg_name="$1" -pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) -pkg_namver=("${pkg_dir[@]##*/}") - -# -# Checks everything is in place -# +# Check for an existing database if [[ ! -d $pac_db ]]; then error "pacman database directory ${pac_db} not found" exit 1 fi -if (( ${#pkg_dir[@]} != 1 )); then - error "%d entries for package %s found in pacman database" \ - ${#pkg_dir[@]} "${pkg_name}" - msg2 "%s" "${pkg_dir[@]}" - exit 1 -fi - -if [[ ! -d $pkg_dir ]]; then - error "package %s is found in pacman database," "${pkg_name}" - plain " but '%s' is not a directory" "${pkg_dir}" - exit 1 -fi - -# -# Begin -# -msg "Package: ${pkg_namver}" -work_dir=$(mktemp -d "${TMPDIR:-/tmp}/bacman.XXXXXXXXXX") -cd "$work_dir" || exit 1 - -# -# File copying -# -msg2 "Copying package files..." - -while read i; do - if [[ -z $i ]]; then - continue +# Assemble a single package: $1 = pkgname +fakebuild() { + pkg_name="$1" + pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) + pkg_namver=("${pkg_dir[@]##*/}") + + # Checks database for specified package + if (( ${#pkg_dir[@]} != 1 )); then + error "%d entries for package %s found in pacman database" \ + ${#pkg_dir[@]} "${pkg_name}" + msg2 "%s" "${pkg_dir[@]}" + exit 1 fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + if [[ ! -d $pkg_dir ]]; then + error "package %s is found in pacman database," "${pkg_name}" + plain " but '%s' is not a directory" "${pkg_dir}" + exit 1 fi - case "$current" in - %FILES%) - local_file="/$i" - package_file="$work_dir/$i" + # Create working directory + msg "Package: ${pkg_namver}" + work_dir=$(mktemp -d "${work_dir_root}.XXXXXXXXXX") + cd "$work_dir" || exit 1 - if [[ ! -e $local_file ]]; then - warning "package file $local_file is missing" - continue - fi - ;; - - %BACKUP%) - # Get the MD5 checksum. - original_md5="${i##*$'\t'}" - # Strip the md5sum after the tab. - i="${i%$'\t'*}" - local_file="/$i.pacnew" - package_file="$work_dir/$i" - - # Include unmodified .pacnew files. - local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" - if [[ $INCLUDE_PACNEW == 'n' ]] \ - || [[ ! -e $local_file ]] \ - || [[ $local_md5 != $original_md5 ]]; then - # Warn about modified files. - local_md5="$(md5sum "/$i" | cut -d' ' -f1)" - if [[ $local_md5 != $original_md5 ]]; then - warning "package file /$i has been modified" - fi - # Let the normal file be included in the %FILES% list. - continue - fi - ;; + # Assemble list of files which belong to the package and tar them + msg2 "Copying package files..." + while read i; do + if [[ -z $i ]]; then + continue + fi - *) + if [[ $i == %+([A-Z])% ]]; then + current=$i continue - ;; - esac + fi - ret=0 - bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + case "$current" in + %FILES%) + local_file="/$i" + package_file="$work_dir/$i" - # Workaround to bsdtar not reporting a missing file as an error - if ! [[ -e $package_file || -L $package_file ]]; then - error "unable to add $local_file to the package" - plain " If your user does not have permission to read this file, then" - plain " you will need to run $myname as root." + if [[ ! -e $local_file ]]; then + warning "package file $local_file is missing" + continue + fi + ;; + + %BACKUP%) + # Get the MD5 checksum. + original_md5="${i##*$'\t'}" + # Strip the md5sum after the tab. + i="${i%$'\t'*}" + local_file="/$i.pacnew" + package_file="$work_dir/$i" + + # Include unmodified .pacnew files. + local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" + if [[ $INCLUDE_PACNEW == 'n' ]] \ + || [[ ! -e $local_file ]] \ + || [[ $local_md5 != $original_md5 ]]; then + # Warn about modified files. + local_md5="$(md5sum "/$i" | cut -d' ' -f1)" + if [[ $local_md5 != $original_md5 ]]; then + warning "package file /$i has been modified" + fi + # Let the normal file be included in the %FILES% list. + continue + fi + ;; + + *) + continue + ;; + esac + + # Tar files + ret=0 + bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + # Workaround to bsdtar not reporting a missing file as an error + if ! [[ -e $package_file || -L $package_file ]]; then + error "unable to add $local_file to the package" + plain " If your user does not have permission to read this file, then" + plain " you will need to run $myname as root." + rm -rf "$work_dir" + exit 1 + fi + done < "$pkg_dir"/files + + ret=$? + if (( ret )); then rm -rf "$work_dir" exit 1 fi -done < "$pkg_dir"/files -ret=$? -if (( ret )); then - rm -rf "$work_dir" - exit 1 -fi - -pkg_size=$(du -sk | awk '{print $1 * 1024}') - -# -# .PKGINFO stuff -# TODO adopt makepkg's write_pkginfo() into this or scripts/library -# -msg2 "Generating .PKGINFO metadata..." -echo "# Generated by $myname $myver" > .PKGINFO -if [[ $INFAKEROOT == "1" ]]; then - echo "# Using $(fakeroot -v)" >> .PKGINFO -fi -echo "# $(LC_ALL=C date)" >> .PKGINFO -echo "#" >> .PKGINFO + # Calculate package size + pkg_size=$(du -sk | awk '{print $1 * 1024}') -while read i; do - if [[ -z $i ]]; then - continue; + # Reconstruct .PKGINFO from database + # TODO adopt makepkg's write_pkginfo() into this or scripts/library + msg2 "Generating .PKGINFO metadata..." + echo "# Generated by $myname $myver" > .PKGINFO + if [[ $INFAKEROOT == "1" ]]; then + echo "# Using $(fakeroot -v)" >> .PKGINFO fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + echo "# $(LC_ALL=C date)" >> .PKGINFO + echo "#" >> .PKGINFO + while read i; do + if [[ -z $i ]]; then + continue; + fi + if [[ $i == %+([A-Z])% ]]; then + current=$i + continue + fi + + case "$current" in + # desc + %NAME%) + echo "pkgname = $i" >> .PKGINFO + ;; + %VERSION%) + echo "pkgver = $i" >> .PKGINFO + ;; + %DESC%) + echo "pkgdesc = $i" >> .PKGINFO + ;; + %URL%) + echo "url = $i" >> .PKGINFO + ;; + %LICENSE%) + echo "license = $i" >> .PKGINFO + ;; + %ARCH%) + echo "arch = $i" >> .PKGINFO + pkg_arch="$i" + ;; + %BUILDDATE%) + echo "builddate = $(date -u "+%s")" >> .PKGINFO + ;; + %PACKAGER%) + echo "packager = $pkg_pkger" >> .PKGINFO + ;; + %SIZE%) + echo "size = $pkg_size" >> .PKGINFO + ;; + %GROUPS%) + echo "group = $i" >> .PKGINFO + ;; + %REPLACES%) + echo "replaces = $i" >> .PKGINFO + ;; + %DEPENDS%) + echo "depend = $i" >> .PKGINFO + ;; + %OPTDEPENDS%) + echo "optdepend = $i" >> .PKGINFO + ;; + %CONFLICTS%) + echo "conflict = $i" >> .PKGINFO + ;; + %PROVIDES%) + echo "provides = $i" >> .PKGINFO + ;; + %BACKUP%) + # Strip the md5sum after the tab + echo "backup = ${i%%$'\t'*}" >> .PKGINFO + ;; + esac + done < <(cat "$pkg_dir"/{desc,files}) + + comp_files=".PKGINFO" + + # Add instal file if present + if [[ -f $pkg_dir/install ]]; then + cp "$pkg_dir/install" "$work_dir/.INSTALL" + comp_files+=" .INSTALL" + fi + if [[ -f $pkg_dir/changelog ]]; then + cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" + comp_files+=" .CHANGELOG" fi - case "$current" in - # desc - %NAME%) - echo "pkgname = $i" >> .PKGINFO - ;; - %VERSION%) - echo "pkgver = $i" >> .PKGINFO - ;; - %DESC%) - echo "pkgdesc = $i" >> .PKGINFO - ;; - %URL%) - echo "url = $i" >> .PKGINFO - ;; - %LICENSE%) - echo "license = $i" >> .PKGINFO - ;; - %ARCH%) - echo "arch = $i" >> .PKGINFO - pkg_arch="$i" - ;; - %BUILDDATE%) - echo "builddate = $(date -u "+%s")" >> .PKGINFO - ;; - %PACKAGER%) - echo "packager = $pkg_pkger" >> .PKGINFO - ;; - %SIZE%) - echo "size = $pkg_size" >> .PKGINFO - ;; - %GROUPS%) - echo "group = $i" >> .PKGINFO - ;; - %REPLACES%) - echo "replaces = $i" >> .PKGINFO - ;; - %DEPENDS%) - echo "depend = $i" >> .PKGINFO - ;; - %OPTDEPENDS%) - echo "optdepend = $i" >> .PKGINFO - ;; - %CONFLICTS%) - echo "conflict = $i" >> .PKGINFO - ;; - %PROVIDES%) - echo "provides = $i" >> .PKGINFO - ;; - - # files - %BACKUP%) - # Strip the md5sum after the tab - echo "backup = ${i%%$'\t'*}" >> .PKGINFO - ;; - esac -done < <(cat "$pkg_dir"/{desc,files}) - -comp_files=".PKGINFO" - -if [[ -f $pkg_dir/install ]]; then - cp "$pkg_dir/install" "$work_dir/.INSTALL" - comp_files+=" .INSTALL" -fi -if [[ -f $pkg_dir/changelog ]]; then - cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" - comp_files+=" .CHANGELOG" -fi + # Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL + chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -# -# Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL -# -chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + # Generate the package + msg2 "Generating the package..." -# -# Generate the package -# -msg2 "Generating the package..." - -pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" -ret=0 - -# TODO: Maybe this can be set globally for robustness -shopt -s -o pipefail -bsdtar -cf - $comp_files * | -case "$PKGEXT" in - *tar.gz) gzip -c -f -n ;; - *tar.bz2) bzip2 -c -f ;; - *tar.xz) xz -c -z - ;; - *tar.Z) compress -c -f ;; - *tar) cat ;; - *) warning "'%s' is not a valid archive extension." \ - "$PKGEXT"; cat ;; -esac > "${pkg_file}"; ret=$? - -if (( ret )); then - error "Unable to write package to $pkg_dest" - plain " Maybe the disk is full or you do not have write access" + pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" + ret=0 + + # TODO: Maybe this can be set globally for robustness + shopt -s -o pipefail + bsdtar -cf - $comp_files * | + case "$PKGEXT" in + *tar.gz) gzip -c -f -n ;; + *tar.bz2) bzip2 -c -f ;; + *tar.xz) xz -c -z - ;; + *tar.Z) compress -c -f ;; + *tar) cat ;; + *) warning "'%s' is not a valid archive extension." \ + "$PKGEXT"; cat ;; + esac > "${pkg_file}"; ret=$? + + # Move compressed package to destination + if (( ret )); then + error "Unable to write package to $pkg_dest" + plain " Maybe the disk is full or you do not have write access" + rm -rf "$work_dir" + exit 1 + fi + + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi +} -rm -rf "$work_dir" +for PKG in $@; do fakebuild $PKG; done msg "Done." exit 0 -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 512973a..a718a18 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -347,8 +347,29 @@ fakebuild() { rm -rf "$work_dir" } +# Run fakebuild in parralel with at maximum $MAX_JOBS jobs +# By default only run one job +MAX_JOBS=${MAX_JOBS:-1} +# TODO: Use wait -n instead of this hack as soon as it is ok +# to bump the version of the bash dependency +parallelize() { + while [[ $# -gt 0 ]]; do + job_count=($(jobs -p)) + if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then + fakebuild "$1" & + shift + fi + sleep 0.1 + done + wait +} -for PKG in $@; do fakebuild $PKG; done +# Initiate assembly function +if [[ $MAX_JOBS -gt "1" ]]; then + parallelize "$@" +else + for PKG in $@; do fakebuild $PKG; done +fi msg "Done." exit 0 -- 2.9.3
Switch to parseopts instead of merely checking the first argument. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Remove the leading colon in the OPT_SHORT variable. Thanks Allen for the hint. contrib/bacman.sh.in | 63 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 22 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a718a18..51283f3 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -32,6 +32,7 @@ INCLUDE_PACNEW='n' ARGS=("$@") m4_include(../scripts/library/output_format.sh) +m4_include(../scripts/library/parseopts.sh) # Lazy recursive clean up of temporary dirs work_dir_root="${TMPDIR:-/tmp}/bacman" @@ -62,35 +63,51 @@ version() { echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } -# Check for specified arguments -while [[ ! -z $1 ]]; do - if [[ $1 == "--nocolor" ]]; then - USE_COLOR='n' - shift - elif [[ $1 == "--pacnew" ]]; then - INCLUDE_PACNEW='y' - shift - else - break - fi -done - # Configure colored output m4_include(../scripts/library/term_colors.sh) # Break if no argument was given -if (( $# < 1 )); then +if (( $# == 0 )); then + usage + exit 1 +fi + +# Printing the usage information takes precedence over every other parameter +for option in "$@"; do + [[ $option == "-h" || $option == "--help" ]] && usage && exit 0 +done + +# Parse arguments +OPT_SHORT='v' +OPT_LONG=('nocolor' 'pacnew' 'version') +if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 fi +set -- "${OPTRET[@]}" +unset OPT_SHORT OPT_LONG OPTRET + +while :; do + case "$1" in + --nocolor) + USE_COLOR='n' ;; + --pacnew) + INCLUDE_PACNEW='y' ;; + -v|--version) + version + exit 0 ;; + --) + shift + break 2 ;; + esac + shift +done -# Print usage or version if requested -if [[ $1 = -@(h|-help) ]]; then +# Retrieve the list of packages to be assembled and break if none was specified +pkg_list=($*) +if [[ ${#pkg_list[@]} == 0 ]]; then usage - exit 0 -elif [[ $1 = -@(V|-version) ]]; then - version - exit 0 + exit 1 fi # Run with fake root privileges if EUID is not root @@ -366,9 +383,11 @@ parallelize() { # Initiate assembly function if [[ $MAX_JOBS -gt "1" ]]; then - parallelize "$@" + parallelize "${pkg_list[@]}" else - for PKG in $@; do fakebuild $PKG; done + for PKG in ${pkg_list[@]}; do + fakebuild $PKG + done fi msg "Done." -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Move the '-o' occurrance in the usage page to its corresponding commit. Use the old descrption instead of the new one. (Fixes typo) Thanks again Allan! contrib/bacman.sh.in | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 51283f3..20f5939 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -41,25 +41,32 @@ clean_up() { echo exit } - # Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM # Print usage information usage() { - echo "${myname} (pacman) v${myver}" - echo - echo "Recreate a package using pacman's database and system files" - echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" - echo - echo "Example: ${myname} linux-headers" + cat <<-EOF + $myname (pacman) v$myver + Recreate packages using pacman's database and system files + + Usage: $myname [options] <package(s)> + + -h, --help display this help message and exit + --nocolor turn off color in output + --pacnew package .pacnew files + + Examples: # $myname linux-headers + # $myname --nocolor --pacnew gzip munge binutils + # $myname \$(pacman -Qsq) + EOF } # Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' + echo 'Copyright (C) 2016 Gordian Edenhofer <gordian.edenhofer@gmail.com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } @@ -123,7 +130,7 @@ if (( EUID )); then fi fi -# Source environmental variables and specify fallbacks +# Read in environmental variables if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 @@ -139,7 +146,7 @@ if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi pkg_dest="${PKGDEST:-$PWD}" -pkg_pkger=${PACKAGER:-'Unknown Packager'} +pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database if [[ ! -d $pac_db ]]; then @@ -339,6 +346,7 @@ fakebuild() { pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 + # Move compressed package to destination # TODO: Maybe this can be set globally for robustness shopt -s -o pipefail bsdtar -cf - $comp_files * | @@ -352,7 +360,7 @@ fakebuild() { "$PKGEXT"; cat ;; esac > "${pkg_file}"; ret=$? - # Move compressed package to destination + # Evaluate return code if (( ret )); then error "Unable to write package to $pkg_dest" plain " Maybe the disk is full or you do not have write access" -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Switch to the native QUIET option of msg2(). Sorry for not using it in the first place. contrib/bacman.sh.in | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 20f5939..0ae9da0 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -28,6 +28,7 @@ declare -r myname='bacman' declare -r myver='@PACKAGE_VERSION@' USE_COLOR='y' INCLUDE_PACNEW='n' +QUIET=0 # Required for fakeroot because options are shifted off the array. ARGS=("$@") @@ -53,6 +54,7 @@ usage() { Usage: $myname [options] <package(s)> -h, --help display this help message and exit + -q, --quiet silence most of the status reporting --nocolor turn off color in output --pacnew package .pacnew files @@ -85,8 +87,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='v' -OPT_LONG=('nocolor' 'pacnew' 'version') +OPT_SHORT='qv' +OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -96,6 +98,8 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -q|--quiet) + QUIET=1 ;; --nocolor) USE_COLOR='n' ;; --pacnew) @@ -180,6 +184,7 @@ fakebuild() { # Assemble list of files which belong to the package and tar them msg2 "Copying package files..." + while read i; do if [[ -z $i ]]; then continue -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Switch to the build-in check for a directory existance instead of using 'readlink -e'. contrib/bacman.sh.in | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 0ae9da0..fef5e78 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,9 +57,11 @@ usage() { -q, --quiet silence most of the status reporting --nocolor turn off color in output --pacnew package .pacnew files + -o, --out <dir> write output to <dir> Examples: # $myname linux-headers - # $myname --nocolor --pacnew gzip munge binutils + # $myname gzip munge binutils -o ~/Downloads + # $myname --nocolor --pacnew -o /tmp -j 5 gzip munge binutils # $myname \$(pacman -Qsq) EOF } @@ -87,8 +89,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='qv' -OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT='o:qv' +OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -98,6 +100,10 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -o|--out) + pkg_dest=$2 + [[ ! -d "$2" ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 + shift ;; -q|--quiet) QUIET=1 ;; --nocolor) @@ -149,7 +155,8 @@ source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi -pkg_dest="${PKGDEST:-$PWD}" +PKGDEST="${PKGDEST:-$PWD}" +pkg_dest="${pkg_dest:-$PKGDEST}" pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index fef5e78..6158a2c 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,6 +57,7 @@ usage() { -q, --quiet silence most of the status reporting --nocolor turn off color in output --pacnew package .pacnew files + -j, --jobs <jobs> build in parallel - you may want to set XZ_OPT -o, --out <dir> write output to <dir> Examples: # $myname linux-headers @@ -89,8 +90,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='o:qv' -OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT='o:j:qv' +OPT_LONG=('out:' 'jobs:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -104,6 +105,14 @@ while :; do pkg_dest=$2 [[ ! -d "$2" ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 shift ;; + -j|--jobs) + if [[ $2 =~ ^-?[0-9]+$ ]]; then + MAX_JOBS=$2 + else + echo -e "\e[39;1m$2\e[0m is not a valid integer!" + exit -1 + fi + shift ;; -q|--quiet) QUIET=1 ;; --nocolor) @@ -190,7 +199,11 @@ fakebuild() { cd "$work_dir" || exit 1 # Assemble list of files which belong to the package and tar them - msg2 "Copying package files..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Copying package files..." + else + msg2 "Copying package files..." + fi while read i; do if [[ -z $i ]]; then @@ -265,7 +278,11 @@ fakebuild() { # Reconstruct .PKGINFO from database # TODO adopt makepkg's write_pkginfo() into this or scripts/library - msg2 "Generating .PKGINFO metadata..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Generating .PKGINFO metadata..." + else + msg2 "Generating .PKGINFO metadata..." + fi echo "# Generated by $myname $myver" > .PKGINFO if [[ $INFAKEROOT == "1" ]]; then echo "# Using $(fakeroot -v)" >> .PKGINFO @@ -353,7 +370,11 @@ fakebuild() { chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null # Generate the package - msg2 "Generating the package..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Generating the package..." + else + msg2 "Generating the package..." + fi pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/doc/.gitignore | 1 + contrib/doc/Makefile.am | 5 +++- contrib/doc/bacman.8.txt | 71 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 1 deletion(-) create mode 100644 contrib/doc/bacman.8.txt diff --git a/contrib/doc/.gitignore b/contrib/doc/.gitignore index c5612bc..3ab2035 100644 --- a/contrib/doc/.gitignore +++ b/contrib/doc/.gitignore @@ -1 +1,2 @@ verify-pacman-repo-db.1 +bacman.8 diff --git a/contrib/doc/Makefile.am b/contrib/doc/Makefile.am index 4c316bb..d5725b1 100644 --- a/contrib/doc/Makefile.am +++ b/contrib/doc/Makefile.am @@ -4,12 +4,14 @@ # man_MANS if --enable-asciidoc and/or --enable-doxygen are used. ASCIIDOC_MANS = \ - verify-pacman-repo-db.1 + verify-pacman-repo-db.1 \ + bacman.8 EXTRA_DIST = \ asciidoc.conf \ footer.txt \ verify-pacman-repo-db.1.txt \ + bacman.8.txt \ $(ASCIIDOC_MANS) # Files that should be removed, but which Automake does not know. @@ -53,5 +55,6 @@ $(ASCIIDOC_MANS): asciidoc.conf footer.txt Makefile.am # Dependency rules verify-pacman-repo-db.1: verify-pacman-repo-db.1.txt +bacman.8: bacman.8.txt # vim:set noet: diff --git a/contrib/doc/bacman.8.txt b/contrib/doc/bacman.8.txt new file mode 100644 index 0000000..2c3112f --- /dev/null +++ b/contrib/doc/bacman.8.txt @@ -0,0 +1,71 @@ +///// +vim:set ts=4 sw=4 syntax=asciidoc noet spell spelllang=en_us: +///// +bacman(8) +========== + +Name +---- +bacman - reassemble installed packages + + +Synopsis +-------- +'bacman' [options] <package(s)> + + +Description +----------- +'bacman' was designed to reassemble installed packages from its deliverd files. +It comes in handy if there is no internet connection available and you have no +access to a up-to-date package cache. Otherwise packages from the Arch Linux +Archive should always be preferred. + +'bacman' fully honors linkman:makepkg.conf[8] and all compression environment +variables, notably ``XZ_OPT''. + + +Options +------- +*-h, \--help*:: + Display usage information. + +*-q, \--quiet*:: + Silence most of the status reporting. + +*\--nocolor*:: + Disable colored output. + +*\--pacnew*:: + Package .pacnew files instead of the concerning files currently in place. + +*-j, \--jobs <jobs>*:: + Assemble multiple packages in parallel. + +*-o, \--out <dir>*:: + Write the assembled package(s) to the specified directory. + + +Examples +-------- +'bacman' linux-headers + +Recreate the package ``linux-headers''. + +'bacman' gzip munge binutils -o ~/Downloads + +Assemble the packages gzip munge binutils and place the packages at +``~/Downloads''. + +'bacman' --nocolor --pacnew -o ~/backup -j 5 $(pacman -Qsq) + +Assemble all currently installed packages using 5 concurrent jobs, use +``.pacnew'' whenever available, suppress colored output and place the desired +packages in ``~/backup''. + + +See Also +-------- +linkman:makepkg[8], linkman:pacman[8] + +include::footer.txt[] -- 2.9.3
To enable the creation of multiple packages with one command move the assembly process into its own function. Handle SIGHUP, SIGINT, SIGTERM and remove working directories accordingly. Add some comments. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 440 +++++++++++++++++++++++++-------------------------- 1 file changed, 219 insertions(+), 221 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a611c1a..512973a 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -33,9 +33,18 @@ ARGS=("$@") m4_include(../scripts/library/output_format.sh) -# -# User Friendliness -# +# Lazy recursive clean up of temporary dirs +work_dir_root="${TMPDIR:-/tmp}/bacman" +clean_up() { + rm -rf "$work_dir_root".* + echo + exit +} + +# Trap termination signals +trap clean_up SIGHUP SIGINT SIGTERM + +# Print usage information usage() { echo "${myname} (pacman) v${myver}" echo @@ -46,12 +55,14 @@ usage() { echo "Example: ${myname} linux-headers" } +# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } +# Check for specified arguments while [[ ! -z $1 ]]; do if [[ $1 == "--nocolor" ]]; then USE_COLOR='n' @@ -64,13 +75,16 @@ while [[ ! -z $1 ]]; do fi done +# Configure colored output m4_include(../scripts/library/term_colors.sh) -if (( $# != 1 )); then +# Break if no argument was given +if (( $# < 1 )); then usage exit 1 fi +# Print usage or version if requested if [[ $1 = -@(h|-help) ]]; then usage exit 0 @@ -79,9 +93,7 @@ elif [[ $1 = -@(V|-version) ]]; then exit 0 fi -# -# Fakeroot support -# +# Run with fake root privileges if EUID is not root if (( EUID )); then if [[ -f /usr/bin/fakeroot ]]; then msg "Entering fakeroot environment" @@ -94,263 +106,249 @@ if (( EUID )); then fi fi -# -# Setting environmental variables -# +# Source environmental variables and specify fallbacks if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 fi - eval $(awk '/DBPath/ {print $1$2$3}' @sysconfdir@/pacman.conf) pac_db="${DBPath:-@localstatedir@/lib/pacman/}/local" - if [[ ! -r @sysconfdir@/makepkg.conf ]]; then error "unable to read @sysconfdir@/makepkg.conf" exit 1 fi - source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi - pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -pkg_name="$1" -pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) -pkg_namver=("${pkg_dir[@]##*/}") - -# -# Checks everything is in place -# +# Check for an existing database if [[ ! -d $pac_db ]]; then error "pacman database directory ${pac_db} not found" exit 1 fi -if (( ${#pkg_dir[@]} != 1 )); then - error "%d entries for package %s found in pacman database" \ - ${#pkg_dir[@]} "${pkg_name}" - msg2 "%s" "${pkg_dir[@]}" - exit 1 -fi - -if [[ ! -d $pkg_dir ]]; then - error "package %s is found in pacman database," "${pkg_name}" - plain " but '%s' is not a directory" "${pkg_dir}" - exit 1 -fi - -# -# Begin -# -msg "Package: ${pkg_namver}" -work_dir=$(mktemp -d "${TMPDIR:-/tmp}/bacman.XXXXXXXXXX") -cd "$work_dir" || exit 1 - -# -# File copying -# -msg2 "Copying package files..." - -while read i; do - if [[ -z $i ]]; then - continue +# Assemble a single package: $1 = pkgname +fakebuild() { + pkg_name="$1" + pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) + pkg_namver=("${pkg_dir[@]##*/}") + + # Checks database for specified package + if (( ${#pkg_dir[@]} != 1 )); then + error "%d entries for package %s found in pacman database" \ + ${#pkg_dir[@]} "${pkg_name}" + msg2 "%s" "${pkg_dir[@]}" + exit 1 fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + if [[ ! -d $pkg_dir ]]; then + error "package %s is found in pacman database," "${pkg_name}" + plain " but '%s' is not a directory" "${pkg_dir}" + exit 1 fi - case "$current" in - %FILES%) - local_file="/$i" - package_file="$work_dir/$i" + # Create working directory + msg "Package: ${pkg_namver}" + work_dir=$(mktemp -d "${work_dir_root}.XXXXXXXXXX") + cd "$work_dir" || exit 1 - if [[ ! -e $local_file ]]; then - warning "package file $local_file is missing" - continue - fi - ;; - - %BACKUP%) - # Get the MD5 checksum. - original_md5="${i##*$'\t'}" - # Strip the md5sum after the tab. - i="${i%$'\t'*}" - local_file="/$i.pacnew" - package_file="$work_dir/$i" - - # Include unmodified .pacnew files. - local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" - if [[ $INCLUDE_PACNEW == 'n' ]] \ - || [[ ! -e $local_file ]] \ - || [[ $local_md5 != $original_md5 ]]; then - # Warn about modified files. - local_md5="$(md5sum "/$i" | cut -d' ' -f1)" - if [[ $local_md5 != $original_md5 ]]; then - warning "package file /$i has been modified" - fi - # Let the normal file be included in the %FILES% list. - continue - fi - ;; + # Assemble list of files which belong to the package and tar them + msg2 "Copying package files..." + while read i; do + if [[ -z $i ]]; then + continue + fi - *) + if [[ $i == %+([A-Z])% ]]; then + current=$i continue - ;; - esac + fi - ret=0 - bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + case "$current" in + %FILES%) + local_file="/$i" + package_file="$work_dir/$i" - # Workaround to bsdtar not reporting a missing file as an error - if ! [[ -e $package_file || -L $package_file ]]; then - error "unable to add $local_file to the package" - plain " If your user does not have permission to read this file, then" - plain " you will need to run $myname as root." + if [[ ! -e $local_file ]]; then + warning "package file $local_file is missing" + continue + fi + ;; + + %BACKUP%) + # Get the MD5 checksum. + original_md5="${i##*$'\t'}" + # Strip the md5sum after the tab. + i="${i%$'\t'*}" + local_file="/$i.pacnew" + package_file="$work_dir/$i" + + # Include unmodified .pacnew files. + local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" + if [[ $INCLUDE_PACNEW == 'n' ]] \ + || [[ ! -e $local_file ]] \ + || [[ $local_md5 != $original_md5 ]]; then + # Warn about modified files. + local_md5="$(md5sum "/$i" | cut -d' ' -f1)" + if [[ $local_md5 != $original_md5 ]]; then + warning "package file /$i has been modified" + fi + # Let the normal file be included in the %FILES% list. + continue + fi + ;; + + *) + continue + ;; + esac + + # Tar files + ret=0 + bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + # Workaround to bsdtar not reporting a missing file as an error + if ! [[ -e $package_file || -L $package_file ]]; then + error "unable to add $local_file to the package" + plain " If your user does not have permission to read this file, then" + plain " you will need to run $myname as root." + rm -rf "$work_dir" + exit 1 + fi + done < "$pkg_dir"/files + + ret=$? + if (( ret )); then rm -rf "$work_dir" exit 1 fi -done < "$pkg_dir"/files -ret=$? -if (( ret )); then - rm -rf "$work_dir" - exit 1 -fi - -pkg_size=$(du -sk | awk '{print $1 * 1024}') - -# -# .PKGINFO stuff -# TODO adopt makepkg's write_pkginfo() into this or scripts/library -# -msg2 "Generating .PKGINFO metadata..." -echo "# Generated by $myname $myver" > .PKGINFO -if [[ $INFAKEROOT == "1" ]]; then - echo "# Using $(fakeroot -v)" >> .PKGINFO -fi -echo "# $(LC_ALL=C date)" >> .PKGINFO -echo "#" >> .PKGINFO + # Calculate package size + pkg_size=$(du -sk | awk '{print $1 * 1024}') -while read i; do - if [[ -z $i ]]; then - continue; + # Reconstruct .PKGINFO from database + # TODO adopt makepkg's write_pkginfo() into this or scripts/library + msg2 "Generating .PKGINFO metadata..." + echo "# Generated by $myname $myver" > .PKGINFO + if [[ $INFAKEROOT == "1" ]]; then + echo "# Using $(fakeroot -v)" >> .PKGINFO fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + echo "# $(LC_ALL=C date)" >> .PKGINFO + echo "#" >> .PKGINFO + while read i; do + if [[ -z $i ]]; then + continue; + fi + if [[ $i == %+([A-Z])% ]]; then + current=$i + continue + fi + + case "$current" in + # desc + %NAME%) + echo "pkgname = $i" >> .PKGINFO + ;; + %VERSION%) + echo "pkgver = $i" >> .PKGINFO + ;; + %DESC%) + echo "pkgdesc = $i" >> .PKGINFO + ;; + %URL%) + echo "url = $i" >> .PKGINFO + ;; + %LICENSE%) + echo "license = $i" >> .PKGINFO + ;; + %ARCH%) + echo "arch = $i" >> .PKGINFO + pkg_arch="$i" + ;; + %BUILDDATE%) + echo "builddate = $(date -u "+%s")" >> .PKGINFO + ;; + %PACKAGER%) + echo "packager = $pkg_pkger" >> .PKGINFO + ;; + %SIZE%) + echo "size = $pkg_size" >> .PKGINFO + ;; + %GROUPS%) + echo "group = $i" >> .PKGINFO + ;; + %REPLACES%) + echo "replaces = $i" >> .PKGINFO + ;; + %DEPENDS%) + echo "depend = $i" >> .PKGINFO + ;; + %OPTDEPENDS%) + echo "optdepend = $i" >> .PKGINFO + ;; + %CONFLICTS%) + echo "conflict = $i" >> .PKGINFO + ;; + %PROVIDES%) + echo "provides = $i" >> .PKGINFO + ;; + %BACKUP%) + # Strip the md5sum after the tab + echo "backup = ${i%%$'\t'*}" >> .PKGINFO + ;; + esac + done < <(cat "$pkg_dir"/{desc,files}) + + comp_files=".PKGINFO" + + # Add instal file if present + if [[ -f $pkg_dir/install ]]; then + cp "$pkg_dir/install" "$work_dir/.INSTALL" + comp_files+=" .INSTALL" + fi + if [[ -f $pkg_dir/changelog ]]; then + cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" + comp_files+=" .CHANGELOG" fi - case "$current" in - # desc - %NAME%) - echo "pkgname = $i" >> .PKGINFO - ;; - %VERSION%) - echo "pkgver = $i" >> .PKGINFO - ;; - %DESC%) - echo "pkgdesc = $i" >> .PKGINFO - ;; - %URL%) - echo "url = $i" >> .PKGINFO - ;; - %LICENSE%) - echo "license = $i" >> .PKGINFO - ;; - %ARCH%) - echo "arch = $i" >> .PKGINFO - pkg_arch="$i" - ;; - %BUILDDATE%) - echo "builddate = $(date -u "+%s")" >> .PKGINFO - ;; - %PACKAGER%) - echo "packager = $pkg_pkger" >> .PKGINFO - ;; - %SIZE%) - echo "size = $pkg_size" >> .PKGINFO - ;; - %GROUPS%) - echo "group = $i" >> .PKGINFO - ;; - %REPLACES%) - echo "replaces = $i" >> .PKGINFO - ;; - %DEPENDS%) - echo "depend = $i" >> .PKGINFO - ;; - %OPTDEPENDS%) - echo "optdepend = $i" >> .PKGINFO - ;; - %CONFLICTS%) - echo "conflict = $i" >> .PKGINFO - ;; - %PROVIDES%) - echo "provides = $i" >> .PKGINFO - ;; - - # files - %BACKUP%) - # Strip the md5sum after the tab - echo "backup = ${i%%$'\t'*}" >> .PKGINFO - ;; - esac -done < <(cat "$pkg_dir"/{desc,files}) - -comp_files=".PKGINFO" - -if [[ -f $pkg_dir/install ]]; then - cp "$pkg_dir/install" "$work_dir/.INSTALL" - comp_files+=" .INSTALL" -fi -if [[ -f $pkg_dir/changelog ]]; then - cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" - comp_files+=" .CHANGELOG" -fi + # Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL + chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -# -# Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL -# -chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + # Generate the package + msg2 "Generating the package..." -# -# Generate the package -# -msg2 "Generating the package..." - -pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" -ret=0 - -# TODO: Maybe this can be set globally for robustness -shopt -s -o pipefail -bsdtar -cf - $comp_files * | -case "$PKGEXT" in - *tar.gz) gzip -c -f -n ;; - *tar.bz2) bzip2 -c -f ;; - *tar.xz) xz -c -z - ;; - *tar.Z) compress -c -f ;; - *tar) cat ;; - *) warning "'%s' is not a valid archive extension." \ - "$PKGEXT"; cat ;; -esac > "${pkg_file}"; ret=$? - -if (( ret )); then - error "Unable to write package to $pkg_dest" - plain " Maybe the disk is full or you do not have write access" + pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" + ret=0 + + # TODO: Maybe this can be set globally for robustness + shopt -s -o pipefail + bsdtar -cf - $comp_files * | + case "$PKGEXT" in + *tar.gz) gzip -c -f -n ;; + *tar.bz2) bzip2 -c -f ;; + *tar.xz) xz -c -z - ;; + *tar.Z) compress -c -f ;; + *tar) cat ;; + *) warning "'%s' is not a valid archive extension." \ + "$PKGEXT"; cat ;; + esac > "${pkg_file}"; ret=$? + + # Move compressed package to destination + if (( ret )); then + error "Unable to write package to $pkg_dest" + plain " Maybe the disk is full or you do not have write access" + rm -rf "$work_dir" + exit 1 + fi + + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi +} -rm -rf "$work_dir" +for PKG in $@; do fakebuild $PKG; done msg "Done." exit 0 -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 512973a..a718a18 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -347,8 +347,29 @@ fakebuild() { rm -rf "$work_dir" } +# Run fakebuild in parralel with at maximum $MAX_JOBS jobs +# By default only run one job +MAX_JOBS=${MAX_JOBS:-1} +# TODO: Use wait -n instead of this hack as soon as it is ok +# to bump the version of the bash dependency +parallelize() { + while [[ $# -gt 0 ]]; do + job_count=($(jobs -p)) + if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then + fakebuild "$1" & + shift + fi + sleep 0.1 + done + wait +} -for PKG in $@; do fakebuild $PKG; done +# Initiate assembly function +if [[ $MAX_JOBS -gt "1" ]]; then + parallelize "$@" +else + for PKG in $@; do fakebuild $PKG; done +fi msg "Done." exit 0 -- 2.9.3
Switch to parseopts instead of merely checking the first argument. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 63 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 22 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a718a18..8656c5c 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -32,6 +32,7 @@ INCLUDE_PACNEW='n' ARGS=("$@") m4_include(../scripts/library/output_format.sh) +m4_include(../scripts/library/parseopts.sh) # Lazy recursive clean up of temporary dirs work_dir_root="${TMPDIR:-/tmp}/bacman" @@ -62,35 +63,51 @@ version() { echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } -# Check for specified arguments -while [[ ! -z $1 ]]; do - if [[ $1 == "--nocolor" ]]; then - USE_COLOR='n' - shift - elif [[ $1 == "--pacnew" ]]; then - INCLUDE_PACNEW='y' - shift - else - break - fi -done - # Configure colored output m4_include(../scripts/library/term_colors.sh) # Break if no argument was given -if (( $# < 1 )); then +if (( $# == 0 )); then + usage + exit 1 +fi + +# Printing the usage information takes precedence over every other parameter +for option in "$@"; do + [[ $option == "-h" || $option == "--help" ]] && usage && exit 0 +done + +# Parse arguments +OPT_SHORT='mv' +OPT_LONG=('nocolor' 'pacnew' 'version') +if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 fi +set -- "${OPTRET[@]}" +unset OPT_SHORT OPT_LONG OPTRET + +while :; do + case "$1" in + -m|--nocolor) + USE_COLOR='n' ;; + --pacnew) + INCLUDE_PACNEW='y' ;; + -v|--version) + version + exit 0 ;; + --) + shift + break 2 ;; + esac + shift +done -# Print usage or version if requested -if [[ $1 = -@(h|-help) ]]; then +# Retrieve the list of packages to be assembled and break if none was specified +pkg_list=($*) +if [[ ${#pkg_list[@]} == 0 ]]; then usage - exit 0 -elif [[ $1 = -@(V|-version) ]]; then - version - exit 0 + exit 1 fi # Run with fake root privileges if EUID is not root @@ -366,9 +383,11 @@ parallelize() { # Initiate assembly function if [[ $MAX_JOBS -gt "1" ]]; then - parallelize "$@" + parallelize "${pkg_list[@]}" else - for PKG in $@; do fakebuild $PKG; done + for PKG in ${pkg_list[@]}; do + fakebuild $PKG + done fi msg "Done." -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Use the same format for the usage section as makepkg. contrib/bacman.sh.in | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 8656c5c..3868750 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -41,25 +41,33 @@ clean_up() { echo exit } - # Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM # Print usage information usage() { - echo "${myname} (pacman) v${myver}" + printf "%s (pacman) %s\n" "$myname" "$myver" + echo + printf -- "$(gettext "Recreate packages using pacman's database and system files")\n" + echo + printf -- "$(gettext "Usage: %s [options] <package(s)>")\n" "$0" echo - echo "Recreate a package using pacman's database and system files" + printf -- "$(gettext "Options:")\n" + printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" + printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo - echo "Example: ${myname} linux-headers" } # Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' + echo 'Copyright (C) 2016 Gordian Edenhofer <gordian.edenhofer@gmail.com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } @@ -123,7 +131,7 @@ if (( EUID )); then fi fi -# Source environmental variables and specify fallbacks +# Read in environmental variables if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 @@ -139,7 +147,7 @@ if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi pkg_dest="${PKGDEST:-$PWD}" -pkg_pkger=${PACKAGER:-'Unknown Packager'} +pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database if [[ ! -d $pac_db ]]; then @@ -339,6 +347,7 @@ fakebuild() { pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 + # Move compressed package to destination # TODO: Maybe this can be set globally for robustness shopt -s -o pipefail bsdtar -cf - $comp_files * | @@ -352,7 +361,7 @@ fakebuild() { "$PKGEXT"; cat ;; esac > "${pkg_file}"; ret=$? - # Move compressed package to destination + # Evaluate return code if (( ret )); then error "Unable to write package to $pkg_dest" plain " Maybe the disk is full or you do not have write access" -- 2.9.3
On 02/09/16 22:51, Gordian Edenhofer wrote:
echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' + echo 'Copyright (C) 2016 Gordian Edenhofer <gordian.edenhofer@gmail.com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>'
You want your name included beyond the "Pacman Development Team". There are currently 11 other contributors to that script beyond the original author (who has their own copyright line). A
On Fri, 2016-09-02 at 23:37 +1000, Allan McRae wrote:
On 02/09/16 22:51, Gordian Edenhofer wrote:
echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' + echo 'Copyright (C) 2016 Gordian Edenhofer <gordian.edenho fer@gmail.com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pac man-dev@archlinux.org>'
You want your name included beyond the "Pacman Development Team". There are currently 11 other contributors to that script beyond the original author (who has their own copyright line).
I did not intended to insult anyone or diminish other contributions. I just though of the patch as a significant revamp considering the patch allows for multiple packages, handles abort signals, added various options and introduced a man page. Since apparently this seems to offend you, I removed my name.
On 03/09/16 01:07, Gordian Edenhofer wrote:
On Fri, 2016-09-02 at 23:37 +1000, Allan McRae wrote:
On 02/09/16 22:51, Gordian Edenhofer wrote:
echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' + echo 'Copyright (C) 2016 Gordian Edenhofer <gordian.edenho fer@gmail.com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pac man-dev@archlinux.org>'
You want your name included beyond the "Pacman Development Team". There are currently 11 other contributors to that script beyond the original author (who has their own copyright line).
I did not intended to insult anyone or diminish other contributions. I just though of the patch as a significant revamp considering the patch allows for multiple packages, handles abort signals, added various options and introduced a man page. Since apparently this seems to offend you, I removed my name.
I wasn't being insulted. My email was more intended to point out the usual process that is followed in this code base. I was probably blunter than intended as I was rushing through patch review late at night. Allan
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 3868750..da5d30d 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -28,6 +28,7 @@ declare -r myname='bacman' declare -r myver='@PACKAGE_VERSION@' USE_COLOR='y' INCLUDE_PACNEW='n' +QUIET=0 # Required for fakeroot because options are shifted off the array. ARGS=("$@") @@ -54,6 +55,7 @@ usage() { echo printf -- "$(gettext "Options:")\n" printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo @@ -86,8 +88,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='mv' -OPT_LONG=('nocolor' 'pacnew' 'version') +OPT_SHORT='qmv' +OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -97,6 +99,8 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -q|--quiet) + QUIET=1 ;; -m|--nocolor) USE_COLOR='n' ;; --pacnew) @@ -181,6 +185,7 @@ fakebuild() { # Assemble list of files which belong to the package and tar them msg2 "Copying package files..." + while read i; do if [[ -z $i ]]; then continue -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index da5d30d..5c59758 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,10 +57,12 @@ usage() { printf -- "$(gettext " -h, --help Show this help message and exit")\n" printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " -o, --out <dir> Write output to specified directory (instead of \$PKGDEST)")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" - printf -- "$(gettext " %s --nocolor --pacnew gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s gzip munge binutils -o ~/Downloads")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew -o /tmp gzip munge binutils")\n" "$myname" printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo } @@ -88,8 +90,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='qmv' -OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT='o:qmv' +OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -99,6 +101,10 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -o|--out) + pkg_dest=$2 + [[ ! -d "$2" ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 + shift ;; -q|--quiet) QUIET=1 ;; -m|--nocolor) @@ -150,7 +156,8 @@ source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi -pkg_dest="${PKGDEST:-$PWD}" +PKGDEST="${PKGDEST:-$PWD}" +pkg_dest="${pkg_dest:-$PKGDEST}" pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 5c59758..58a9fe3 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,12 +57,13 @@ usage() { printf -- "$(gettext " -h, --help Show this help message and exit")\n" printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " -j, --jobs <N> Build in parallel with N jobs - you may want to set XZ_OPT")\n" printf -- "$(gettext " -o, --out <dir> Write output to specified directory (instead of \$PKGDEST)")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" printf -- "$(gettext " %s gzip munge binutils -o ~/Downloads")\n" "$myname" - printf -- "$(gettext " %s --nocolor --pacnew -o /tmp gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew -o /tmp -j 5 gzip munge binutils")\n" "$myname" printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo } @@ -90,8 +91,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='o:qmv' -OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT='o:j:qmv' +OPT_LONG=('out:' 'jobs:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -105,6 +106,14 @@ while :; do pkg_dest=$2 [[ ! -d "$2" ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 shift ;; + -j|--jobs) + if [[ $2 =~ ^-?[0-9]+$ ]]; then + MAX_JOBS=$2 + else + echo -e "\e[39;1m$2\e[0m is not a valid integer!" + exit -1 + fi + shift ;; -q|--quiet) QUIET=1 ;; -m|--nocolor) @@ -191,7 +200,11 @@ fakebuild() { cd "$work_dir" || exit 1 # Assemble list of files which belong to the package and tar them - msg2 "Copying package files..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Copying package files..." + else + msg2 "Copying package files..." + fi while read i; do if [[ -z $i ]]; then @@ -266,7 +279,11 @@ fakebuild() { # Reconstruct .PKGINFO from database # TODO adopt makepkg's write_pkginfo() into this or scripts/library - msg2 "Generating .PKGINFO metadata..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Generating .PKGINFO metadata..." + else + msg2 "Generating .PKGINFO metadata..." + fi echo "# Generated by $myname $myver" > .PKGINFO if [[ $INFAKEROOT == "1" ]]; then echo "# Using $(fakeroot -v)" >> .PKGINFO @@ -354,7 +371,11 @@ fakebuild() { chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null # Generate the package - msg2 "Generating the package..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Generating the package..." + else + msg2 "Generating the package..." + fi pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/doc/.gitignore | 1 + contrib/doc/Makefile.am | 5 +++- contrib/doc/bacman.8.txt | 71 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 1 deletion(-) create mode 100644 contrib/doc/bacman.8.txt diff --git a/contrib/doc/.gitignore b/contrib/doc/.gitignore index c5612bc..3ab2035 100644 --- a/contrib/doc/.gitignore +++ b/contrib/doc/.gitignore @@ -1 +1,2 @@ verify-pacman-repo-db.1 +bacman.8 diff --git a/contrib/doc/Makefile.am b/contrib/doc/Makefile.am index 4c316bb..d5725b1 100644 --- a/contrib/doc/Makefile.am +++ b/contrib/doc/Makefile.am @@ -4,12 +4,14 @@ # man_MANS if --enable-asciidoc and/or --enable-doxygen are used. ASCIIDOC_MANS = \ - verify-pacman-repo-db.1 + verify-pacman-repo-db.1 \ + bacman.8 EXTRA_DIST = \ asciidoc.conf \ footer.txt \ verify-pacman-repo-db.1.txt \ + bacman.8.txt \ $(ASCIIDOC_MANS) # Files that should be removed, but which Automake does not know. @@ -53,5 +55,6 @@ $(ASCIIDOC_MANS): asciidoc.conf footer.txt Makefile.am # Dependency rules verify-pacman-repo-db.1: verify-pacman-repo-db.1.txt +bacman.8: bacman.8.txt # vim:set noet: diff --git a/contrib/doc/bacman.8.txt b/contrib/doc/bacman.8.txt new file mode 100644 index 0000000..2c3112f --- /dev/null +++ b/contrib/doc/bacman.8.txt @@ -0,0 +1,71 @@ +///// +vim:set ts=4 sw=4 syntax=asciidoc noet spell spelllang=en_us: +///// +bacman(8) +========== + +Name +---- +bacman - reassemble installed packages + + +Synopsis +-------- +'bacman' [options] <package(s)> + + +Description +----------- +'bacman' was designed to reassemble installed packages from its deliverd files. +It comes in handy if there is no internet connection available and you have no +access to a up-to-date package cache. Otherwise packages from the Arch Linux +Archive should always be preferred. + +'bacman' fully honors linkman:makepkg.conf[8] and all compression environment +variables, notably ``XZ_OPT''. + + +Options +------- +*-h, \--help*:: + Display usage information. + +*-q, \--quiet*:: + Silence most of the status reporting. + +*\--nocolor*:: + Disable colored output. + +*\--pacnew*:: + Package .pacnew files instead of the concerning files currently in place. + +*-j, \--jobs <jobs>*:: + Assemble multiple packages in parallel. + +*-o, \--out <dir>*:: + Write the assembled package(s) to the specified directory. + + +Examples +-------- +'bacman' linux-headers + +Recreate the package ``linux-headers''. + +'bacman' gzip munge binutils -o ~/Downloads + +Assemble the packages gzip munge binutils and place the packages at +``~/Downloads''. + +'bacman' --nocolor --pacnew -o ~/backup -j 5 $(pacman -Qsq) + +Assemble all currently installed packages using 5 concurrent jobs, use +``.pacnew'' whenever available, suppress colored output and place the desired +packages in ``~/backup''. + + +See Also +-------- +linkman:makepkg[8], linkman:pacman[8] + +include::footer.txt[] -- 2.9.3
On 02/09/16 22:51, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/doc/.gitignore | 1 + contrib/doc/Makefile.am | 5 +++- contrib/doc/bacman.8.txt | 71 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 1 deletion(-) create mode 100644 contrib/doc/bacman.8.txt
diff --git a/contrib/doc/.gitignore b/contrib/doc/.gitignore index c5612bc..3ab2035 100644 --- a/contrib/doc/.gitignore +++ b/contrib/doc/.gitignore @@ -1 +1,2 @@ verify-pacman-repo-db.1 +bacman.8 diff --git a/contrib/doc/Makefile.am b/contrib/doc/Makefile.am index 4c316bb..d5725b1 100644 --- a/contrib/doc/Makefile.am +++ b/contrib/doc/Makefile.am @@ -4,12 +4,14 @@ # man_MANS if --enable-asciidoc and/or --enable-doxygen are used.
ASCIIDOC_MANS = \ - verify-pacman-repo-db.1 + verify-pacman-repo-db.1 \ + bacman.8
EXTRA_DIST = \ asciidoc.conf \ footer.txt \ verify-pacman-repo-db.1.txt \ + bacman.8.txt \ $(ASCIIDOC_MANS)
# Files that should be removed, but which Automake does not know. @@ -53,5 +55,6 @@ $(ASCIIDOC_MANS): asciidoc.conf footer.txt Makefile.am
# Dependency rules verify-pacman-repo-db.1: verify-pacman-repo-db.1.txt +bacman.8: bacman.8.txt
# vim:set noet: diff --git a/contrib/doc/bacman.8.txt b/contrib/doc/bacman.8.txt new file mode 100644 index 0000000..2c3112f --- /dev/null +++ b/contrib/doc/bacman.8.txt @@ -0,0 +1,71 @@ +///// +vim:set ts=4 sw=4 syntax=asciidoc noet spell spelllang=en_us: +///// +bacman(8) +========== + +Name +---- +bacman - reassemble installed packages + + +Synopsis +-------- +'bacman' [options] <package(s)> + + +Description +----------- +'bacman' was designed to reassemble installed packages from its deliverd files. +It comes in handy if there is no internet connection available and you have no +access to a up-to-date package cache. Otherwise packages from the Arch Linux +Archive should always be preferred. +
pacman is not Arch Linux specific. Remove any references.
+'bacman' fully honors linkman:makepkg.conf[8] and all compression environment +variables, notably ``XZ_OPT''. + + +Options +------- +*-h, \--help*:: + Display usage information. + +*-q, \--quiet*:: + Silence most of the status reporting. + +*\--nocolor*:: + Disable colored output. + +*\--pacnew*:: + Package .pacnew files instead of the concerning files currently in place. + +*-j, \--jobs <jobs>*:: + Assemble multiple packages in parallel. + +*-o, \--out <dir>*:: + Write the assembled package(s) to the specified directory. + + +Examples +-------- +'bacman' linux-headers + +Recreate the package ``linux-headers''. + +'bacman' gzip munge binutils -o ~/Downloads + +Assemble the packages gzip munge binutils and place the packages at +``~/Downloads''. + +'bacman' --nocolor --pacnew -o ~/backup -j 5 $(pacman -Qsq) + +Assemble all currently installed packages using 5 concurrent jobs, use +``.pacnew'' whenever available, suppress colored output and place the desired +packages in ``~/backup''. +
Parallel patch is not being accepted (as stated multiple times). Remove references to it.
+ +See Also +-------- +linkman:makepkg[8], linkman:pacman[8] + +include::footer.txt[]
To enable the creation of multiple packages with one command move the assembly process into its own function. Handle SIGHUP, SIGINT, SIGTERM and remove working directories accordingly. Add some comments. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 440 +++++++++++++++++++++++++-------------------------- 1 file changed, 219 insertions(+), 221 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a611c1a..512973a 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -33,9 +33,18 @@ ARGS=("$@") m4_include(../scripts/library/output_format.sh) -# -# User Friendliness -# +# Lazy recursive clean up of temporary dirs +work_dir_root="${TMPDIR:-/tmp}/bacman" +clean_up() { + rm -rf "$work_dir_root".* + echo + exit +} + +# Trap termination signals +trap clean_up SIGHUP SIGINT SIGTERM + +# Print usage information usage() { echo "${myname} (pacman) v${myver}" echo @@ -46,12 +55,14 @@ usage() { echo "Example: ${myname} linux-headers" } +# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } +# Check for specified arguments while [[ ! -z $1 ]]; do if [[ $1 == "--nocolor" ]]; then USE_COLOR='n' @@ -64,13 +75,16 @@ while [[ ! -z $1 ]]; do fi done +# Configure colored output m4_include(../scripts/library/term_colors.sh) -if (( $# != 1 )); then +# Break if no argument was given +if (( $# < 1 )); then usage exit 1 fi +# Print usage or version if requested if [[ $1 = -@(h|-help) ]]; then usage exit 0 @@ -79,9 +93,7 @@ elif [[ $1 = -@(V|-version) ]]; then exit 0 fi -# -# Fakeroot support -# +# Run with fake root privileges if EUID is not root if (( EUID )); then if [[ -f /usr/bin/fakeroot ]]; then msg "Entering fakeroot environment" @@ -94,263 +106,249 @@ if (( EUID )); then fi fi -# -# Setting environmental variables -# +# Source environmental variables and specify fallbacks if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 fi - eval $(awk '/DBPath/ {print $1$2$3}' @sysconfdir@/pacman.conf) pac_db="${DBPath:-@localstatedir@/lib/pacman/}/local" - if [[ ! -r @sysconfdir@/makepkg.conf ]]; then error "unable to read @sysconfdir@/makepkg.conf" exit 1 fi - source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi - pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -pkg_name="$1" -pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) -pkg_namver=("${pkg_dir[@]##*/}") - -# -# Checks everything is in place -# +# Check for an existing database if [[ ! -d $pac_db ]]; then error "pacman database directory ${pac_db} not found" exit 1 fi -if (( ${#pkg_dir[@]} != 1 )); then - error "%d entries for package %s found in pacman database" \ - ${#pkg_dir[@]} "${pkg_name}" - msg2 "%s" "${pkg_dir[@]}" - exit 1 -fi - -if [[ ! -d $pkg_dir ]]; then - error "package %s is found in pacman database," "${pkg_name}" - plain " but '%s' is not a directory" "${pkg_dir}" - exit 1 -fi - -# -# Begin -# -msg "Package: ${pkg_namver}" -work_dir=$(mktemp -d "${TMPDIR:-/tmp}/bacman.XXXXXXXXXX") -cd "$work_dir" || exit 1 - -# -# File copying -# -msg2 "Copying package files..." - -while read i; do - if [[ -z $i ]]; then - continue +# Assemble a single package: $1 = pkgname +fakebuild() { + pkg_name="$1" + pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) + pkg_namver=("${pkg_dir[@]##*/}") + + # Checks database for specified package + if (( ${#pkg_dir[@]} != 1 )); then + error "%d entries for package %s found in pacman database" \ + ${#pkg_dir[@]} "${pkg_name}" + msg2 "%s" "${pkg_dir[@]}" + exit 1 fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + if [[ ! -d $pkg_dir ]]; then + error "package %s is found in pacman database," "${pkg_name}" + plain " but '%s' is not a directory" "${pkg_dir}" + exit 1 fi - case "$current" in - %FILES%) - local_file="/$i" - package_file="$work_dir/$i" + # Create working directory + msg "Package: ${pkg_namver}" + work_dir=$(mktemp -d "${work_dir_root}.XXXXXXXXXX") + cd "$work_dir" || exit 1 - if [[ ! -e $local_file ]]; then - warning "package file $local_file is missing" - continue - fi - ;; - - %BACKUP%) - # Get the MD5 checksum. - original_md5="${i##*$'\t'}" - # Strip the md5sum after the tab. - i="${i%$'\t'*}" - local_file="/$i.pacnew" - package_file="$work_dir/$i" - - # Include unmodified .pacnew files. - local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" - if [[ $INCLUDE_PACNEW == 'n' ]] \ - || [[ ! -e $local_file ]] \ - || [[ $local_md5 != $original_md5 ]]; then - # Warn about modified files. - local_md5="$(md5sum "/$i" | cut -d' ' -f1)" - if [[ $local_md5 != $original_md5 ]]; then - warning "package file /$i has been modified" - fi - # Let the normal file be included in the %FILES% list. - continue - fi - ;; + # Assemble list of files which belong to the package and tar them + msg2 "Copying package files..." + while read i; do + if [[ -z $i ]]; then + continue + fi - *) + if [[ $i == %+([A-Z])% ]]; then + current=$i continue - ;; - esac + fi - ret=0 - bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + case "$current" in + %FILES%) + local_file="/$i" + package_file="$work_dir/$i" - # Workaround to bsdtar not reporting a missing file as an error - if ! [[ -e $package_file || -L $package_file ]]; then - error "unable to add $local_file to the package" - plain " If your user does not have permission to read this file, then" - plain " you will need to run $myname as root." + if [[ ! -e $local_file ]]; then + warning "package file $local_file is missing" + continue + fi + ;; + + %BACKUP%) + # Get the MD5 checksum. + original_md5="${i##*$'\t'}" + # Strip the md5sum after the tab. + i="${i%$'\t'*}" + local_file="/$i.pacnew" + package_file="$work_dir/$i" + + # Include unmodified .pacnew files. + local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" + if [[ $INCLUDE_PACNEW == 'n' ]] \ + || [[ ! -e $local_file ]] \ + || [[ $local_md5 != $original_md5 ]]; then + # Warn about modified files. + local_md5="$(md5sum "/$i" | cut -d' ' -f1)" + if [[ $local_md5 != $original_md5 ]]; then + warning "package file /$i has been modified" + fi + # Let the normal file be included in the %FILES% list. + continue + fi + ;; + + *) + continue + ;; + esac + + # Tar files + ret=0 + bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + # Workaround to bsdtar not reporting a missing file as an error + if ! [[ -e $package_file || -L $package_file ]]; then + error "unable to add $local_file to the package" + plain " If your user does not have permission to read this file, then" + plain " you will need to run $myname as root." + rm -rf "$work_dir" + exit 1 + fi + done < "$pkg_dir"/files + + ret=$? + if (( ret )); then rm -rf "$work_dir" exit 1 fi -done < "$pkg_dir"/files -ret=$? -if (( ret )); then - rm -rf "$work_dir" - exit 1 -fi - -pkg_size=$(du -sk | awk '{print $1 * 1024}') - -# -# .PKGINFO stuff -# TODO adopt makepkg's write_pkginfo() into this or scripts/library -# -msg2 "Generating .PKGINFO metadata..." -echo "# Generated by $myname $myver" > .PKGINFO -if [[ $INFAKEROOT == "1" ]]; then - echo "# Using $(fakeroot -v)" >> .PKGINFO -fi -echo "# $(LC_ALL=C date)" >> .PKGINFO -echo "#" >> .PKGINFO + # Calculate package size + pkg_size=$(du -sk | awk '{print $1 * 1024}') -while read i; do - if [[ -z $i ]]; then - continue; + # Reconstruct .PKGINFO from database + # TODO adopt makepkg's write_pkginfo() into this or scripts/library + msg2 "Generating .PKGINFO metadata..." + echo "# Generated by $myname $myver" > .PKGINFO + if [[ $INFAKEROOT == "1" ]]; then + echo "# Using $(fakeroot -v)" >> .PKGINFO fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + echo "# $(LC_ALL=C date)" >> .PKGINFO + echo "#" >> .PKGINFO + while read i; do + if [[ -z $i ]]; then + continue; + fi + if [[ $i == %+([A-Z])% ]]; then + current=$i + continue + fi + + case "$current" in + # desc + %NAME%) + echo "pkgname = $i" >> .PKGINFO + ;; + %VERSION%) + echo "pkgver = $i" >> .PKGINFO + ;; + %DESC%) + echo "pkgdesc = $i" >> .PKGINFO + ;; + %URL%) + echo "url = $i" >> .PKGINFO + ;; + %LICENSE%) + echo "license = $i" >> .PKGINFO + ;; + %ARCH%) + echo "arch = $i" >> .PKGINFO + pkg_arch="$i" + ;; + %BUILDDATE%) + echo "builddate = $(date -u "+%s")" >> .PKGINFO + ;; + %PACKAGER%) + echo "packager = $pkg_pkger" >> .PKGINFO + ;; + %SIZE%) + echo "size = $pkg_size" >> .PKGINFO + ;; + %GROUPS%) + echo "group = $i" >> .PKGINFO + ;; + %REPLACES%) + echo "replaces = $i" >> .PKGINFO + ;; + %DEPENDS%) + echo "depend = $i" >> .PKGINFO + ;; + %OPTDEPENDS%) + echo "optdepend = $i" >> .PKGINFO + ;; + %CONFLICTS%) + echo "conflict = $i" >> .PKGINFO + ;; + %PROVIDES%) + echo "provides = $i" >> .PKGINFO + ;; + %BACKUP%) + # Strip the md5sum after the tab + echo "backup = ${i%%$'\t'*}" >> .PKGINFO + ;; + esac + done < <(cat "$pkg_dir"/{desc,files}) + + comp_files=".PKGINFO" + + # Add instal file if present + if [[ -f $pkg_dir/install ]]; then + cp "$pkg_dir/install" "$work_dir/.INSTALL" + comp_files+=" .INSTALL" + fi + if [[ -f $pkg_dir/changelog ]]; then + cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" + comp_files+=" .CHANGELOG" fi - case "$current" in - # desc - %NAME%) - echo "pkgname = $i" >> .PKGINFO - ;; - %VERSION%) - echo "pkgver = $i" >> .PKGINFO - ;; - %DESC%) - echo "pkgdesc = $i" >> .PKGINFO - ;; - %URL%) - echo "url = $i" >> .PKGINFO - ;; - %LICENSE%) - echo "license = $i" >> .PKGINFO - ;; - %ARCH%) - echo "arch = $i" >> .PKGINFO - pkg_arch="$i" - ;; - %BUILDDATE%) - echo "builddate = $(date -u "+%s")" >> .PKGINFO - ;; - %PACKAGER%) - echo "packager = $pkg_pkger" >> .PKGINFO - ;; - %SIZE%) - echo "size = $pkg_size" >> .PKGINFO - ;; - %GROUPS%) - echo "group = $i" >> .PKGINFO - ;; - %REPLACES%) - echo "replaces = $i" >> .PKGINFO - ;; - %DEPENDS%) - echo "depend = $i" >> .PKGINFO - ;; - %OPTDEPENDS%) - echo "optdepend = $i" >> .PKGINFO - ;; - %CONFLICTS%) - echo "conflict = $i" >> .PKGINFO - ;; - %PROVIDES%) - echo "provides = $i" >> .PKGINFO - ;; - - # files - %BACKUP%) - # Strip the md5sum after the tab - echo "backup = ${i%%$'\t'*}" >> .PKGINFO - ;; - esac -done < <(cat "$pkg_dir"/{desc,files}) - -comp_files=".PKGINFO" - -if [[ -f $pkg_dir/install ]]; then - cp "$pkg_dir/install" "$work_dir/.INSTALL" - comp_files+=" .INSTALL" -fi -if [[ -f $pkg_dir/changelog ]]; then - cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" - comp_files+=" .CHANGELOG" -fi + # Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL + chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -# -# Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL -# -chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + # Generate the package + msg2 "Generating the package..." -# -# Generate the package -# -msg2 "Generating the package..." - -pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" -ret=0 - -# TODO: Maybe this can be set globally for robustness -shopt -s -o pipefail -bsdtar -cf - $comp_files * | -case "$PKGEXT" in - *tar.gz) gzip -c -f -n ;; - *tar.bz2) bzip2 -c -f ;; - *tar.xz) xz -c -z - ;; - *tar.Z) compress -c -f ;; - *tar) cat ;; - *) warning "'%s' is not a valid archive extension." \ - "$PKGEXT"; cat ;; -esac > "${pkg_file}"; ret=$? - -if (( ret )); then - error "Unable to write package to $pkg_dest" - plain " Maybe the disk is full or you do not have write access" + pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" + ret=0 + + # TODO: Maybe this can be set globally for robustness + shopt -s -o pipefail + bsdtar -cf - $comp_files * | + case "$PKGEXT" in + *tar.gz) gzip -c -f -n ;; + *tar.bz2) bzip2 -c -f ;; + *tar.xz) xz -c -z - ;; + *tar.Z) compress -c -f ;; + *tar) cat ;; + *) warning "'%s' is not a valid archive extension." \ + "$PKGEXT"; cat ;; + esac > "${pkg_file}"; ret=$? + + # Move compressed package to destination + if (( ret )); then + error "Unable to write package to $pkg_dest" + plain " Maybe the disk is full or you do not have write access" + rm -rf "$work_dir" + exit 1 + fi + + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi +} -rm -rf "$work_dir" +for PKG in $@; do fakebuild $PKG; done msg "Done." exit 0 -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 512973a..a718a18 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -347,8 +347,29 @@ fakebuild() { rm -rf "$work_dir" } +# Run fakebuild in parralel with at maximum $MAX_JOBS jobs +# By default only run one job +MAX_JOBS=${MAX_JOBS:-1} +# TODO: Use wait -n instead of this hack as soon as it is ok +# to bump the version of the bash dependency +parallelize() { + while [[ $# -gt 0 ]]; do + job_count=($(jobs -p)) + if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then + fakebuild "$1" & + shift + fi + sleep 0.1 + done + wait +} -for PKG in $@; do fakebuild $PKG; done +# Initiate assembly function +if [[ $MAX_JOBS -gt "1" ]]; then + parallelize "$@" +else + for PKG in $@; do fakebuild $PKG; done +fi msg "Done." exit 0 -- 2.9.3
On 03/09/16 00:50, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> ---
To be completely explicit, I am not going to accept parallel packaging patches (2, 7, and 9 in patchset v6). There are many other tools available to handle running scripts in parallel, and the boilerplate overhead in bacman is minimal. This is not a feature that we should be providing. Allan
Switch to parseopts instead of merely checking the first argument. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 63 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 22 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a718a18..8656c5c 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -32,6 +32,7 @@ INCLUDE_PACNEW='n' ARGS=("$@") m4_include(../scripts/library/output_format.sh) +m4_include(../scripts/library/parseopts.sh) # Lazy recursive clean up of temporary dirs work_dir_root="${TMPDIR:-/tmp}/bacman" @@ -62,35 +63,51 @@ version() { echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } -# Check for specified arguments -while [[ ! -z $1 ]]; do - if [[ $1 == "--nocolor" ]]; then - USE_COLOR='n' - shift - elif [[ $1 == "--pacnew" ]]; then - INCLUDE_PACNEW='y' - shift - else - break - fi -done - # Configure colored output m4_include(../scripts/library/term_colors.sh) # Break if no argument was given -if (( $# < 1 )); then +if (( $# == 0 )); then + usage + exit 1 +fi + +# Printing the usage information takes precedence over every other parameter +for option in "$@"; do + [[ $option == "-h" || $option == "--help" ]] && usage && exit 0 +done + +# Parse arguments +OPT_SHORT='mv' +OPT_LONG=('nocolor' 'pacnew' 'version') +if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 fi +set -- "${OPTRET[@]}" +unset OPT_SHORT OPT_LONG OPTRET + +while :; do + case "$1" in + -m|--nocolor) + USE_COLOR='n' ;; + --pacnew) + INCLUDE_PACNEW='y' ;; + -v|--version) + version + exit 0 ;; + --) + shift + break 2 ;; + esac + shift +done -# Print usage or version if requested -if [[ $1 = -@(h|-help) ]]; then +# Retrieve the list of packages to be assembled and break if none was specified +pkg_list=($*) +if [[ ${#pkg_list[@]} == 0 ]]; then usage - exit 0 -elif [[ $1 = -@(V|-version) ]]; then - version - exit 0 + exit 1 fi # Run with fake root privileges if EUID is not root @@ -366,9 +383,11 @@ parallelize() { # Initiate assembly function if [[ $MAX_JOBS -gt "1" ]]; then - parallelize "$@" + parallelize "${pkg_list[@]}" else - for PKG in $@; do fakebuild $PKG; done + for PKG in ${pkg_list[@]}; do + fakebuild $PKG + done fi msg "Done." -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 8656c5c..f5807b5 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -41,19 +41,26 @@ clean_up() { echo exit } - # Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM # Print usage information usage() { - echo "${myname} (pacman) v${myver}" + printf "%s (pacman) %s\n" "$myname" "$myver" + echo + printf -- "$(gettext "Recreate packages using pacman's database and system files")\n" + echo + printf -- "$(gettext "Usage: %s [options] <package(s)>")\n" "$0" echo - echo "Recreate a package using pacman's database and system files" + printf -- "$(gettext "Options:")\n" + printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" + printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo - echo "Example: ${myname} linux-headers" } # Print version information @@ -123,7 +130,7 @@ if (( EUID )); then fi fi -# Source environmental variables and specify fallbacks +# Read in environmental variables if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 @@ -139,7 +146,7 @@ if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi pkg_dest="${PKGDEST:-$PWD}" -pkg_pkger=${PACKAGER:-'Unknown Packager'} +pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database if [[ ! -d $pac_db ]]; then @@ -339,6 +346,7 @@ fakebuild() { pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 + # Move compressed package to destination # TODO: Maybe this can be set globally for robustness shopt -s -o pipefail bsdtar -cf - $comp_files * | @@ -352,7 +360,7 @@ fakebuild() { "$PKGEXT"; cat ;; esac > "${pkg_file}"; ret=$? - # Move compressed package to destination + # Evaluate return code if (( ret )); then error "Unable to write package to $pkg_dest" plain " Maybe the disk is full or you do not have write access" -- 2.9.3
On 03/09/16 00:50, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 8656c5c..f5807b5 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -41,19 +41,26 @@ clean_up() { echo exit } -
Unrelated change
# Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM
# Print usage information usage() { - echo "${myname} (pacman) v${myver}" + printf "%s (pacman) %s\n" "$myname" "$myver" + echo + printf -- "$(gettext "Recreate packages using pacman's database and system files")\n" + echo + printf -- "$(gettext "Usage: %s [options] <package(s)>")\n" "$0" echo - echo "Recreate a package using pacman's database and system files" + printf -- "$(gettext "Options:")\n" + printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" + printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo - echo "Example: ${myname} linux-headers" }
# Print version information @@ -123,7 +130,7 @@ if (( EUID )); then fi fi
-# Source environmental variables and specify fallbacks +# Read in environmental variables
Unrelated change
if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 @@ -139,7 +146,7 @@ if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi pkg_dest="${PKGDEST:-$PWD}" -pkg_pkger=${PACKAGER:-'Unknown Packager'} +pkg_pkger="${PACKAGER:-'Unknown Packager'}"
Unrelated change
# Check for an existing database if [[ ! -d $pac_db ]]; then @@ -339,6 +346,7 @@ fakebuild() { pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0
+ # Move compressed package to destination
Unrelated change
# TODO: Maybe this can be set globally for robustness shopt -s -o pipefail bsdtar -cf - $comp_files * | @@ -352,7 +360,7 @@ fakebuild() { "$PKGEXT"; cat ;; esac > "${pkg_file}"; ret=$?
- # Move compressed package to destination + # Evaluate return code
Unrelated change
if (( ret )); then error "Unable to write package to $pkg_dest" plain " Maybe the disk is full or you do not have write access"
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index f5807b5..a567e7d 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -28,6 +28,7 @@ declare -r myname='bacman' declare -r myver='@PACKAGE_VERSION@' USE_COLOR='y' INCLUDE_PACNEW='n' +QUIET=0 # Required for fakeroot because options are shifted off the array. ARGS=("$@") @@ -54,6 +55,7 @@ usage() { echo printf -- "$(gettext "Options:")\n" printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo @@ -85,8 +87,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='mv' -OPT_LONG=('nocolor' 'pacnew' 'version') +OPT_SHORT='qmv' +OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -96,6 +98,8 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -q|--quiet) + QUIET=1 ;; -m|--nocolor) USE_COLOR='n' ;; --pacnew) @@ -180,6 +184,7 @@ fakebuild() { # Assemble list of files which belong to the package and tar them msg2 "Copying package files..." + while read i; do if [[ -z $i ]]; then continue -- 2.9.3
On 03/09/16 00:50, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index f5807b5..a567e7d 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -28,6 +28,7 @@ declare -r myname='bacman' declare -r myver='@PACKAGE_VERSION@' USE_COLOR='y' INCLUDE_PACNEW='n' +QUIET=0 # Required for fakeroot because options are shifted off the array. ARGS=("$@")
@@ -54,6 +55,7 @@ usage() { echo printf -- "$(gettext "Options:")\n" printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo @@ -85,8 +87,8 @@ for option in "$@"; do done
# Parse arguments -OPT_SHORT='mv' -OPT_LONG=('nocolor' 'pacnew' 'version') +OPT_SHORT='qmv' +OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -96,6 +98,8 @@ unset OPT_SHORT OPT_LONG OPTRET
while :; do case "$1" in + -q|--quiet) + QUIET=1 ;; -m|--nocolor) USE_COLOR='n' ;; --pacnew) @@ -180,6 +184,7 @@ fakebuild() {
# Assemble list of files which belong to the package and tar them msg2 "Copying package files..." +
Unrelated change
while read i; do if [[ -z $i ]]; then continue
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a567e7d..b636cb6 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,10 +57,12 @@ usage() { printf -- "$(gettext " -h, --help Show this help message and exit")\n" printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " -o, --out <dir> Write output to specified directory (instead of \$PKGDEST)")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" - printf -- "$(gettext " %s --nocolor --pacnew gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s gzip munge binutils -o ~/Downloads")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew -o /tmp gzip munge binutils")\n" "$myname" printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo } @@ -87,8 +89,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='qmv' -OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT='o:qmv' +OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -98,6 +100,10 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -o|--out) + pkg_dest=$2 + [[ ! -d "$2" ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 + shift ;; -q|--quiet) QUIET=1 ;; -m|--nocolor) @@ -149,7 +155,8 @@ source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi -pkg_dest="${PKGDEST:-$PWD}" +PKGDEST="${PKGDEST:-$PWD}" +pkg_dest="${pkg_dest:-$PKGDEST}" pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index b636cb6..120bece 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,12 +57,13 @@ usage() { printf -- "$(gettext " -h, --help Show this help message and exit")\n" printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " -j, --jobs <N> Build in parallel with N jobs - you may want to set XZ_OPT")\n" printf -- "$(gettext " -o, --out <dir> Write output to specified directory (instead of \$PKGDEST)")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" printf -- "$(gettext " %s gzip munge binutils -o ~/Downloads")\n" "$myname" - printf -- "$(gettext " %s --nocolor --pacnew -o /tmp gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew -o /tmp -j 5 gzip munge binutils")\n" "$myname" printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo } @@ -89,8 +90,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='o:qmv' -OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT='o:j:qmv' +OPT_LONG=('out:' 'jobs:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -104,6 +105,14 @@ while :; do pkg_dest=$2 [[ ! -d "$2" ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 shift ;; + -j|--jobs) + if [[ $2 =~ ^-?[0-9]+$ ]]; then + MAX_JOBS=$2 + else + echo -e "\e[39;1m$2\e[0m is not a valid integer!" + exit -1 + fi + shift ;; -q|--quiet) QUIET=1 ;; -m|--nocolor) @@ -190,7 +199,11 @@ fakebuild() { cd "$work_dir" || exit 1 # Assemble list of files which belong to the package and tar them - msg2 "Copying package files..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Copying package files..." + else + msg2 "Copying package files..." + fi while read i; do if [[ -z $i ]]; then @@ -265,7 +278,11 @@ fakebuild() { # Reconstruct .PKGINFO from database # TODO adopt makepkg's write_pkginfo() into this or scripts/library - msg2 "Generating .PKGINFO metadata..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Generating .PKGINFO metadata..." + else + msg2 "Generating .PKGINFO metadata..." + fi echo "# Generated by $myname $myver" > .PKGINFO if [[ $INFAKEROOT == "1" ]]; then echo "# Using $(fakeroot -v)" >> .PKGINFO @@ -353,7 +370,11 @@ fakebuild() { chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null # Generate the package - msg2 "Generating the package..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Generating the package..." + else + msg2 "Generating the package..." + fi pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Remove a reference to the Arch Linux Archive. Reorder some options according to the usage function. Add -m the abbreviated form of --nocolor. contrib/doc/.gitignore | 1 + contrib/doc/Makefile.am | 5 +++- contrib/doc/bacman.8.txt | 66 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 71 insertions(+), 1 deletion(-) create mode 100644 contrib/doc/bacman.8.txt diff --git a/contrib/doc/.gitignore b/contrib/doc/.gitignore index c5612bc..3ab2035 100644 --- a/contrib/doc/.gitignore +++ b/contrib/doc/.gitignore @@ -1 +1,2 @@ verify-pacman-repo-db.1 +bacman.8 diff --git a/contrib/doc/Makefile.am b/contrib/doc/Makefile.am index 4c316bb..d5725b1 100644 --- a/contrib/doc/Makefile.am +++ b/contrib/doc/Makefile.am @@ -4,12 +4,14 @@ # man_MANS if --enable-asciidoc and/or --enable-doxygen are used. ASCIIDOC_MANS = \ - verify-pacman-repo-db.1 + verify-pacman-repo-db.1 \ + bacman.8 EXTRA_DIST = \ asciidoc.conf \ footer.txt \ verify-pacman-repo-db.1.txt \ + bacman.8.txt \ $(ASCIIDOC_MANS) # Files that should be removed, but which Automake does not know. @@ -53,5 +55,6 @@ $(ASCIIDOC_MANS): asciidoc.conf footer.txt Makefile.am # Dependency rules verify-pacman-repo-db.1: verify-pacman-repo-db.1.txt +bacman.8: bacman.8.txt # vim:set noet: diff --git a/contrib/doc/bacman.8.txt b/contrib/doc/bacman.8.txt new file mode 100644 index 0000000..a9d7ba9 --- /dev/null +++ b/contrib/doc/bacman.8.txt @@ -0,0 +1,66 @@ +///// +vim:set ts=4 sw=4 syntax=asciidoc noet spell spelllang=en_us: +///// +bacman(8) +========== + +Name +---- +bacman - recreate installed packages + + +Synopsis +-------- +'bacman' [options] <package(s)> + + +Description +----------- +'bacman' was designed to reassemble installed packages from its deliverd files. +It comes in handy if there is no internet connection available and you have no +access to a up-to-date package cache. + +'bacman' fully honors linkman:makepkg.conf[8] and all compression environment +variables, notably ``XZ_OPT''. + + +Options +------- +*-h, \--help*:: + Display usage information. + +*-q, \--quiet*:: + Silence most of the status reporting. + +*-m, \--nocolor*:: + Disable colored output. + +*-o, \--out <dir>*:: + Write the assembled package(s) to the specified directory. + +*\--pacnew*:: + Package .pacnew files instead of the concerning files currently in place. + + +Examples +-------- +'bacman' linux-headers + +Recreate the package ``linux-headers''. + +'bacman' gzip munge binutils -o ~/Downloads + +Assemble the packages gzip munge binutils and place the packages at +``~/Downloads''. + +'bacman' --nocolor --pacnew -o ~/backup $(pacman -Qsq) + +Assemble all currently installed packages using ``.pacnew'' whenever available, +suppress colored output and place the desired packages in ``~/backup''. + + +See Also +-------- +linkman:makepkg[8], linkman:pacman[8] + +include::footer.txt[] -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Move the description of the jobs option into its own commit. contrib/doc/bacman.8.txt | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/contrib/doc/bacman.8.txt b/contrib/doc/bacman.8.txt index a9d7ba9..4738189 100644 --- a/contrib/doc/bacman.8.txt +++ b/contrib/doc/bacman.8.txt @@ -35,6 +35,9 @@ Options *-m, \--nocolor*:: Disable colored output. +*-j, \--jobs <jobs>*:: + Assemble multiple packages in parallel. + *-o, \--out <dir>*:: Write the assembled package(s) to the specified directory. @@ -53,10 +56,11 @@ Recreate the package ``linux-headers''. Assemble the packages gzip munge binutils and place the packages at ``~/Downloads''. -'bacman' --nocolor --pacnew -o ~/backup $(pacman -Qsq) +'bacman' --nocolor --pacnew -o ~/backup -j 5 $(pacman -Qsq) -Assemble all currently installed packages using ``.pacnew'' whenever available, -suppress colored output and place the desired packages in ``~/backup''. +Assemble all currently installed packages using 5 concurrent jobs, use +``.pacnew'' whenever available, suppress colored output and place the desired +packages in ``~/backup''. See Also -- 2.9.3
To enable the creation of multiple packages with one command move the assembly process into its own function. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Split up first commit into three distinct parts: Moving stuff into function, handle abort signals, write and alter some comments. Merge the altered comments into the code structuring commit. contrib/bacman.sh.in | 405 +++++++++++++++++++++++++-------------------------- 1 file changed, 200 insertions(+), 205 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a611c1a..5e6fce5 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -66,7 +66,8 @@ done m4_include(../scripts/library/term_colors.sh) -if (( $# != 1 )); then +# Break if no argument was given +if (( $# < 1 )); then usage exit 1 fi @@ -118,10 +119,6 @@ fi pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -pkg_name="$1" -pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) -pkg_namver=("${pkg_dir[@]##*/}") - # # Checks everything is in place # @@ -130,227 +127,225 @@ if [[ ! -d $pac_db ]]; then exit 1 fi -if (( ${#pkg_dir[@]} != 1 )); then - error "%d entries for package %s found in pacman database" \ - ${#pkg_dir[@]} "${pkg_name}" - msg2 "%s" "${pkg_dir[@]}" - exit 1 -fi - -if [[ ! -d $pkg_dir ]]; then - error "package %s is found in pacman database," "${pkg_name}" - plain " but '%s' is not a directory" "${pkg_dir}" - exit 1 -fi - -# -# Begin -# -msg "Package: ${pkg_namver}" -work_dir=$(mktemp -d "${TMPDIR:-/tmp}/bacman.XXXXXXXXXX") -cd "$work_dir" || exit 1 - -# -# File copying -# -msg2 "Copying package files..." - -while read i; do - if [[ -z $i ]]; then - continue +# Assemble a single package: $1 = pkgname +fakebuild() { + pkg_name="$1" + pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) + pkg_namver=("${pkg_dir[@]##*/}") + + # Checks database for specified package + if (( ${#pkg_dir[@]} != 1 )); then + error "%d entries for package %s found in pacman database" \ + ${#pkg_dir[@]} "${pkg_name}" + msg2 "%s" "${pkg_dir[@]}" + exit 1 fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + if [[ ! -d $pkg_dir ]]; then + error "package %s is found in pacman database," "${pkg_name}" + plain " but '%s' is not a directory" "${pkg_dir}" + exit 1 fi - case "$current" in - %FILES%) - local_file="/$i" - package_file="$work_dir/$i" + # Create working directory + msg "Package: ${pkg_namver}" + work_dir=$(mktemp -d "${work_dir_root}.XXXXXXXXXX") + cd "$work_dir" || exit 1 - if [[ ! -e $local_file ]]; then - warning "package file $local_file is missing" - continue - fi - ;; - - %BACKUP%) - # Get the MD5 checksum. - original_md5="${i##*$'\t'}" - # Strip the md5sum after the tab. - i="${i%$'\t'*}" - local_file="/$i.pacnew" - package_file="$work_dir/$i" - - # Include unmodified .pacnew files. - local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" - if [[ $INCLUDE_PACNEW == 'n' ]] \ - || [[ ! -e $local_file ]] \ - || [[ $local_md5 != $original_md5 ]]; then - # Warn about modified files. - local_md5="$(md5sum "/$i" | cut -d' ' -f1)" - if [[ $local_md5 != $original_md5 ]]; then - warning "package file /$i has been modified" - fi - # Let the normal file be included in the %FILES% list. - continue - fi - ;; + # Assemble list of files which belong to the package and tar them + msg2 "Copying package files..." + while read i; do + if [[ -z $i ]]; then + continue + fi - *) + if [[ $i == %+([A-Z])% ]]; then + current=$i continue - ;; - esac + fi - ret=0 - bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + case "$current" in + %FILES%) + local_file="/$i" + package_file="$work_dir/$i" + + if [[ ! -e $local_file ]]; then + warning "package file $local_file is missing" + continue + fi + ;; + + %BACKUP%) + # Get the MD5 checksum. + original_md5="${i##*$'\t'}" + # Strip the md5sum after the tab. + i="${i%$'\t'*}" + local_file="/$i.pacnew" + package_file="$work_dir/$i" + + # Include unmodified .pacnew files. + local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" + if [[ $INCLUDE_PACNEW == 'n' ]] \ + || [[ ! -e $local_file ]] \ + || [[ $local_md5 != $original_md5 ]]; then + # Warn about modified files. + local_md5="$(md5sum "/$i" | cut -d' ' -f1)" + if [[ $local_md5 != $original_md5 ]]; then + warning "package file /$i has been modified" + fi + # Let the normal file be included in the %FILES% list. + continue + fi + ;; - # Workaround to bsdtar not reporting a missing file as an error - if ! [[ -e $package_file || -L $package_file ]]; then - error "unable to add $local_file to the package" - plain " If your user does not have permission to read this file, then" - plain " you will need to run $myname as root." + *) + continue + ;; + esac + + # Tar files + ret=0 + bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + # Workaround to bsdtar not reporting a missing file as an error + if ! [[ -e $package_file || -L $package_file ]]; then + error "unable to add $local_file to the package" + plain " If your user does not have permission to read this file, then" + plain " you will need to run $myname as root." + rm -rf "$work_dir" + exit 1 + fi + done < "$pkg_dir"/files + + ret=$? + if (( ret )); then rm -rf "$work_dir" exit 1 fi -done < "$pkg_dir"/files -ret=$? -if (( ret )); then - rm -rf "$work_dir" - exit 1 -fi - -pkg_size=$(du -sk | awk '{print $1 * 1024}') - -# -# .PKGINFO stuff -# TODO adopt makepkg's write_pkginfo() into this or scripts/library -# -msg2 "Generating .PKGINFO metadata..." -echo "# Generated by $myname $myver" > .PKGINFO -if [[ $INFAKEROOT == "1" ]]; then - echo "# Using $(fakeroot -v)" >> .PKGINFO -fi -echo "# $(LC_ALL=C date)" >> .PKGINFO -echo "#" >> .PKGINFO + # Calculate package size + pkg_size=$(du -sk | awk '{print $1 * 1024}') -while read i; do - if [[ -z $i ]]; then - continue; + # Reconstruct .PKGINFO from database + # TODO adopt makepkg's write_pkginfo() into this or scripts/library + msg2 "Generating .PKGINFO metadata..." + echo "# Generated by $myname $myver" > .PKGINFO + if [[ $INFAKEROOT == "1" ]]; then + echo "# Using $(fakeroot -v)" >> .PKGINFO fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + echo "# $(LC_ALL=C date)" >> .PKGINFO + echo "#" >> .PKGINFO + while read i; do + if [[ -z $i ]]; then + continue; + fi + if [[ $i == %+([A-Z])% ]]; then + current=$i + continue + fi + + case "$current" in + # desc + %NAME%) + echo "pkgname = $i" >> .PKGINFO + ;; + %VERSION%) + echo "pkgver = $i" >> .PKGINFO + ;; + %DESC%) + echo "pkgdesc = $i" >> .PKGINFO + ;; + %URL%) + echo "url = $i" >> .PKGINFO + ;; + %LICENSE%) + echo "license = $i" >> .PKGINFO + ;; + %ARCH%) + echo "arch = $i" >> .PKGINFO + pkg_arch="$i" + ;; + %BUILDDATE%) + echo "builddate = $(date -u "+%s")" >> .PKGINFO + ;; + %PACKAGER%) + echo "packager = $pkg_pkger" >> .PKGINFO + ;; + %SIZE%) + echo "size = $pkg_size" >> .PKGINFO + ;; + %GROUPS%) + echo "group = $i" >> .PKGINFO + ;; + %REPLACES%) + echo "replaces = $i" >> .PKGINFO + ;; + %DEPENDS%) + echo "depend = $i" >> .PKGINFO + ;; + %OPTDEPENDS%) + echo "optdepend = $i" >> .PKGINFO + ;; + %CONFLICTS%) + echo "conflict = $i" >> .PKGINFO + ;; + %PROVIDES%) + echo "provides = $i" >> .PKGINFO + ;; + %BACKUP%) + # Strip the md5sum after the tab + echo "backup = ${i%%$'\t'*}" >> .PKGINFO + ;; + esac + done < <(cat "$pkg_dir"/{desc,files}) + + comp_files=".PKGINFO" + + # Add instal file if present + if [[ -f $pkg_dir/install ]]; then + cp "$pkg_dir/install" "$work_dir/.INSTALL" + comp_files+=" .INSTALL" + fi + if [[ -f $pkg_dir/changelog ]]; then + cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" + comp_files+=" .CHANGELOG" fi - case "$current" in - # desc - %NAME%) - echo "pkgname = $i" >> .PKGINFO - ;; - %VERSION%) - echo "pkgver = $i" >> .PKGINFO - ;; - %DESC%) - echo "pkgdesc = $i" >> .PKGINFO - ;; - %URL%) - echo "url = $i" >> .PKGINFO - ;; - %LICENSE%) - echo "license = $i" >> .PKGINFO - ;; - %ARCH%) - echo "arch = $i" >> .PKGINFO - pkg_arch="$i" - ;; - %BUILDDATE%) - echo "builddate = $(date -u "+%s")" >> .PKGINFO - ;; - %PACKAGER%) - echo "packager = $pkg_pkger" >> .PKGINFO - ;; - %SIZE%) - echo "size = $pkg_size" >> .PKGINFO - ;; - %GROUPS%) - echo "group = $i" >> .PKGINFO - ;; - %REPLACES%) - echo "replaces = $i" >> .PKGINFO - ;; - %DEPENDS%) - echo "depend = $i" >> .PKGINFO - ;; - %OPTDEPENDS%) - echo "optdepend = $i" >> .PKGINFO - ;; - %CONFLICTS%) - echo "conflict = $i" >> .PKGINFO - ;; - %PROVIDES%) - echo "provides = $i" >> .PKGINFO - ;; - - # files - %BACKUP%) - # Strip the md5sum after the tab - echo "backup = ${i%%$'\t'*}" >> .PKGINFO - ;; - esac -done < <(cat "$pkg_dir"/{desc,files}) - -comp_files=".PKGINFO" - -if [[ -f $pkg_dir/install ]]; then - cp "$pkg_dir/install" "$work_dir/.INSTALL" - comp_files+=" .INSTALL" -fi -if [[ -f $pkg_dir/changelog ]]; then - cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" - comp_files+=" .CHANGELOG" -fi + # Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL + chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -# -# Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL -# -chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + # Generate the package + msg2 "Generating the package..." -# -# Generate the package -# -msg2 "Generating the package..." - -pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" -ret=0 - -# TODO: Maybe this can be set globally for robustness -shopt -s -o pipefail -bsdtar -cf - $comp_files * | -case "$PKGEXT" in - *tar.gz) gzip -c -f -n ;; - *tar.bz2) bzip2 -c -f ;; - *tar.xz) xz -c -z - ;; - *tar.Z) compress -c -f ;; - *tar) cat ;; - *) warning "'%s' is not a valid archive extension." \ - "$PKGEXT"; cat ;; -esac > "${pkg_file}"; ret=$? - -if (( ret )); then - error "Unable to write package to $pkg_dest" - plain " Maybe the disk is full or you do not have write access" + pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" + ret=0 + + # TODO: Maybe this can be set globally for robustness + shopt -s -o pipefail + bsdtar -cf - $comp_files * | + case "$PKGEXT" in + *tar.gz) gzip -c -f -n ;; + *tar.bz2) bzip2 -c -f ;; + *tar.xz) xz -c -z - ;; + *tar.Z) compress -c -f ;; + *tar) cat ;; + *) warning "'%s' is not a valid archive extension." \ + "$PKGEXT"; cat ;; + esac > "${pkg_file}"; ret=$? + + # Move compressed package to destination + if (( ret )); then + error "Unable to write package to $pkg_dest" + plain " Maybe the disk is full or you do not have write access" + rm -rf "$work_dir" + exit 1 + fi + + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi +} -rm -rf "$work_dir" +for PKG in $@; do fakebuild $PKG; done msg "Done." exit 0 -- 2.9.3
Trap SIGHUP, SIGINT, SIGTERM and remove working directories accordingly. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 5e6fce5..e7a7f57 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -33,6 +33,16 @@ ARGS=("$@") m4_include(../scripts/library/output_format.sh) +# Lazy recursive clean up of temporary dirs +work_dir_root="${TMPDIR:-/tmp}/bacman" +clean_up() { + rm -rf "$work_dir_root".* + echo + exit +} +# Trap termination signals +trap clean_up SIGHUP SIGINT SIGTERM + # # User Friendliness # -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index e7a7f57..0c71541 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -354,8 +354,29 @@ fakebuild() { rm -rf "$work_dir" } +# Run fakebuild in parralel with at maximum $MAX_JOBS jobs +# By default only run one job +MAX_JOBS=${MAX_JOBS:-1} +# TODO: Use wait -n instead of this hack as soon as it is ok +# to bump the version of the bash dependency +parallelize() { + while [[ $# -gt 0 ]]; do + job_count=($(jobs -p)) + if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then + fakebuild "$1" & + shift + fi + sleep 0.1 + done + wait +} -for PKG in $@; do fakebuild $PKG; done +# Initiate assembly function +if [[ $MAX_JOBS -gt "1" ]]; then + parallelize "$@" +else + for PKG in $@; do fakebuild $PKG; done +fi msg "Done." exit 0 -- 2.9.3
Switch to parseopts instead of merely checking the first argument. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 62 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 42 insertions(+), 20 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 0c71541..639a538 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -32,6 +32,7 @@ INCLUDE_PACNEW='n' ARGS=("$@") m4_include(../scripts/library/output_format.sh) +m4_include(../scripts/library/parseopts.sh) # Lazy recursive clean up of temporary dirs work_dir_root="${TMPDIR:-/tmp}/bacman" @@ -62,32 +63,51 @@ version() { echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } -while [[ ! -z $1 ]]; do - if [[ $1 == "--nocolor" ]]; then - USE_COLOR='n' - shift - elif [[ $1 == "--pacnew" ]]; then - INCLUDE_PACNEW='y' - shift - else - break - fi -done - +# Configure colored output m4_include(../scripts/library/term_colors.sh) # Break if no argument was given -if (( $# < 1 )); then +if (( $# == 0 )); then + usage + exit 1 +fi + +# Printing the usage information takes precedence over every other parameter +for option in "$@"; do + [[ $option == "-h" || $option == "--help" ]] && usage && exit 0 +done + +# Parse arguments +OPT_SHORT='mv' +OPT_LONG=('nocolor' 'pacnew' 'version') +if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 fi +set -- "${OPTRET[@]}" +unset OPT_SHORT OPT_LONG OPTRET + +while :; do + case "$1" in + -m|--nocolor) + USE_COLOR='n' ;; + --pacnew) + INCLUDE_PACNEW='y' ;; + -v|--version) + version + exit 0 ;; + --) + shift + break 2 ;; + esac + shift +done -if [[ $1 = -@(h|-help) ]]; then +# Retrieve the list of packages to be assembled and break if none was specified +pkg_list=($*) +if [[ ${#pkg_list[@]} == 0 ]]; then usage - exit 0 -elif [[ $1 = -@(V|-version) ]]; then - version - exit 0 + exit 1 fi # @@ -373,9 +393,11 @@ parallelize() { # Initiate assembly function if [[ $MAX_JOBS -gt "1" ]]; then - parallelize "$@" + parallelize "${pkg_list[@]}" else - for PKG in $@; do fakebuild $PKG; done + for PKG in ${pkg_list[@]}; do + fakebuild $PKG + done fi msg "Done." -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Move indentation fixes into its own commit. contrib/bacman.sh.in | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 639a538..c853958 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -48,13 +48,21 @@ trap clean_up SIGHUP SIGINT SIGTERM # User Friendliness # usage() { - echo "${myname} (pacman) v${myver}" + printf "%s (pacman) %s\n" "$myname" "$myver" echo - echo "Recreate a package using pacman's database and system files" + printf -- "$(gettext "Recreate packages using pacman's database and system files")\n" echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" + printf -- "$(gettext "Usage: %s [options] <package(s)>")\n" "$0" + echo + printf -- "$(gettext "Options:")\n" + printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " --pacnew Package .pacnew files")\n" + echo + printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo - echo "Example: ${myname} linux-headers" } version() { -- 2.9.3
Adding and clarifying comments. Adding and removing some new lines. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 25 ++++++++----------------- 1 file changed, 8 insertions(+), 17 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index c853958..d0309b6 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -44,9 +44,7 @@ clean_up() { # Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM -# -# User Friendliness -# +# Print usage information usage() { printf "%s (pacman) %s\n" "$myname" "$myver" echo @@ -65,6 +63,7 @@ usage() { echo } +# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' @@ -118,9 +117,7 @@ if [[ ${#pkg_list[@]} == 0 ]]; then exit 1 fi -# -# Fakeroot support -# +# Run with fake root privileges if EUID is not root if (( EUID )); then if [[ -f /usr/bin/fakeroot ]]; then msg "Entering fakeroot environment" @@ -133,33 +130,25 @@ if (( EUID )); then fi fi -# -# Setting environmental variables -# +# Source environmental variables and specify fallbacks if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 fi - eval $(awk '/DBPath/ {print $1$2$3}' @sysconfdir@/pacman.conf) pac_db="${DBPath:-@localstatedir@/lib/pacman/}/local" - if [[ ! -r @sysconfdir@/makepkg.conf ]]; then error "unable to read @sysconfdir@/makepkg.conf" exit 1 fi - source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi - pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -# -# Checks everything is in place -# +# Check for an existing database if [[ ! -d $pac_db ]]; then error "pacman database directory ${pac_db} not found" exit 1 @@ -191,6 +180,7 @@ fakebuild() { # Assemble list of files which belong to the package and tar them msg2 "Copying package files..." + while read i; do if [[ -z $i ]]; then continue @@ -357,6 +347,7 @@ fakebuild() { pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 + # Move compressed package to destination # TODO: Maybe this can be set globally for robustness shopt -s -o pipefail bsdtar -cf - $comp_files * | @@ -370,7 +361,7 @@ fakebuild() { "$PKGEXT"; cat ;; esac > "${pkg_file}"; ret=$? - # Move compressed package to destination + # Evaluate return code if (( ret )); then error "Unable to write package to $pkg_dest" plain " Maybe the disk is full or you do not have write access" -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index d0309b6..6f943a1 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -146,7 +146,7 @@ if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi pkg_dest="${PKGDEST:-$PWD}" -pkg_pkger=${PACKAGER:-'Unknown Packager'} +pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database if [[ ! -d $pac_db ]]; then -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Move indentation fixes into its own commit. contrib/bacman.sh.in | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 6f943a1..94b09d7 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -28,6 +28,7 @@ declare -r myname='bacman' declare -r myver='@PACKAGE_VERSION@' USE_COLOR='y' INCLUDE_PACNEW='n' +QUIET=0 # Required for fakeroot because options are shifted off the array. ARGS=("$@") @@ -54,6 +55,7 @@ usage() { echo printf -- "$(gettext "Options:")\n" printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo @@ -85,8 +87,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='mv' -OPT_LONG=('nocolor' 'pacnew' 'version') +OPT_SHORT='qmv' +OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -96,6 +98,8 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -q|--quiet) + QUIET=1 ;; -m|--nocolor) USE_COLOR='n' ;; --pacnew) -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 94b09d7..b92b4e4 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,10 +57,12 @@ usage() { printf -- "$(gettext " -h, --help Show this help message and exit")\n" printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " -o, --out <dir> Write output to specified directory (instead of \$PKGDEST)")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" - printf -- "$(gettext " %s --nocolor --pacnew gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s gzip munge binutils -o ~/Downloads")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew -o /tmp gzip munge binutils")\n" "$myname" printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo } @@ -87,8 +89,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='qmv' -OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT='o:qmv' +OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -98,6 +100,10 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -o|--out) + pkg_dest=$2 + [[ ! -d "$2" ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 + shift ;; -q|--quiet) QUIET=1 ;; -m|--nocolor) @@ -149,7 +155,8 @@ source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi -pkg_dest="${PKGDEST:-$PWD}" +PKGDEST="${PKGDEST:-$PWD}" +pkg_dest="${pkg_dest:-$PKGDEST}" pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index b92b4e4..7ae9058 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,12 +57,13 @@ usage() { printf -- "$(gettext " -h, --help Show this help message and exit")\n" printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " -j, --jobs <N> Build in parallel with N jobs - you may want to set XZ_OPT")\n" printf -- "$(gettext " -o, --out <dir> Write output to specified directory (instead of \$PKGDEST)")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" printf -- "$(gettext " %s gzip munge binutils -o ~/Downloads")\n" "$myname" - printf -- "$(gettext " %s --nocolor --pacnew -o /tmp gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew -o /tmp -j 5 gzip munge binutils")\n" "$myname" printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo } @@ -89,8 +90,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT='o:qmv' -OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT='o:j:qmv' +OPT_LONG=('out:' 'jobs:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -104,6 +105,14 @@ while :; do pkg_dest=$2 [[ ! -d "$2" ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 shift ;; + -j|--jobs) + if [[ $2 =~ ^-?[0-9]+$ ]]; then + MAX_JOBS=$2 + else + echo -e "\e[39;1m$2\e[0m is not a valid integer!" + exit -1 + fi + shift ;; -q|--quiet) QUIET=1 ;; -m|--nocolor) @@ -190,7 +199,11 @@ fakebuild() { cd "$work_dir" || exit 1 # Assemble list of files which belong to the package and tar them - msg2 "Copying package files..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Copying package files..." + else + msg2 "Copying package files..." + fi while read i; do if [[ -z $i ]]; then @@ -265,7 +278,11 @@ fakebuild() { # Reconstruct .PKGINFO from database # TODO adopt makepkg's write_pkginfo() into this or scripts/library - msg2 "Generating .PKGINFO metadata..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Generating .PKGINFO metadata..." + else + msg2 "Generating .PKGINFO metadata..." + fi echo "# Generated by $myname $myver" > .PKGINFO if [[ $INFAKEROOT == "1" ]]; then echo "# Using $(fakeroot -v)" >> .PKGINFO @@ -353,7 +370,11 @@ fakebuild() { chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null # Generate the package - msg2 "Generating the package..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Generating the package..." + else + msg2 "Generating the package..." + fi pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/doc/.gitignore | 1 + contrib/doc/Makefile.am | 5 +++- contrib/doc/bacman.8.txt | 66 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 71 insertions(+), 1 deletion(-) create mode 100644 contrib/doc/bacman.8.txt diff --git a/contrib/doc/.gitignore b/contrib/doc/.gitignore index c5612bc..3ab2035 100644 --- a/contrib/doc/.gitignore +++ b/contrib/doc/.gitignore @@ -1 +1,2 @@ verify-pacman-repo-db.1 +bacman.8 diff --git a/contrib/doc/Makefile.am b/contrib/doc/Makefile.am index 4c316bb..d5725b1 100644 --- a/contrib/doc/Makefile.am +++ b/contrib/doc/Makefile.am @@ -4,12 +4,14 @@ # man_MANS if --enable-asciidoc and/or --enable-doxygen are used. ASCIIDOC_MANS = \ - verify-pacman-repo-db.1 + verify-pacman-repo-db.1 \ + bacman.8 EXTRA_DIST = \ asciidoc.conf \ footer.txt \ verify-pacman-repo-db.1.txt \ + bacman.8.txt \ $(ASCIIDOC_MANS) # Files that should be removed, but which Automake does not know. @@ -53,5 +55,6 @@ $(ASCIIDOC_MANS): asciidoc.conf footer.txt Makefile.am # Dependency rules verify-pacman-repo-db.1: verify-pacman-repo-db.1.txt +bacman.8: bacman.8.txt # vim:set noet: diff --git a/contrib/doc/bacman.8.txt b/contrib/doc/bacman.8.txt new file mode 100644 index 0000000..a9d7ba9 --- /dev/null +++ b/contrib/doc/bacman.8.txt @@ -0,0 +1,66 @@ +///// +vim:set ts=4 sw=4 syntax=asciidoc noet spell spelllang=en_us: +///// +bacman(8) +========== + +Name +---- +bacman - recreate installed packages + + +Synopsis +-------- +'bacman' [options] <package(s)> + + +Description +----------- +'bacman' was designed to reassemble installed packages from its deliverd files. +It comes in handy if there is no internet connection available and you have no +access to a up-to-date package cache. + +'bacman' fully honors linkman:makepkg.conf[8] and all compression environment +variables, notably ``XZ_OPT''. + + +Options +------- +*-h, \--help*:: + Display usage information. + +*-q, \--quiet*:: + Silence most of the status reporting. + +*-m, \--nocolor*:: + Disable colored output. + +*-o, \--out <dir>*:: + Write the assembled package(s) to the specified directory. + +*\--pacnew*:: + Package .pacnew files instead of the concerning files currently in place. + + +Examples +-------- +'bacman' linux-headers + +Recreate the package ``linux-headers''. + +'bacman' gzip munge binutils -o ~/Downloads + +Assemble the packages gzip munge binutils and place the packages at +``~/Downloads''. + +'bacman' --nocolor --pacnew -o ~/backup $(pacman -Qsq) + +Assemble all currently installed packages using ``.pacnew'' whenever available, +suppress colored output and place the desired packages in ``~/backup''. + + +See Also +-------- +linkman:makepkg[8], linkman:pacman[8] + +include::footer.txt[] -- 2.9.3
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/doc/bacman.8.txt | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/contrib/doc/bacman.8.txt b/contrib/doc/bacman.8.txt index a9d7ba9..4738189 100644 --- a/contrib/doc/bacman.8.txt +++ b/contrib/doc/bacman.8.txt @@ -35,6 +35,9 @@ Options *-m, \--nocolor*:: Disable colored output. +*-j, \--jobs <jobs>*:: + Assemble multiple packages in parallel. + *-o, \--out <dir>*:: Write the assembled package(s) to the specified directory. @@ -53,10 +56,11 @@ Recreate the package ``linux-headers''. Assemble the packages gzip munge binutils and place the packages at ``~/Downloads''. -'bacman' --nocolor --pacnew -o ~/backup $(pacman -Qsq) +'bacman' --nocolor --pacnew -o ~/backup -j 5 $(pacman -Qsq) -Assemble all currently installed packages using ``.pacnew'' whenever available, -suppress colored output and place the desired packages in ``~/backup''. +Assemble all currently installed packages using 5 concurrent jobs, use +``.pacnew'' whenever available, suppress colored output and place the desired +packages in ``~/backup''. See Also -- 2.9.3
To enable the creation of multiple packages with one command move the assembly process into its own function. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 405 +++++++++++++++++++++++++-------------------------- 1 file changed, 200 insertions(+), 205 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index a611c1a..5e6fce5 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -66,7 +66,8 @@ done m4_include(../scripts/library/term_colors.sh) -if (( $# != 1 )); then +# Break if no argument was given +if (( $# < 1 )); then usage exit 1 fi @@ -118,10 +119,6 @@ fi pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -pkg_name="$1" -pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) -pkg_namver=("${pkg_dir[@]##*/}") - # # Checks everything is in place # @@ -130,227 +127,225 @@ if [[ ! -d $pac_db ]]; then exit 1 fi -if (( ${#pkg_dir[@]} != 1 )); then - error "%d entries for package %s found in pacman database" \ - ${#pkg_dir[@]} "${pkg_name}" - msg2 "%s" "${pkg_dir[@]}" - exit 1 -fi - -if [[ ! -d $pkg_dir ]]; then - error "package %s is found in pacman database," "${pkg_name}" - plain " but '%s' is not a directory" "${pkg_dir}" - exit 1 -fi - -# -# Begin -# -msg "Package: ${pkg_namver}" -work_dir=$(mktemp -d "${TMPDIR:-/tmp}/bacman.XXXXXXXXXX") -cd "$work_dir" || exit 1 - -# -# File copying -# -msg2 "Copying package files..." - -while read i; do - if [[ -z $i ]]; then - continue +# Assemble a single package: $1 = pkgname +fakebuild() { + pkg_name="$1" + pkg_dir=("$pac_db/$pkg_name"-+([^-])-+([^-])) + pkg_namver=("${pkg_dir[@]##*/}") + + # Checks database for specified package + if (( ${#pkg_dir[@]} != 1 )); then + error "%d entries for package %s found in pacman database" \ + ${#pkg_dir[@]} "${pkg_name}" + msg2 "%s" "${pkg_dir[@]}" + exit 1 fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + if [[ ! -d $pkg_dir ]]; then + error "package %s is found in pacman database," "${pkg_name}" + plain " but '%s' is not a directory" "${pkg_dir}" + exit 1 fi - case "$current" in - %FILES%) - local_file="/$i" - package_file="$work_dir/$i" + # Create working directory + msg "Package: ${pkg_namver}" + work_dir=$(mktemp -d "${work_dir_root}.XXXXXXXXXX") + cd "$work_dir" || exit 1 - if [[ ! -e $local_file ]]; then - warning "package file $local_file is missing" - continue - fi - ;; - - %BACKUP%) - # Get the MD5 checksum. - original_md5="${i##*$'\t'}" - # Strip the md5sum after the tab. - i="${i%$'\t'*}" - local_file="/$i.pacnew" - package_file="$work_dir/$i" - - # Include unmodified .pacnew files. - local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" - if [[ $INCLUDE_PACNEW == 'n' ]] \ - || [[ ! -e $local_file ]] \ - || [[ $local_md5 != $original_md5 ]]; then - # Warn about modified files. - local_md5="$(md5sum "/$i" | cut -d' ' -f1)" - if [[ $local_md5 != $original_md5 ]]; then - warning "package file /$i has been modified" - fi - # Let the normal file be included in the %FILES% list. - continue - fi - ;; + # Assemble list of files which belong to the package and tar them + msg2 "Copying package files..." + while read i; do + if [[ -z $i ]]; then + continue + fi - *) + if [[ $i == %+([A-Z])% ]]; then + current=$i continue - ;; - esac + fi - ret=0 - bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + case "$current" in + %FILES%) + local_file="/$i" + package_file="$work_dir/$i" + + if [[ ! -e $local_file ]]; then + warning "package file $local_file is missing" + continue + fi + ;; + + %BACKUP%) + # Get the MD5 checksum. + original_md5="${i##*$'\t'}" + # Strip the md5sum after the tab. + i="${i%$'\t'*}" + local_file="/$i.pacnew" + package_file="$work_dir/$i" + + # Include unmodified .pacnew files. + local_md5="$(md5sum "$local_file" | cut -d' ' -f1)" + if [[ $INCLUDE_PACNEW == 'n' ]] \ + || [[ ! -e $local_file ]] \ + || [[ $local_md5 != $original_md5 ]]; then + # Warn about modified files. + local_md5="$(md5sum "/$i" | cut -d' ' -f1)" + if [[ $local_md5 != $original_md5 ]]; then + warning "package file /$i has been modified" + fi + # Let the normal file be included in the %FILES% list. + continue + fi + ;; - # Workaround to bsdtar not reporting a missing file as an error - if ! [[ -e $package_file || -L $package_file ]]; then - error "unable to add $local_file to the package" - plain " If your user does not have permission to read this file, then" - plain " you will need to run $myname as root." + *) + continue + ;; + esac + + # Tar files + ret=0 + bsdtar -cnf - -s'/.pacnew$//' "$local_file" 2> /dev/null | bsdtar -xpf - 2> /dev/null + # Workaround to bsdtar not reporting a missing file as an error + if ! [[ -e $package_file || -L $package_file ]]; then + error "unable to add $local_file to the package" + plain " If your user does not have permission to read this file, then" + plain " you will need to run $myname as root." + rm -rf "$work_dir" + exit 1 + fi + done < "$pkg_dir"/files + + ret=$? + if (( ret )); then rm -rf "$work_dir" exit 1 fi -done < "$pkg_dir"/files -ret=$? -if (( ret )); then - rm -rf "$work_dir" - exit 1 -fi - -pkg_size=$(du -sk | awk '{print $1 * 1024}') - -# -# .PKGINFO stuff -# TODO adopt makepkg's write_pkginfo() into this or scripts/library -# -msg2 "Generating .PKGINFO metadata..." -echo "# Generated by $myname $myver" > .PKGINFO -if [[ $INFAKEROOT == "1" ]]; then - echo "# Using $(fakeroot -v)" >> .PKGINFO -fi -echo "# $(LC_ALL=C date)" >> .PKGINFO -echo "#" >> .PKGINFO + # Calculate package size + pkg_size=$(du -sk | awk '{print $1 * 1024}') -while read i; do - if [[ -z $i ]]; then - continue; + # Reconstruct .PKGINFO from database + # TODO adopt makepkg's write_pkginfo() into this or scripts/library + msg2 "Generating .PKGINFO metadata..." + echo "# Generated by $myname $myver" > .PKGINFO + if [[ $INFAKEROOT == "1" ]]; then + echo "# Using $(fakeroot -v)" >> .PKGINFO fi - - if [[ $i == %+([A-Z])% ]]; then - current=$i - continue + echo "# $(LC_ALL=C date)" >> .PKGINFO + echo "#" >> .PKGINFO + while read i; do + if [[ -z $i ]]; then + continue; + fi + if [[ $i == %+([A-Z])% ]]; then + current=$i + continue + fi + + case "$current" in + # desc + %NAME%) + echo "pkgname = $i" >> .PKGINFO + ;; + %VERSION%) + echo "pkgver = $i" >> .PKGINFO + ;; + %DESC%) + echo "pkgdesc = $i" >> .PKGINFO + ;; + %URL%) + echo "url = $i" >> .PKGINFO + ;; + %LICENSE%) + echo "license = $i" >> .PKGINFO + ;; + %ARCH%) + echo "arch = $i" >> .PKGINFO + pkg_arch="$i" + ;; + %BUILDDATE%) + echo "builddate = $(date -u "+%s")" >> .PKGINFO + ;; + %PACKAGER%) + echo "packager = $pkg_pkger" >> .PKGINFO + ;; + %SIZE%) + echo "size = $pkg_size" >> .PKGINFO + ;; + %GROUPS%) + echo "group = $i" >> .PKGINFO + ;; + %REPLACES%) + echo "replaces = $i" >> .PKGINFO + ;; + %DEPENDS%) + echo "depend = $i" >> .PKGINFO + ;; + %OPTDEPENDS%) + echo "optdepend = $i" >> .PKGINFO + ;; + %CONFLICTS%) + echo "conflict = $i" >> .PKGINFO + ;; + %PROVIDES%) + echo "provides = $i" >> .PKGINFO + ;; + %BACKUP%) + # Strip the md5sum after the tab + echo "backup = ${i%%$'\t'*}" >> .PKGINFO + ;; + esac + done < <(cat "$pkg_dir"/{desc,files}) + + comp_files=".PKGINFO" + + # Add instal file if present + if [[ -f $pkg_dir/install ]]; then + cp "$pkg_dir/install" "$work_dir/.INSTALL" + comp_files+=" .INSTALL" + fi + if [[ -f $pkg_dir/changelog ]]; then + cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" + comp_files+=" .CHANGELOG" fi - case "$current" in - # desc - %NAME%) - echo "pkgname = $i" >> .PKGINFO - ;; - %VERSION%) - echo "pkgver = $i" >> .PKGINFO - ;; - %DESC%) - echo "pkgdesc = $i" >> .PKGINFO - ;; - %URL%) - echo "url = $i" >> .PKGINFO - ;; - %LICENSE%) - echo "license = $i" >> .PKGINFO - ;; - %ARCH%) - echo "arch = $i" >> .PKGINFO - pkg_arch="$i" - ;; - %BUILDDATE%) - echo "builddate = $(date -u "+%s")" >> .PKGINFO - ;; - %PACKAGER%) - echo "packager = $pkg_pkger" >> .PKGINFO - ;; - %SIZE%) - echo "size = $pkg_size" >> .PKGINFO - ;; - %GROUPS%) - echo "group = $i" >> .PKGINFO - ;; - %REPLACES%) - echo "replaces = $i" >> .PKGINFO - ;; - %DEPENDS%) - echo "depend = $i" >> .PKGINFO - ;; - %OPTDEPENDS%) - echo "optdepend = $i" >> .PKGINFO - ;; - %CONFLICTS%) - echo "conflict = $i" >> .PKGINFO - ;; - %PROVIDES%) - echo "provides = $i" >> .PKGINFO - ;; - - # files - %BACKUP%) - # Strip the md5sum after the tab - echo "backup = ${i%%$'\t'*}" >> .PKGINFO - ;; - esac -done < <(cat "$pkg_dir"/{desc,files}) - -comp_files=".PKGINFO" - -if [[ -f $pkg_dir/install ]]; then - cp "$pkg_dir/install" "$work_dir/.INSTALL" - comp_files+=" .INSTALL" -fi -if [[ -f $pkg_dir/changelog ]]; then - cp "$pkg_dir/changelog" "$work_dir/.CHANGELOG" - comp_files+=" .CHANGELOG" -fi + # Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL + chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -# -# Fixes owner:group and permissions for .PKGINFO, .CHANGELOG, .INSTALL -# -chown root:root "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null -chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null + # Generate the package + msg2 "Generating the package..." -# -# Generate the package -# -msg2 "Generating the package..." - -pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" -ret=0 - -# TODO: Maybe this can be set globally for robustness -shopt -s -o pipefail -bsdtar -cf - $comp_files * | -case "$PKGEXT" in - *tar.gz) gzip -c -f -n ;; - *tar.bz2) bzip2 -c -f ;; - *tar.xz) xz -c -z - ;; - *tar.Z) compress -c -f ;; - *tar) cat ;; - *) warning "'%s' is not a valid archive extension." \ - "$PKGEXT"; cat ;; -esac > "${pkg_file}"; ret=$? - -if (( ret )); then - error "Unable to write package to $pkg_dest" - plain " Maybe the disk is full or you do not have write access" + pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" + ret=0 + + # TODO: Maybe this can be set globally for robustness + shopt -s -o pipefail + bsdtar -cf - $comp_files * | + case "$PKGEXT" in + *tar.gz) gzip -c -f -n ;; + *tar.bz2) bzip2 -c -f ;; + *tar.xz) xz -c -z - ;; + *tar.Z) compress -c -f ;; + *tar) cat ;; + *) warning "'%s' is not a valid archive extension." \ + "$PKGEXT"; cat ;; + esac > "${pkg_file}"; ret=$? + + # Move compressed package to destination + if (( ret )); then + error "Unable to write package to $pkg_dest" + plain " Maybe the disk is full or you do not have write access" + rm -rf "$work_dir" + exit 1 + fi + + # Clean up working directory rm -rf "$work_dir" - exit 1 -fi +} -rm -rf "$work_dir" +for PKG in $@; do fakebuild $PKG; done msg "Done." exit 0 -- 2.9.3
Trap SIGHUP, SIGINT, SIGTERM and remove working directories accordingly. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 5e6fce5..e7a7f57 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -33,6 +33,16 @@ ARGS=("$@") m4_include(../scripts/library/output_format.sh) +# Lazy recursive clean up of temporary dirs +work_dir_root="${TMPDIR:-/tmp}/bacman" +clean_up() { + rm -rf "$work_dir_root".* + echo + exit +} +# Trap termination signals +trap clean_up SIGHUP SIGINT SIGTERM + # # User Friendliness # -- 2.9.3
On 05/09/16 02:13, Gordian Edenhofer wrote:
Trap SIGHUP, SIGINT, SIGTERM and remove working directories accordingly.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com>
OK
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index e7a7f57..0c71541 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -354,8 +354,29 @@ fakebuild() { rm -rf "$work_dir" } +# Run fakebuild in parralel with at maximum $MAX_JOBS jobs +# By default only run one job +MAX_JOBS=${MAX_JOBS:-1} +# TODO: Use wait -n instead of this hack as soon as it is ok +# to bump the version of the bash dependency +parallelize() { + while [[ $# -gt 0 ]]; do + job_count=($(jobs -p)) + if [[ ${#job_count[@]} -lt $MAX_JOBS ]] ; then + fakebuild "$1" & + shift + fi + sleep 0.1 + done + wait +} -for PKG in $@; do fakebuild $PKG; done +# Initiate assembly function +if [[ $MAX_JOBS -gt "1" ]]; then + parallelize "$@" +else + for PKG in $@; do fakebuild $PKG; done +fi msg "Done." exit 0 -- 2.9.3
On 05/09/16 02:13, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com>
As discussed, this will not be pulled.
Switch to parseopts instead of merely checking the first argument. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- Use parseopts to quit in case no argument was given instead of manual checking. contrib/bacman.sh.in | 58 +++++++++++++++++++++++++++++++++------------------- 1 file changed, 37 insertions(+), 21 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 0c71541..e56f31b 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -32,6 +32,7 @@ INCLUDE_PACNEW='n' ARGS=("$@") m4_include(../scripts/library/output_format.sh) +m4_include(../scripts/library/parseopts.sh) # Lazy recursive clean up of temporary dirs work_dir_root="${TMPDIR:-/tmp}/bacman" @@ -62,32 +63,45 @@ version() { echo 'Copyright (C) 2008-2016 Pacman Development Team <pacman-dev@archlinux.org>' } -while [[ ! -z $1 ]]; do - if [[ $1 == "--nocolor" ]]; then - USE_COLOR='n' - shift - elif [[ $1 == "--pacnew" ]]; then - INCLUDE_PACNEW='y' - shift - else - break - fi -done - +# Configure colored output m4_include(../scripts/library/term_colors.sh) -# Break if no argument was given -if (( $# < 1 )); then +# Printing the usage information takes precedence over every other parameter +for option in "$@"; do + [[ $option == "-h" || $option == "--help" ]] && usage && exit 0 +done + +# Parse arguments +OPT_SHORT=':mv' +OPT_LONG=('nocolor' 'pacnew' 'version') +if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 fi +set -- "${OPTRET[@]}" +unset OPT_SHORT OPT_LONG OPTRET + +while :; do + case "$1" in + -m|--nocolor) + USE_COLOR='n' ;; + --pacnew) + INCLUDE_PACNEW='y' ;; + -v|--version) + version + exit 0 ;; + --) + shift + break 2 ;; + esac + shift +done -if [[ $1 = -@(h|-help) ]]; then +# Retrieve the list of packages to be assembled and break if none was specified +pkg_list=($*) +if [[ ${#pkg_list[@]} == 0 ]]; then usage - exit 0 -elif [[ $1 = -@(V|-version) ]]; then - version - exit 0 + exit 1 fi # @@ -373,9 +387,11 @@ parallelize() { # Initiate assembly function if [[ $MAX_JOBS -gt "1" ]]; then - parallelize "$@" + parallelize "${pkg_list[@]}" else - for PKG in $@; do fakebuild $PKG; done + for PKG in ${pkg_list[@]}; do + fakebuild $PKG + done fi msg "Done." -- 2.9.3
On 05/09/16 02:13, Gordian Edenhofer wrote:
Switch to parseopts instead of merely checking the first argument.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com>
OK. I made minor modifications due to not include patch #3. A
On 20/09/16 23:32, Allan McRae wrote:
On 05/09/16 02:13, Gordian Edenhofer wrote:
Switch to parseopts instead of merely checking the first argument.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com>
OK. I made minor modifications due to not include patch #3.
Well... I also had to fix: - -m|--nocolor stopped working due to moving where colors were set up - OPT_SHORT had an extra ":"
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index e56f31b..5045d6e 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -48,13 +48,21 @@ trap clean_up SIGHUP SIGINT SIGTERM # User Friendliness # usage() { - echo "${myname} (pacman) v${myver}" + printf "%s (pacman) %s\n" "$myname" "$myver" echo - echo "Recreate a package using pacman's database and system files" + printf -- "$(gettext "Recreate packages using pacman's database and system files")\n" echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" + printf -- "$(gettext "Usage: %s [options] <package(s)>")\n" "$0" + echo + printf -- "$(gettext "Options:")\n" + printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " --pacnew Package .pacnew files")\n" + echo + printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo - echo "Example: ${myname} linux-headers" } version() { -- 2.9.3
On 05/09/16 02:14, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index e56f31b..5045d6e 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -48,13 +48,21 @@ trap clean_up SIGHUP SIGINT SIGTERM # User Friendliness # usage() { - echo "${myname} (pacman) v${myver}" + printf "%s (pacman) %s\n" "$myname" "$myver" echo - echo "Recreate a package using pacman's database and system files" + printf -- "$(gettext "Recreate packages using pacman's database and system files")\n" echo - echo "Usage: ${myname} [--nocolor] [--pacnew] <installed package name>" + printf -- "$(gettext "Usage: %s [options] <package(s)>")\n" "$0" + echo + printf -- "$(gettext "Options:")\n" + printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " --pacnew Package .pacnew files")\n" + echo + printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname"
Moved "Examples:" to its own line for formatting for later examples to be consistent. Removed gettext calls to non-translatable examples
+ printf -- "$(gettext " %s --nocolor --pacnew gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname"
Minor modification... Changed -Qsq to -Qq
echo - echo "Example: ${myname} linux-headers" }
version() {
Adding and clarifying comments. Adding and removing some new lines. Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 25 ++++++++----------------- 1 file changed, 8 insertions(+), 17 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 5045d6e..7bf1aa0 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -44,9 +44,7 @@ clean_up() { # Trap termination signals trap clean_up SIGHUP SIGINT SIGTERM -# -# User Friendliness -# +# Print usage information usage() { printf "%s (pacman) %s\n" "$myname" "$myver" echo @@ -65,6 +63,7 @@ usage() { echo } +# Print version information version() { printf "%s %s\n" "$myname" "$myver" echo 'Copyright (C) 2008 locci <carlocci_at_gmail_dot_com>' @@ -112,9 +111,7 @@ if [[ ${#pkg_list[@]} == 0 ]]; then exit 1 fi -# -# Fakeroot support -# +# Run with fake root privileges if EUID is not root if (( EUID )); then if [[ -f /usr/bin/fakeroot ]]; then msg "Entering fakeroot environment" @@ -127,33 +124,25 @@ if (( EUID )); then fi fi -# -# Setting environmental variables -# +# Source environmental variables and specify fallbacks if [[ ! -r @sysconfdir@/pacman.conf ]]; then error "unable to read @sysconfdir@/pacman.conf" exit 1 fi - eval $(awk '/DBPath/ {print $1$2$3}' @sysconfdir@/pacman.conf) pac_db="${DBPath:-@localstatedir@/lib/pacman/}/local" - if [[ ! -r @sysconfdir@/makepkg.conf ]]; then error "unable to read @sysconfdir@/makepkg.conf" exit 1 fi - source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi - pkg_dest="${PKGDEST:-$PWD}" pkg_pkger=${PACKAGER:-'Unknown Packager'} -# -# Checks everything is in place -# +# Check for an existing database if [[ ! -d $pac_db ]]; then error "pacman database directory ${pac_db} not found" exit 1 @@ -185,6 +174,7 @@ fakebuild() { # Assemble list of files which belong to the package and tar them msg2 "Copying package files..." + while read i; do if [[ -z $i ]]; then continue @@ -351,6 +341,7 @@ fakebuild() { pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 + # Move compressed package to destination # TODO: Maybe this can be set globally for robustness shopt -s -o pipefail bsdtar -cf - $comp_files * | @@ -364,7 +355,7 @@ fakebuild() { "$PKGEXT"; cat ;; esac > "${pkg_file}"; ret=$? - # Move compressed package to destination + # Evaluate return code if (( ret )); then error "Unable to write package to $pkg_dest" plain " Maybe the disk is full or you do not have write access" -- 2.9.3
On 05/09/16 02:14, Gordian Edenhofer wrote:
Adding and clarifying comments. Adding and removing some new lines.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com>
OK.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 7bf1aa0..d98088a 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -140,7 +140,7 @@ if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi pkg_dest="${PKGDEST:-$PWD}" -pkg_pkger=${PACKAGER:-'Unknown Packager'} +pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database if [[ ! -d $pac_db ]]; then -- 2.9.3
On 05/09/16 02:14, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> ---
OK.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index d98088a..4e9e0a3 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -28,6 +28,7 @@ declare -r myname='bacman' declare -r myver='@PACKAGE_VERSION@' USE_COLOR='y' INCLUDE_PACNEW='n' +QUIET=0 # Required for fakeroot because options are shifted off the array. ARGS=("$@") @@ -54,6 +55,7 @@ usage() { echo printf -- "$(gettext "Options:")\n" printf -- "$(gettext " -h, --help Show this help message and exit")\n" + printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo @@ -79,8 +81,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT=':mv' -OPT_LONG=('nocolor' 'pacnew' 'version') +OPT_SHORT=':qmv' +OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -90,6 +92,8 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -q|--quiet) + QUIET=1 ;; -m|--nocolor) USE_COLOR='n' ;; --pacnew) -- 2.9.3
On 05/09/16 02:14, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
OK.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 4e9e0a3..7e511d6 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,10 +57,12 @@ usage() { printf -- "$(gettext " -h, --help Show this help message and exit")\n" printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " -o, --out <dir> Write output to specified directory (instead of \$PKGDEST)")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" - printf -- "$(gettext " %s --nocolor --pacnew gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s gzip munge binutils -o ~/Downloads")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew -o /tmp gzip munge binutils")\n" "$myname" printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo } @@ -81,8 +83,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT=':qmv' -OPT_LONG=('quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT=':o:qmv' +OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -92,6 +94,10 @@ unset OPT_SHORT OPT_LONG OPTRET while :; do case "$1" in + -o|--out) + pkg_dest=$2 + [[ ! -d "$2" ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 + shift ;; -q|--quiet) QUIET=1 ;; -m|--nocolor) @@ -143,7 +149,8 @@ source "@sysconfdir@/makepkg.conf" if [[ -r ~/.makepkg.conf ]]; then source ~/.makepkg.conf fi -pkg_dest="${PKGDEST:-$PWD}" +PKGDEST="${PKGDEST:-$PWD}" +pkg_dest="${pkg_dest:-$PKGDEST}" pkg_pkger="${PACKAGER:-'Unknown Packager'}" # Check for an existing database -- 2.9.3
On 05/09/16 02:14, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
OK - fixed typo in subject
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/bacman.sh.in | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/contrib/bacman.sh.in b/contrib/bacman.sh.in index 7e511d6..d0c15c9 100644 --- a/contrib/bacman.sh.in +++ b/contrib/bacman.sh.in @@ -57,12 +57,13 @@ usage() { printf -- "$(gettext " -h, --help Show this help message and exit")\n" printf -- "$(gettext " -q, --quiet Silence most of the status reporting")\n" printf -- "$(gettext " -m, --nocolor Disable colorized output messages")\n" + printf -- "$(gettext " -j, --jobs <N> Build in parallel with N jobs - you may want to set XZ_OPT")\n" printf -- "$(gettext " -o, --out <dir> Write output to specified directory (instead of \$PKGDEST)")\n" printf -- "$(gettext " --pacnew Package .pacnew files")\n" echo printf -- "$(gettext "Examples: %s linux-headers")\n" "$myname" printf -- "$(gettext " %s gzip munge binutils -o ~/Downloads")\n" "$myname" - printf -- "$(gettext " %s --nocolor --pacnew -o /tmp gzip munge binutils")\n" "$myname" + printf -- "$(gettext " %s --nocolor --pacnew -o /tmp -j 5 gzip munge binutils")\n" "$myname" printf -- "$(gettext " %s \$(pacman -Qsq)")\n" "$myname" echo } @@ -83,8 +84,8 @@ for option in "$@"; do done # Parse arguments -OPT_SHORT=':o:qmv' -OPT_LONG=('out:' 'quiet' 'nocolor' 'pacnew' 'version') +OPT_SHORT=':o:j:qmv' +OPT_LONG=('out:' 'jobs:' 'quiet' 'nocolor' 'pacnew' 'version') if ! parseopts "$OPT_SHORT" "${OPT_LONG[@]}" -- "$@"; then usage exit 1 @@ -98,6 +99,14 @@ while :; do pkg_dest=$2 [[ ! -d "$2" ]] && echo -e "The directory \e[39;1m$2\e[0m does not exist!" && exit 3 shift ;; + -j|--jobs) + if [[ $2 =~ ^-?[0-9]+$ ]]; then + MAX_JOBS=$2 + else + echo -e "\e[39;1m$2\e[0m is not a valid integer!" + exit -1 + fi + shift ;; -q|--quiet) QUIET=1 ;; -m|--nocolor) @@ -184,7 +193,11 @@ fakebuild() { cd "$work_dir" || exit 1 # Assemble list of files which belong to the package and tar them - msg2 "Copying package files..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Copying package files..." + else + msg2 "Copying package files..." + fi while read i; do if [[ -z $i ]]; then @@ -259,7 +272,11 @@ fakebuild() { # Reconstruct .PKGINFO from database # TODO adopt makepkg's write_pkginfo() into this or scripts/library - msg2 "Generating .PKGINFO metadata..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Generating .PKGINFO metadata..." + else + msg2 "Generating .PKGINFO metadata..." + fi echo "# Generated by $myname $myver" > .PKGINFO if [[ $INFAKEROOT == "1" ]]; then echo "# Using $(fakeroot -v)" >> .PKGINFO @@ -347,7 +364,11 @@ fakebuild() { chmod 644 "$work_dir"/{.PKGINFO,.CHANGELOG,.INSTALL} 2> /dev/null # Generate the package - msg2 "Generating the package..." + if [[ $MAX_JOBS -gt 1 ]]; then + msg2 "${pkg_name}: Generating the package..." + else + msg2 "Generating the package..." + fi pkg_file="$pkg_dest/$pkg_namver-$pkg_arch${PKGEXT}" ret=0 -- 2.9.3
On 05/09/16 02:14, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com>
As discussed, this will not be included. A
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/doc/.gitignore | 1 + contrib/doc/Makefile.am | 5 +++- contrib/doc/bacman.8.txt | 66 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 71 insertions(+), 1 deletion(-) create mode 100644 contrib/doc/bacman.8.txt diff --git a/contrib/doc/.gitignore b/contrib/doc/.gitignore index c5612bc..3ab2035 100644 --- a/contrib/doc/.gitignore +++ b/contrib/doc/.gitignore @@ -1 +1,2 @@ verify-pacman-repo-db.1 +bacman.8 diff --git a/contrib/doc/Makefile.am b/contrib/doc/Makefile.am index 4c316bb..d5725b1 100644 --- a/contrib/doc/Makefile.am +++ b/contrib/doc/Makefile.am @@ -4,12 +4,14 @@ # man_MANS if --enable-asciidoc and/or --enable-doxygen are used. ASCIIDOC_MANS = \ - verify-pacman-repo-db.1 + verify-pacman-repo-db.1 \ + bacman.8 EXTRA_DIST = \ asciidoc.conf \ footer.txt \ verify-pacman-repo-db.1.txt \ + bacman.8.txt \ $(ASCIIDOC_MANS) # Files that should be removed, but which Automake does not know. @@ -53,5 +55,6 @@ $(ASCIIDOC_MANS): asciidoc.conf footer.txt Makefile.am # Dependency rules verify-pacman-repo-db.1: verify-pacman-repo-db.1.txt +bacman.8: bacman.8.txt # vim:set noet: diff --git a/contrib/doc/bacman.8.txt b/contrib/doc/bacman.8.txt new file mode 100644 index 0000000..a9d7ba9 --- /dev/null +++ b/contrib/doc/bacman.8.txt @@ -0,0 +1,66 @@ +///// +vim:set ts=4 sw=4 syntax=asciidoc noet spell spelllang=en_us: +///// +bacman(8) +========== + +Name +---- +bacman - recreate installed packages + + +Synopsis +-------- +'bacman' [options] <package(s)> + + +Description +----------- +'bacman' was designed to reassemble installed packages from its deliverd files. +It comes in handy if there is no internet connection available and you have no +access to a up-to-date package cache. + +'bacman' fully honors linkman:makepkg.conf[8] and all compression environment +variables, notably ``XZ_OPT''. + + +Options +------- +*-h, \--help*:: + Display usage information. + +*-q, \--quiet*:: + Silence most of the status reporting. + +*-m, \--nocolor*:: + Disable colored output. + +*-o, \--out <dir>*:: + Write the assembled package(s) to the specified directory. + +*\--pacnew*:: + Package .pacnew files instead of the concerning files currently in place. + + +Examples +-------- +'bacman' linux-headers + +Recreate the package ``linux-headers''. + +'bacman' gzip munge binutils -o ~/Downloads + +Assemble the packages gzip munge binutils and place the packages at +``~/Downloads''. + +'bacman' --nocolor --pacnew -o ~/backup $(pacman -Qsq) + +Assemble all currently installed packages using ``.pacnew'' whenever available, +suppress colored output and place the desired packages in ``~/backup''. + + +See Also +-------- +linkman:makepkg[8], linkman:pacman[8] + +include::footer.txt[] -- 2.9.3
On 05/09/16 02:14, Gordian Edenhofer wrote:
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com>
I made some minor wording changes and adjusted the examples section formatting. A
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com> --- contrib/doc/bacman.8.txt | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/contrib/doc/bacman.8.txt b/contrib/doc/bacman.8.txt index a9d7ba9..4738189 100644 --- a/contrib/doc/bacman.8.txt +++ b/contrib/doc/bacman.8.txt @@ -35,6 +35,9 @@ Options *-m, \--nocolor*:: Disable colored output. +*-j, \--jobs <jobs>*:: + Assemble multiple packages in parallel. + *-o, \--out <dir>*:: Write the assembled package(s) to the specified directory. @@ -53,10 +56,11 @@ Recreate the package ``linux-headers''. Assemble the packages gzip munge binutils and place the packages at ``~/Downloads''. -'bacman' --nocolor --pacnew -o ~/backup $(pacman -Qsq) +'bacman' --nocolor --pacnew -o ~/backup -j 5 $(pacman -Qsq) -Assemble all currently installed packages using ``.pacnew'' whenever available, -suppress colored output and place the desired packages in ``~/backup''. +Assemble all currently installed packages using 5 concurrent jobs, use +``.pacnew'' whenever available, suppress colored output and place the desired +packages in ``~/backup''. See Also -- 2.9.3
On Sun, 4 Sep 2016 18:13:56 +0200 Gordian Edenhofer <gordian.edenhofer@gmail.com> wrote:
To enable the creation of multiple packages with one command move the assembly process into its own function.
Why do you keep sending these? Allan has already said they will not be merged.
On Sun, 2016-09-04 at 11:21 -0500, Doug Newgard wrote:
On Sun, 4 Sep 2016 18:13:56 +0200 Gordian Edenhofer <gordian.edenhofer@gmail.com> wrote:
To enable the creation of multiple packages with one command move the assembly process into its own function.
Why do you keep sending these? Allan has already said they will not be merged.
Allen objected parallelization as far as I understood him not the patches as a whole. What harm should a man page and the use of parseopts bring? Most of the recent mails were in reply to proposed amendments and the patchset should be properly split up into different parts by now to allow for cherry-picking the parts that are acceptable. Since I moved some things into the wrong commit when I divided 3 patches into 12 I had to submit it a couple of more times and I am sorry for spamming the mailing list with those mails. Furthermore until recently I was still fighting for parallelization but after talking to Allen on IRC I realised this is a lost case. Therefore upon request I will submit the patches once more without support for parallelization. Once again, please excuse me for sending those many mails. Best Regards, Gordian Edenhofer
On 05/09/16 03:37, Gordian Edenhofer wrote:
On Sun, 2016-09-04 at 11:21 -0500, Doug Newgard wrote:
On Sun, 4 Sep 2016 18:13:56 +0200 Gordian Edenhofer <gordian.edenhofer@gmail.com> wrote:
To enable the creation of multiple packages with one command move the assembly process into its own function.
Why do you keep sending these? Allan has already said they will not be merged.
Allen objected parallelization as far as I understood him not the patches as a whole. What harm should a man page and the use of parseopts bring? Most of the recent mails were in reply to proposed amendments and the patchset should be properly split up into different parts by now to allow for cherry-picking the parts that are acceptable. Since I moved some things into the wrong commit when I divided 3 patches into 12 I had to submit it a couple of more times and I am sorry for spamming the mailing list with those mails. Furthermore until recently I was still fighting for parallelization but after talking to Allen on IRC I realised this is a lost case. Therefore upon request I will submit the patches once more without support for parallelization.
Correct - most changes will be pulled from this patchset. Just not the parallelization patches. I will pull the other patches to my patchqueue branch soon. Allan
On 09/09/16 16:21, Allan McRae wrote:
Correct - most changes will be pulled from this patchset. Just not the parallelization patches. I will pull the other patches to my patchqueue branch soon.
You can now see what I intend to commit on my "bacman" branch: https://git.archlinux.org/users/allan/pacman.git/log/?h=bacman Allan
On Wed, 2016-09-21 at 00:08 +1000, Allan McRae wrote:
On 09/09/16 16:21, Allan McRae wrote:
Correct - most changes will be pulled from this patchset. Just not the parallelization patches. I will pull the other patches to my patchqueue branch soon.
You can now see what I intend to commit on my "bacman" branch:
https://git.archlinux.org/users/allan/pacman.git/log/?h=bacman
You forgot to change `pacman -Qsq` to `pacman -Qq` in the usage function. Besides that it looks good to me. Sorry for hiccups in the patchset.
On 21/09/16 20:08, Gordian Edenhofer wrote:
On Wed, 2016-09-21 at 00:08 +1000, Allan McRae wrote:
On 09/09/16 16:21, Allan McRae wrote:
Correct - most changes will be pulled from this patchset. Just not the parallelization patches. I will pull the other patches to my patchqueue branch soon.
You can now see what I intend to commit on my "bacman" branch:
https://git.archlinux.org/users/allan/pacman.git/log/?h=bacman
You forgot to change `pacman -Qsq` to `pacman -Qq` in the usage function. Besides that it looks good to me. Sorry for hiccups in the patchset.
Thanks - fixed that now. Allan
On 05/09/16 02:13, Gordian Edenhofer wrote:
To enable the creation of multiple packages with one command move the assembly process into its own function.
Signed-off-by: Gordian Edenhofer <gordian.edenhofer@gmail.com>
OK
participants (4)
-
Allan McRae
-
Dave Reisner
-
Doug Newgard
-
Gordian Edenhofer