[pacman-dev] [ABS PATCH] Allow syncing the ABS tree using tarballs
I figured I'd post this here for comments seem it is a fairly decent sized patch. Thanks, Allan Some people are stuck behind restrictive firewalls and can not sync the ABS tree using rsync. This patch adds support for downloading ABS snapshot tarballs from pacman mirrors. Signed-off-by: Allan McRae <allan@archlinux.org> --- abs | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++---- conf/abs.conf | 7 ++++++- 2 files changed, 58 insertions(+), 5 deletions(-) diff --git a/abs b/abs index bd1191d..a369482 100755 --- a/abs +++ b/abs @@ -80,6 +80,7 @@ usage() { echo "Options:" echo " -h, --help Display this help message then exit." echo " -V, --version Display version information then exit." + echo " -t, --tarball Sync ABS tree using tarballs from your pacman mirror." echo echo "abs will synchronize PKGBUILD scripts from the Arch Linux repositories" echo "into $ABSROOT via rsync. If no argument is given, abs will synchronize" @@ -97,7 +98,7 @@ version() { echo "There is NO WARRANTY, to the extent permitted by law." } -update() { +update_rsync() { cd "$ABSROOT" if [ "$CLPARAM" = "" ]; then @@ -138,6 +139,48 @@ update() { $SYNCCMD $SYNCARGS $INCLUDE $EXCLUDE ${SYNCSERVER}::abs/${ARCH}/ $ABSROOT } +update_tarball() { + cd "$ABSROOT" + + if [ ! "$(type -p rsync)" ]; then + error "missing wget download utility. Install wget." + exit $_E_MISSING_PROGRAM + fi + + if [ -f "$MIRRORLIST" ]; then + mirrorlist=$(grep "^Server" $MIRRORLIST | cut -f3 -d" ") + else + error "Could not find pacman mirrorlist file $MIRRORLIST" + exit $_E_CONFIG_ERROR + fi + + msg "Downloading tarballs..." + for repo in "${REPOS[@]}"; do + if [ "$repo" == "${repo#!}" ]; then + echo " ==> ${repo}..." + + ret=0 + for mirror in ${mirrorlist[@]}; do + tarball=$(echo $mirror | sed "s#\$repo#$repo#") + tarball="${tarball}/${repo}.abs.tar.gz" + wget -q $tarball || ret=$? + if [ $ret -eq 0 ]; then + break + fi + done + + if [ $ret -ne 0 ]; then + echo " Download failed" + continue + fi + + rm -rf $repo + tar -xzf ${repo}.abs.tar.gz + rm ${repo}.abs.tar.gz + fi + done +} + ## # Signal Traps ## @@ -148,8 +191,8 @@ trap 'error "An unknown error has occured. Exiting..."; exit 1' ERR ## # Parse Options ## -OPT_SHORT="hV" -OPT_LONG="help,version" +OPT_SHORT="hVt" +OPT_LONG="help,version,tarball" OPT_TEMP="$(getopt -o "$OPT_SHORT" -l "$OPT_LONG" -n "$(basename "$0")" -- "$@" || echo 'GETOPT GO BANG!')" if echo "$OPT_TEMP" | grep -q 'GETOPT GO BANG!'; then # This is a small hack to stop the script bailing with 'set -e' @@ -162,6 +205,7 @@ while true; do case "$1" in -h|--help) usage; exit $_E_OK;; -V|--version) version; exit $_E_OK;; + -t|--tarball) TARBALL=1;; --) OPT_IND=0; shift; break;; *) usage; exit $_E_INVALID_OPTION;; @@ -195,7 +239,11 @@ fi ## # Go-go Update ABS tree! ## -update +if [ "$TARBALL" ]; then + update_tarball +else + update_rsync +fi exit $_E_OK diff --git a/conf/abs.conf b/conf/abs.conf index 7d5fd79..6511a58 100644 --- a/conf/abs.conf +++ b/conf/abs.conf @@ -1,5 +1,5 @@ # -# /etc/abs/abs.conf +# /etc/abs.conf # # the top-level directory of all your PKGBUILDs @@ -17,6 +17,11 @@ SYNCSERVER="rsync.archlinux.org" ARCH="i686" # +# Pacman mirror list used for syncing via tarball +# +MIRRORLIST="/etc/pacman.d/mirrorlist" + +# # REPOS to be parsed by abs (in this order) # (prefix a repo with a ! to disable it) # -- 1.6.0.3
+ if [ ! "$(type -p rsync)" ]; then + error "missing wget download utility. Install wget." + exit $_E_MISSING_PROGRAM + fi You probably meant 'type -p wget' here? You can also just check the exit status of type btw, instead of capturing and comparing its output:. if ! type -p wget >/dev/null; then .... Dieter Allan McRae wrote:
I figured I'd post this here for comments seem it is a fairly decent sized patch. Thanks, Allan
Some people are stuck behind restrictive firewalls and can not sync the ABS tree using rsync. This patch adds support for downloading ABS snapshot tarballs from pacman mirrors.
Signed-off-by: Allan McRae <allan@archlinux.org> --- abs | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++---- conf/abs.conf | 7 ++++++- 2 files changed, 58 insertions(+), 5 deletions(-)
diff --git a/abs b/abs index bd1191d..a369482 100755 --- a/abs +++ b/abs @@ -80,6 +80,7 @@ usage() { echo "Options:" echo " -h, --help Display this help message then exit." echo " -V, --version Display version information then exit." + echo " -t, --tarball Sync ABS tree using tarballs from your pacman mirror." echo echo "abs will synchronize PKGBUILD scripts from the Arch Linux repositories" echo "into $ABSROOT via rsync. If no argument is given, abs will synchronize" @@ -97,7 +98,7 @@ version() { echo "There is NO WARRANTY, to the extent permitted by law." }
-update() { +update_rsync() { cd "$ABSROOT"
if [ "$CLPARAM" = "" ]; then @@ -138,6 +139,48 @@ update() { $SYNCCMD $SYNCARGS $INCLUDE $EXCLUDE ${SYNCSERVER}::abs/${ARCH}/ $ABSROOT }
+update_tarball() { + cd "$ABSROOT" + + if [ ! "$(type -p rsync)" ]; then + error "missing wget download utility. Install wget." + exit $_E_MISSING_PROGRAM + fi + + if [ -f "$MIRRORLIST" ]; then + mirrorlist=$(grep "^Server" $MIRRORLIST | cut -f3 -d" ") + else + error "Could not find pacman mirrorlist file $MIRRORLIST" + exit $_E_CONFIG_ERROR + fi + + msg "Downloading tarballs..." + for repo in "${REPOS[@]}"; do + if [ "$repo" == "${repo#!}" ]; then + echo " ==> ${repo}..." + + ret=0 + for mirror in ${mirrorlist[@]}; do + tarball=$(echo $mirror | sed "s#\$repo#$repo#") + tarball="${tarball}/${repo}.abs.tar.gz" + wget -q $tarball || ret=$? + if [ $ret -eq 0 ]; then + break + fi + done + + if [ $ret -ne 0 ]; then + echo " Download failed" + continue + fi + + rm -rf $repo + tar -xzf ${repo}.abs.tar.gz + rm ${repo}.abs.tar.gz + fi + done +} + ## # Signal Traps ## @@ -148,8 +191,8 @@ trap 'error "An unknown error has occured. Exiting..."; exit 1' ERR ## # Parse Options ## -OPT_SHORT="hV" -OPT_LONG="help,version" +OPT_SHORT="hVt" +OPT_LONG="help,version,tarball" OPT_TEMP="$(getopt -o "$OPT_SHORT" -l "$OPT_LONG" -n "$(basename "$0")" -- "$@" || echo 'GETOPT GO BANG!')" if echo "$OPT_TEMP" | grep -q 'GETOPT GO BANG!'; then # This is a small hack to stop the script bailing with 'set -e' @@ -162,6 +205,7 @@ while true; do case "$1" in -h|--help) usage; exit $_E_OK;; -V|--version) version; exit $_E_OK;; + -t|--tarball) TARBALL=1;;
--) OPT_IND=0; shift; break;; *) usage; exit $_E_INVALID_OPTION;; @@ -195,7 +239,11 @@ fi ## # Go-go Update ABS tree! ## -update +if [ "$TARBALL" ]; then + update_tarball +else + update_rsync +fi
exit $_E_OK
diff --git a/conf/abs.conf b/conf/abs.conf index 7d5fd79..6511a58 100644 --- a/conf/abs.conf +++ b/conf/abs.conf @@ -1,5 +1,5 @@ # -# /etc/abs/abs.conf +# /etc/abs.conf #
# the top-level directory of all your PKGBUILDs @@ -17,6 +17,11 @@ SYNCSERVER="rsync.archlinux.org" ARCH="i686"
# +# Pacman mirror list used for syncing via tarball +# +MIRRORLIST="/etc/pacman.d/mirrorlist" + +# # REPOS to be parsed by abs (in this order) # (prefix a repo with a ! to disable it) #
Dieter Plaetinck wrote:
+ if [ ! "$(type -p rsync)" ]; then + error "missing wget download utility. Install wget." + exit $_E_MISSING_PROGRAM + fi
You probably meant 'type -p wget' here?
Good catch! A bit too much copy/paste here...
You can also just check the exit status of type btw, instead of capturing and comparing its output:. if ! type -p wget >/dev/null; then ....
The current way is used elsewhere in the abs code and given this makes little difference I will stick with it. Thanks, Allan
On Sun, Oct 26, 2008 at 2:17 PM, Allan McRae <allan@archlinux.org> wrote:
I figured I'd post this here for comments seem it is a fairly decent sized patch. Thanks, Allan
Some people are stuck behind restrictive firewalls and can not sync the ABS tree using rsync. This patch adds support for downloading ABS snapshot tarballs from pacman mirrors.
I didn't read the patch, but +1 for the idea since it could have helped me several times :)
participants (3)
-
Allan McRae
-
Dieter Plaetinck
-
Xavier