[aur-dev] [PATCH] Add a upload directory tranform script
This goes with the previous patch that moves uploads into segmented subdirectories. Signed-off-by: Dan McGee <dan@archlinux.org> --- scripts/uploadbuckets.sh | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 42 insertions(+), 0 deletions(-) create mode 100755 scripts/uploadbuckets.sh diff --git a/scripts/uploadbuckets.sh b/scripts/uploadbuckets.sh new file mode 100755 index 0000000..1bb1a90 --- /dev/null +++ b/scripts/uploadbuckets.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +DRYRUN=1 + +source="$1" +dest="$2" + +if [[ -z $source || -z $dest ]]; then + echo 'usage: uploadbuckets.sh <source> <dest>' + exit 1 +fi + +if [[ ! -d $source ]]; then + echo 'error: source is not a directory' + exit 1 +fi + +if [[ -e $dest && ! -d $dest ]]; then + echo 'error: dest is not a directory' + exit 1 +fi + +if [[ ! -d $dest ]]; then + mkdir $dest +fi + +for package in "$source"/*; do + pkgname="${package##*/}" + newfolder="$dest/${pkgname:0:2}" + if [[ ! -d "$newfolder" ]]; then + if [[ DRYRUN -gt 0 ]]; then + echo mkdir -p "$newfolder" + else + mkdir -p "$newfolder" + fi + fi + if [[ DRYRUN -gt 0 ]]; then + echo mv "$source/$pkgname" "$newfolder/$pkgname" + else + mv "$source/$pkgname" "$newfolder/$pkgname" + fi +done -- 1.7.6
On Thu, Jul 28, 2011 at 02:16:37PM -0500, Dan McGee wrote:
This goes with the previous patch that moves uploads into segmented subdirectories.
Signed-off-by: Dan McGee <dan@archlinux.org> --- scripts/uploadbuckets.sh | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 42 insertions(+), 0 deletions(-) create mode 100755 scripts/uploadbuckets.sh
diff --git a/scripts/uploadbuckets.sh b/scripts/uploadbuckets.sh new file mode 100755 index 0000000..1bb1a90 --- /dev/null +++ b/scripts/uploadbuckets.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +DRYRUN=1 + +source="$1" +dest="$2" + +if [[ -z $source || -z $dest ]]; then + echo 'usage: uploadbuckets.sh <source> <dest>' + exit 1 +fi + +if [[ ! -d $source ]]; then + echo 'error: source is not a directory' + exit 1 +fi + +if [[ -e $dest && ! -d $dest ]]; then + echo 'error: dest is not a directory' + exit 1 +fi + +if [[ ! -d $dest ]]; then + mkdir $dest +fi + +for package in "$source"/*; do + pkgname="${package##*/}" + newfolder="$dest/${pkgname:0:2}" + if [[ ! -d "$newfolder" ]]; then + if [[ DRYRUN -gt 0 ]]; then + echo mkdir -p "$newfolder" + else + mkdir -p "$newfolder" + fi + fi + if [[ DRYRUN -gt 0 ]]; then + echo mv "$source/$pkgname" "$newfolder/$pkgname" + else + mv "$source/$pkgname" "$newfolder/$pkgname"
This won't work if "$source" = "$dest" and there's a package name with length 2. We should probably add a check to assert that source and target directory are not equal.
+ fi +done -- 1.7.6
On Thu, Jul 28, 2011 at 02:16:37PM -0500, Dan McGee wrote:
This goes with the previous patch that moves uploads into segmented subdirectories.
Signed-off-by: Dan McGee <dan@archlinux.org> --- scripts/uploadbuckets.sh | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 42 insertions(+), 0 deletions(-) create mode 100755 scripts/uploadbuckets.sh
diff --git a/scripts/uploadbuckets.sh b/scripts/uploadbuckets.sh new file mode 100755 index 0000000..1bb1a90 --- /dev/null +++ b/scripts/uploadbuckets.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +DRYRUN=1 + +source="$1" +dest="$2" + +if [[ -z $source || -z $dest ]]; then + echo 'usage: uploadbuckets.sh <source> <dest>' + exit 1 +fi + +if [[ ! -d $source ]]; then + echo 'error: source is not a directory' + exit 1 +fi + +if [[ -e $dest && ! -d $dest ]]; then + echo 'error: dest is not a directory' + exit 1 +fi + +if [[ ! -d $dest ]]; then + mkdir $dest +fi + +for package in "$source"/*; do + pkgname="${package##*/}" + newfolder="$dest/${pkgname:0:2}" + if [[ ! -d "$newfolder" ]]; then + if [[ DRYRUN -gt 0 ]]; then + echo mkdir -p "$newfolder" + else + mkdir -p "$newfolder" + fi + fi + if [[ DRYRUN -gt 0 ]]; then
Either need to use [[ $DRYRUN -gt 0 ]] or (( DRYRUN > 0 )), otherwise it's always false.
+ echo mv "$source/$pkgname" "$newfolder/$pkgname" + else + mv "$source/$pkgname" "$newfolder/$pkgname" + fi +done -- 1.7.6
On Fri, Jul 29, 2011 at 8:33 PM, Dave Reisner <d@falconindy.com> wrote:
On Thu, Jul 28, 2011 at 02:16:37PM -0500, Dan McGee wrote:
This goes with the previous patch that moves uploads into segmented subdirectories.
Signed-off-by: Dan McGee <dan@archlinux.org> --- scripts/uploadbuckets.sh | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 42 insertions(+), 0 deletions(-) create mode 100755 scripts/uploadbuckets.sh
diff --git a/scripts/uploadbuckets.sh b/scripts/uploadbuckets.sh new file mode 100755 index 0000000..1bb1a90 --- /dev/null +++ b/scripts/uploadbuckets.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +DRYRUN=1 + +source="$1" +dest="$2" + +if [[ -z $source || -z $dest ]]; then + echo 'usage: uploadbuckets.sh <source> <dest>' + exit 1 +fi + +if [[ ! -d $source ]]; then + echo 'error: source is not a directory' + exit 1 +fi + +if [[ -e $dest && ! -d $dest ]]; then + echo 'error: dest is not a directory' + exit 1 +fi + +if [[ ! -d $dest ]]; then + mkdir $dest +fi + +for package in "$source"/*; do + pkgname="${package##*/}" + newfolder="$dest/${pkgname:0:2}" + if [[ ! -d "$newfolder" ]]; then + if [[ DRYRUN -gt 0 ]]; then + echo mkdir -p "$newfolder" + else + mkdir -p "$newfolder" + fi + fi + if [[ DRYRUN -gt 0 ]]; then
Either need to use [[ $DRYRUN -gt 0 ]] or (( DRYRUN > 0 )), otherwise it's always false.
[[ ... ]] does arithmetic expansion as well so this should work as written. like: FOO=1 if [[ FOO -gt 0 ]]; then echo "YUP"; fi Kind of weird, imo, but I don't use [[ so I'm biased. -- -Justin
On Sun, Jul 31, 2011 at 01:20:18PM -0400, Justin Davis wrote:
On Fri, Jul 29, 2011 at 8:33 PM, Dave Reisner <d@falconindy.com> wrote:
On Thu, Jul 28, 2011 at 02:16:37PM -0500, Dan McGee wrote:
This goes with the previous patch that moves uploads into segmented subdirectories.
Signed-off-by: Dan McGee <dan@archlinux.org> --- scripts/uploadbuckets.sh | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 42 insertions(+), 0 deletions(-) create mode 100755 scripts/uploadbuckets.sh
diff --git a/scripts/uploadbuckets.sh b/scripts/uploadbuckets.sh new file mode 100755 index 0000000..1bb1a90 --- /dev/null +++ b/scripts/uploadbuckets.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +DRYRUN=1 + +source="$1" +dest="$2" + +if [[ -z $source || -z $dest ]]; then + echo 'usage: uploadbuckets.sh <source> <dest>' + exit 1 +fi + +if [[ ! -d $source ]]; then + echo 'error: source is not a directory' + exit 1 +fi + +if [[ -e $dest && ! -d $dest ]]; then + echo 'error: dest is not a directory' + exit 1 +fi + +if [[ ! -d $dest ]]; then + mkdir $dest +fi + +for package in "$source"/*; do + pkgname="${package##*/}" + newfolder="$dest/${pkgname:0:2}" + if [[ ! -d "$newfolder" ]]; then + if [[ DRYRUN -gt 0 ]]; then + echo mkdir -p "$newfolder" + else + mkdir -p "$newfolder" + fi + fi + if [[ DRYRUN -gt 0 ]]; then
Either need to use [[ $DRYRUN -gt 0 ]] or (( DRYRUN > 0 )), otherwise it's always false.
[[ ... ]] does arithmetic expansion as well so this should work as written. like:
FOO=1 if [[ FOO -gt 0 ]]; then echo "YUP"; fi
Kind of weird, imo, but I don't use [[ so I'm biased.
Yeah, this was new to me. Didn't realize variable expansion took place with -gt, -lt, -eq, etc... I'll keep using (( )) because its far more obvious at a glance. d
On Fri, Jul 29, 2011 at 7:22 PM, Lukas Fleischer <archlinux@cryptocrack.de> wrote:
On Thu, Jul 28, 2011 at 02:16:37PM -0500, Dan McGee wrote:
This goes with the previous patch that moves uploads into segmented subdirectories.
Signed-off-by: Dan McGee <dan@archlinux.org> --- scripts/uploadbuckets.sh | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 42 insertions(+), 0 deletions(-) create mode 100755 scripts/uploadbuckets.sh
diff --git a/scripts/uploadbuckets.sh b/scripts/uploadbuckets.sh new file mode 100755 index 0000000..1bb1a90 --- /dev/null +++ b/scripts/uploadbuckets.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +DRYRUN=1 + +source="$1" +dest="$2" + +if [[ -z $source || -z $dest ]]; then + echo 'usage: uploadbuckets.sh <source> <dest>' + exit 1 +fi + +if [[ ! -d $source ]]; then + echo 'error: source is not a directory' + exit 1 +fi + +if [[ -e $dest && ! -d $dest ]]; then + echo 'error: dest is not a directory' + exit 1 +fi + +if [[ ! -d $dest ]]; then + mkdir $dest +fi + +for package in "$source"/*; do + pkgname="${package##*/}" + newfolder="$dest/${pkgname:0:2}" + if [[ ! -d "$newfolder" ]]; then + if [[ DRYRUN -gt 0 ]]; then + echo mkdir -p "$newfolder" + else + mkdir -p "$newfolder" + fi + fi + if [[ DRYRUN -gt 0 ]]; then + echo mv "$source/$pkgname" "$newfolder/$pkgname" + else + mv "$source/$pkgname" "$newfolder/$pkgname"
This won't work if "$source" = "$dest" and there's a package name with length 2. We should probably add a check to assert that source and target directory are not equal.
Added in upcoming resubmit. There are a few more usability tweaks as well. I tested it using the following: mkdir /tmp/dir1 /tmp/dir2 cd /tmp/dir1 for name in $(~/projects/pacman/src/pacman/pacman -Qq); do mkdir $name; done mkdir aa zy DRYRUN=0 ./scripts/uploadbuckets.sh /tmp/dir1 /tmp/dir2 and then examining the contents of dir2. A few conditions worked- 'r' ended up in /r/r/ as expected; the two letter fake packages worked fine (and both aa and aalib were correctly in /aa/), and everything else checked out. -Dan
This goes with the previous patch that moves uploads into segmented subdirectories. To actually run, follow the DRYRUN instructions. Signed-off-by: Dan McGee <dan@archlinux.org> --- scripts/uploadbuckets.sh | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 42 insertions(+), 0 deletions(-) create mode 100755 scripts/uploadbuckets.sh diff --git a/scripts/uploadbuckets.sh b/scripts/uploadbuckets.sh new file mode 100755 index 0000000..1bb1a90 --- /dev/null +++ b/scripts/uploadbuckets.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +DRYRUN=1 + +source="$1" +dest="$2" + +if [[ -z $source || -z $dest ]]; then + echo 'usage: uploadbuckets.sh <source> <dest>' + exit 1 +fi + +if [[ ! -d $source ]]; then + echo 'error: source is not a directory' + exit 1 +fi + +if [[ -e $dest && ! -d $dest ]]; then + echo 'error: dest is not a directory' + exit 1 +fi + +if [[ ! -d $dest ]]; then + mkdir $dest +fi + +for package in "$source"/*; do + pkgname="${package##*/}" + newfolder="$dest/${pkgname:0:2}" + if [[ ! -d "$newfolder" ]]; then + if [[ DRYRUN -gt 0 ]]; then + echo mkdir -p "$newfolder" + else + mkdir -p "$newfolder" + fi + fi + if [[ DRYRUN -gt 0 ]]; then + echo mv "$source/$pkgname" "$newfolder/$pkgname" + else + mv "$source/$pkgname" "$newfolder/$pkgname" + fi +done -- 1.7.6
On Tue, Aug 09, 2011 at 05:31:27PM -0500, Dan McGee wrote:
This goes with the previous patch that moves uploads into segmented subdirectories. To actually run, follow the DRYRUN instructions.
Signed-off-by: Dan McGee <dan@archlinux.org> --- scripts/uploadbuckets.sh | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 42 insertions(+), 0 deletions(-) create mode 100755 scripts/uploadbuckets.sh
diff --git a/scripts/uploadbuckets.sh b/scripts/uploadbuckets.sh new file mode 100755 index 0000000..1bb1a90 --- /dev/null +++ b/scripts/uploadbuckets.sh @@ -0,0 +1,42 @@ +#!/bin/bash + +DRYRUN=1 + +source="$1" +dest="$2" + +if [[ -z $source || -z $dest ]]; then + echo 'usage: uploadbuckets.sh <source> <dest>' + exit 1 +fi + +if [[ ! -d $source ]]; then + echo 'error: source is not a directory' + exit 1 +fi + +if [[ -e $dest && ! -d $dest ]]; then + echo 'error: dest is not a directory' + exit 1 +fi + +if [[ ! -d $dest ]]; then + mkdir $dest +fi + +for package in "$source"/*; do + pkgname="${package##*/}" + newfolder="$dest/${pkgname:0:2}" + if [[ ! -d "$newfolder" ]]; then + if [[ DRYRUN -gt 0 ]]; then + echo mkdir -p "$newfolder" + else + mkdir -p "$newfolder" + fi + fi + if [[ DRYRUN -gt 0 ]]; then + echo mv "$source/$pkgname" "$newfolder/$pkgname" + else + mv "$source/$pkgname" "$newfolder/$pkgname" + fi +done
Hm, seems like you forgot to commit here (or probably forgot to `git add`/commit with "-a" since the commit message actually changed).
This goes with the previous patch that moves uploads into segmented subdirectories. To actually run, follow the DRYRUN instructions. Signed-off-by: Dan McGee <dan@archlinux.org> --- scripts/uploadbuckets.sh | 58 ++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 58 insertions(+), 0 deletions(-) create mode 100755 scripts/uploadbuckets.sh diff --git a/scripts/uploadbuckets.sh b/scripts/uploadbuckets.sh new file mode 100755 index 0000000..3252692 --- /dev/null +++ b/scripts/uploadbuckets.sh @@ -0,0 +1,58 @@ +#!/bin/bash + +DRYRUN=${DRYRUN:-1} + +source="$1" +dest="$2" + +if [[ -z $source || -z $dest ]]; then + echo 'usage: uploadbuckets.sh <source> <dest>' + echo 'Script runs in DRYRUN mode by default.' + echo 'To run for real, set DRYRUN=0 in your environment.' + exit 1 +fi + +if [[ ! -d $source ]]; then + echo 'error: source is not a directory' + exit 1 +fi + +if [[ -e $dest && ! -d $dest ]]; then + echo 'error: dest is not a directory' + exit 1 +fi + +if [[ $(readlink -e $dest) = $(readlink -e $source) ]]; then + echo 'error: source and dest cannot be the same. Rotate the result' + echo 'into place once the migration is complete.' + exit 1 +fi + +if [[ ! -d $dest ]]; then + mkdir $dest +fi + +shopt -s nullglob + +for package in "$source"/*; do + pkgname="${package##*/}" + newfolder="$dest/${pkgname:0:2}" + if [[ ! -d "$newfolder" ]]; then + if [[ $DRYRUN -gt 0 ]]; then + echo mkdir -p "$newfolder" + else + mkdir -p "$newfolder" + fi + fi + if [[ $DRYRUN -gt 0 ]]; then + echo mv "$source/$pkgname" "$newfolder/$pkgname" + else + mv "$source/$pkgname" "$newfolder/$pkgname" + fi +done + +if [[ $DRYRUN -gt 0 ]]; then + echo + echo 'DRYRUN mode was enabled.' + echo 'To run for real, set DRYRUN=0 in your environment.' +fi -- 1.7.6
participants (5)
-
Dan McGee
-
Dan McGee
-
Dave Reisner
-
Justin Davis
-
Lukas Fleischer