[arch-dev-public] [PATCH 1/3] sourceballs: Make cleanup more efficient
Moved all cleanup related code in sourceballs-cleanup script and moved,
now common, functions in db-functions. The cleanup script is now ran
after all the new sourceballs have been fetched.
Signed-off-by: Eric Bélanger
--- cron-jobs/sourceballs-cleanup | 61 ----------------------------------------- 1 files changed, 0 insertions(+), 61 deletions(-) delete mode 100755 cron-jobs/sourceballs-cleanup diff --git a/cron-jobs/sourceballs-cleanup b/cron-jobs/sourceballs-cleanup deleted file mode 100755 index c8d7b15..0000000 --- a/cron-jobs/sourceballs-cleanup +++ /dev/null @@ -1,61 +0,0 @@ -#!/bin/bash - -. "$(dirname $0)/../db-functions" -. "$(dirname $0)/../config" - -srcpath="$FTP_BASE/sources/" -logfile="$srcpath/cleanup.txt" - -LOCKFILE="/tmp/.sourceball-cleanup.lock" -WORKDIR="/tmp/sourceball-cleanup.$packagename.$UID" - -cleanup () { - restore_umask - rm -rf "$WORKDIR" - rm -f "$LOCKFILE" - exit 0 -} - -ctrl_c() { - echo "Interrupted" >&2 - cleanup 0 -} - -if [ -f "$LOCKFILE" ]; then - owner="$(/usr/bin/stat -c %U $LOCKFILE)" - echo "error: source tarball generation is already in progress (started by $owner)" - exit 1 -fi - -trap cleanup 0 -trap ctrl_c 2 - -/bin/touch "$LOCKFILE" - -#adjust the nice level to run at a lower priority -/usr/bin/renice +10 -p $$ > /dev/null - -set_umask -/bin/mkdir -p "$WORKDIR" -cd "$WORKDIR" - -[ -e "$logfile" ] && /bin/mv "$logfile" "$logfile.old" -echo "Orphaned sourceballs:" > "$logfile" - -for sourceball in "$srcpath"/*$SRCEXT; do - packagename=$(basename $sourceball) - packagename=${packagename%-*-*$SRCEXT} - - if ! /usr/bin/svn export -q --force "$SVNREPO/$packagename" "$packagename" >/dev/null 2>&1 ; then - echo "$packagename : no longer in svn. Removing sourceball." >> "$logfile" - mv $sourceball $SOURCE_CLEANUP_DESTDIR - elif [ -z "$(ls -A "$packagename/repos")" ]; then - echo "$packagename : no longer in repos but trunk is still in svn. Removing sourceball." >> "$logfile" - mv $sourceball $SOURCE_CLEANUP_DESTDIR - elif ! source "$packagename/trunk/$BUILDSCRIPT" && chk_license ${license[@]}; then - echo "$packagename : source hosting no longer required by license. Removing sourceball." >> "$logfile" - mv $sourceball $SOURCE_CLEANUP_DESTDIR - fi -done - -cleanup 0 -- 1.6.6.1
Signed-off-by: Eric Bélanger
Am Montag, 8. Februar 2010 16:13:44 schrieb Eric Bélanger:
- if ! /usr/bin/svn export -q --force "$SVNREPO/$packagename" "$packagename" >/dev/null 2>&1 ; then + if ! $(/usr/bin/svn export -q --force "$SVNREPO/$packagename" "$packagename" >/dev/null 2>&1) \ + && ! $(/usr/bin/svn export -q --force "$SVNREPOCOMMUNITY/$packagename" "$packagename" >/dev/null 2>&1) ; then
I think replacing "if command" with "if $(command)" is probably not what you want as you are no longer checking the return value but the output. Or I am just wrong here? -- Pierre Schmitz, https://users.archlinux.de/~pierre
On Sat, Mar 13, 2010 at 5:46 AM, Pierre Schmitz
Am Montag, 8. Februar 2010 16:13:44 schrieb Eric Bélanger:
- if ! /usr/bin/svn export -q --force "$SVNREPO/$packagename" "$packagename" >/dev/null 2>&1 ; then + if ! $(/usr/bin/svn export -q --force "$SVNREPO/$packagename" "$packagename" >/dev/null 2>&1) \ + && ! $(/usr/bin/svn export -q --force "$SVNREPOCOMMUNITY/$packagename" "$packagename" >/dev/null 2>&1) ; then
I think replacing "if command" with "if $(command)" is probably not what you want as you are no longer checking the return value but the output. Or I am just wrong here?
I'm not a bash expert but it seem to be doing the same thing: $ if ! ls >/dev/null 2>&1 ;then echo fail; fi $ if ! $(ls >/dev/null 2>&1) ;then echo fail; fi $ if ! foo >/dev/null 2>&1 ;then echo fail; fi fail $ if ! $(foo >/dev/null 2>&1) ;then echo fail; fi fail
--
Pierre Schmitz, https://users.archlinux.de/~pierre
On Sat, Mar 13, 2010 at 2:58 PM, Eric Bélanger
On Sat, Mar 13, 2010 at 5:46 AM, Pierre Schmitz
wrote: Am Montag, 8. Februar 2010 16:13:44 schrieb Eric Bélanger:
- if ! /usr/bin/svn export -q --force "$SVNREPO/$packagename" "$packagename" >/dev/null 2>&1 ; then + if ! $(/usr/bin/svn export -q --force "$SVNREPO/$packagename" "$packagename" >/dev/null 2>&1) \ + && ! $(/usr/bin/svn export -q --force "$SVNREPOCOMMUNITY/$packagename" "$packagename" >/dev/null 2>&1) ; then
I think replacing "if command" with "if $(command)" is probably not what you want as you are no longer checking the return value but the output. Or I am just wrong here?
I'm not a bash expert but it seem to be doing the same thing:
$ if ! ls >/dev/null 2>&1 ;then echo fail; fi $ if ! $(ls >/dev/null 2>&1) ;then echo fail; fi $ if ! foo >/dev/null 2>&1 ;then echo fail; fi fail $ if ! $(foo >/dev/null 2>&1) ;then echo fail; fi fail
This is a very bad example. Try not suppressing all output which is very misleading (why do we do this so damn often in scripts anyway?). dmcgee@galway /tmp/test $ if ! ls; then echo "fail"; fi onefile dmcgee@galway /tmp/test $ if ! $(ls); then echo "fail"; fi bash: onefile: command not found fail dmcgee@galway /tmp/test $ if ! foo; then echo "fail"; fi bash: foo: command not found fail dmcgee@galway /tmp/test $ if ! $(foo); then echo "fail"; fi bash: foo: command not found fail -Dan
Here's a fixed version of the patch. I just removed the parenthesis and it seems to be working without them. I don't remember why I added them originally.
Signed-off-by: Eric Bélanger
On Sun, Jun 27, 2010 at 11:40 PM, Eric Bélanger
Here's a fixed version of the patch. I just removed the parenthesis and it seems to be working without them. I don't remember why I added them originally.
FYI, the patch: http://mailman.archlinux.org/pipermail/arch-dev-public/2010-June/017249.html was submitted in the original ML thread: http://mailman.archlinux.org/pipermail/arch-dev-public/2010-February/015356.... This message was supposed to accompany it but git send-email has messed up.
On 28/06/10 13:47, Eric Bélanger wrote:
On Sun, Jun 27, 2010 at 11:40 PM, Eric Bélanger
wrote: Here's a fixed version of the patch. I just removed the parenthesis and it seems to be working without them. I don't remember why I added them originally.
FYI, the patch:
http://mailman.archlinux.org/pipermail/arch-dev-public/2010-June/017249.html
was submitted in the original ML thread: http://mailman.archlinux.org/pipermail/arch-dev-public/2010-February/015356....
This message was supposed to accompany it but git send-email has messed up.
For future reference, you can add comments directly in the patch after the "---" line and they will be ignored by git when applying. Allan
On Sun, 27 Jun 2010 23:40:38 -0400, Eric Bélanger
Signed-off-by: Eric Bélanger
--- config | 1 + cron-jobs/sourceballs | 2 +- misc-scripts/make-sourceball | 14 +++++++++++--- misc-scripts/sourceballs-cleanup | 5 +++-- 4 files changed, 16 insertions(+), 6 deletions(-) diff --git a/config b/config index 57ae64b..1cd533f 100644
applied, thanks. -- Pierre Schmitz, https://users.archlinux.de/~pierre
participants (4)
-
Allan McRae
-
Dan McGee
-
Eric Bélanger
-
Pierre Schmitz