Re: [arch-dev-public] Time to go forward with arch=any?
Hope you like it ;)
Applied and pulled new changes to /arch-new
Magnifico maestro! [francois@gerolde ~]$ cd /arch-new/misc-scripts/ [francois@gerolde misc-scripts]$ ./ftpdir-cleanup testing /srv/package-cleanup Restoring missing 'any' symlink: mkinitcpio-0.5.26-1-any.pkg.tar.gz Restoring missing 'any' symlink: texlive-latex3-2009.14222-1-any.pkg.tar.gz Restoring missing 'any' symlink: texlive-science-2009.13945-1-any.pkg.tar.gz Scan complete for testing (i686) at /srv/ftp/testing/os/i686 (nothing to do for i686) Restoring missing 'any' symlink: mkinitcpio-0.5.26-1-any.pkg.tar.gz Restoring missing 'any' symlink: texlive-latex3-2009.14222-1-any.pkg.tar.gz Restoring missing 'any' symlink: texlive-science-2009.13945-1-any.pkg.tar.gz Scan complete for testing (x86_64) at /srv/ftp/testing/os/x86_64 (nothing to do for x86_64) Checking arch-independent files... Yay, it works! I have also tested other scenarios at home on a custom repo (extra file, out-of-date file, etc) without any glitch. Unless someone objects, I guess you can now update the cronjob for ftpdir-cleanup. F
On Tue, Jul 21, 2009 at 1:03 PM, Firmicus<Firmicus@gmx.net> wrote:
Hope you like it ;)
Applied and pulled new changes to /arch-new
Magnifico maestro!
[francois@gerolde ~]$ cd /arch-new/misc-scripts/ [francois@gerolde misc-scripts]$ ./ftpdir-cleanup testing /srv/package-cleanup Restoring missing 'any' symlink: mkinitcpio-0.5.26-1-any.pkg.tar.gz Restoring missing 'any' symlink: texlive-latex3-2009.14222-1-any.pkg.tar.gz Restoring missing 'any' symlink: texlive-science-2009.13945-1-any.pkg.tar.gz Scan complete for testing (i686) at /srv/ftp/testing/os/i686 (nothing to do for i686) Restoring missing 'any' symlink: mkinitcpio-0.5.26-1-any.pkg.tar.gz Restoring missing 'any' symlink: texlive-latex3-2009.14222-1-any.pkg.tar.gz Restoring missing 'any' symlink: texlive-science-2009.13945-1-any.pkg.tar.gz Scan complete for testing (x86_64) at /srv/ftp/testing/os/x86_64 (nothing to do for x86_64) Checking arch-independent files...
Yay, it works!
I have also tested other scenarios at home on a custom repo (extra file, out-of-date file, etc) without any glitch.
Unless someone objects, I guess you can now update the cronjob for ftpdir-cleanup.
Ok, one question - if you run this on core or extra.... does it work? That's how I nearly blew up core (thankfully, you can just cp the packages out of package-cleanup)
Aaron wrote:
Ok, one question - if you run this on core or extra.... does it work? That's how I nearly blew up core (thankfully, you can just cp the packages out of package-cleanup)
Well, I am 95% sure it would, but only a test run would demonstrate it. Should I try? ;)
On Tue, Jul 21, 2009 at 2:25 PM, Firmicus<Firmicus@gmx.net> wrote:
Aaron wrote:
Ok, one question - if you run this on core or extra.... does it work? That's how I nearly blew up core (thankfully, you can just cp the packages out of package-cleanup)
Well, I am 95% sure it would, but only a test run would demonstrate it.
Should I try? ;)
You could do what I did: put an "exit 0" right before it does the actual moves
Firmicus a écrit :
Aaron wrote:
Ok, one question - if you run this on core or extra.... does it work? That's how I nearly blew up core (thankfully, you can just cp the packages out of package-cleanup)
Well, I am 95% sure it would, but only a test run would demonstrate it.
Should I try? ;)
I just copied /arch-new/misc-scripts/ftpdir-cleanup to ~francois/bin/misc-scripts/ftpdir-fakecleanup where I have commented out the "dangerous" actions (mv, ln -s, rm). Turns out it ran just fine on extra: [francois@gerolde bin]$ misc-scripts/ftpdir-fakecleanup extra /srv/package-cleanup Scan complete for extra (i686) at /srv/ftp/extra/os/i686 (nothing to do for i686) Scan complete for extra (x86_64) at /srv/ftp/extra/os/x86_64 (nothing to do for x86_64) Checking arch-independent files... and likewise with extra. So, here we go!
Firmicus a écrit :
I just copied /arch-new/misc-scripts/ftpdir-cleanup to ~francois/bin/misc-scripts/ftpdir-fakecleanup where I have commented out the "dangerous" actions (mv, ln -s, rm). Turns out it ran just fine on extra:
[francois@gerolde bin]$ misc-scripts/ftpdir-fakecleanup extra /srv/package-cleanup Scan complete for extra (i686) at /srv/ftp/extra/os/i686 (nothing to do for i686) Scan complete for extra (x86_64) at /srv/ftp/extra/os/x86_64 (nothing to do for x86_64) Checking arch-independent files...
and likewise with extra.
I meant, likewise with *core*
On Tue, Jul 21, 2009 at 2:38 PM, Firmicus<Firmicus@gmx.net> wrote:
Firmicus a écrit :
I just copied /arch-new/misc-scripts/ftpdir-cleanup to ~francois/bin/misc-scripts/ftpdir-fakecleanup where I have commented out the "dangerous" actions (mv, ln -s, rm). Turns out it ran just fine on extra:
[francois@gerolde bin]$ misc-scripts/ftpdir-fakecleanup extra /srv/package-cleanup Scan complete for extra (i686) at /srv/ftp/extra/os/i686 (nothing to do for i686) Scan complete for extra (x86_64) at /srv/ftp/extra/os/x86_64 (nothing to do for x86_64) Checking arch-independent files...
and likewise with extra.
I meant, likewise with *core*
Ok, cron is using the cleanup script from arch-new so we don't blow away the 'any' symlinks while testing this. Please continue to try out any package support. When we get split support completed, I'll push this to /arch and the community server
Aaron Griffin a écrit :
Ok, cron is using the cleanup script from arch-new so we don't blow away the 'any' symlinks while testing this.
Good. Thanks
Please continue to try out any package support. When we get split support completed, I'll push this to /arch and the community server
Just added some more packages. Everything works as expected, but we still have that spurious error message from repo_unlock (probably from a "trapped" cleanup?) at the end. Strange... [francois@gerolde ~]$ /arch-new/db-update testing ==> Processing 3 new/updated arch-independent packages for 'testing'... Checked out revision 46920. Validating package arch (any) texlive-bibtexextra Checking SVN for texlive-bibtexextra Validating package arch (any) texlive-pictures Checking SVN for texlive-pictures Validating package arch (any) texlive-pstricks Checking SVN for texlive-pstricks Updating DB for testing-i686 ==> Copying DB file from 'testing'... ==> Processing 3 new/updated packages for repository 'testing'... Checked out revision 46920. Copying new files to '/srv/ftp/testing/os/any' and symlinking Updating DB for testing-x86_64 ==> Copying DB file from 'testing'... ==> Processing 3 new/updated packages for repository 'testing'... Checked out revision 46920. Copying new files to '/srv/ftp/testing/os/any' and symlinking error: repo lock doesn't exist... something went terribly wrong! <<<<<<<<<<<<<<<<<<< But as I said, all files and symlinks are there: [francois@gerolde ~]$ ls -Rl /srv/ftp/testing/os/*/*-any.* -rw-r--r-- 1 thomas ftp-extra 16067 2009-07-20 19:07 /srv/ftp/testing/os/any/mkinitcpio-0.5.26-1-any.pkg.tar.gz -rw-r--r-- 1 francois ftp-extra 1400167 2009-07-21 15:51 /srv/ftp/testing/os/any/texlive-bibtexextra-2009.14259-1-any.pkg.tar.gz -rw-r--r-- 1 francois ftp-extra 59983 2009-07-20 13:15 /srv/ftp/testing/os/any/texlive-latex3-2009.14222-1-any.pkg.tar.gz -rw-r--r-- 1 francois ftp-extra 825965 2009-07-21 15:51 /srv/ftp/testing/os/any/texlive-pictures-2009.14223-1-any.pkg.tar.gz -rw-r--r-- 1 francois ftp-extra 1019771 2009-07-21 15:51 /srv/ftp/testing/os/any/texlive-pstricks-2009.13822-1-any.pkg.tar.gz -rw-r--r-- 1 francois ftp-extra 386464 2009-07-20 06:00 /srv/ftp/testing/os/any/texlive-science-2009.13945-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 41 2009-07-21 15:24 /srv/ftp/testing/os/i686/mkinitcpio-0.5.26-1-any.pkg.tar.gz -> ../any/mkinitcpio-0.5.26-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 54 2009-07-21 15:51 /srv/ftp/testing/os/i686/texlive-bibtexextra-2009.14259-1-any.pkg.tar.gz -> ../any/texlive-bibtexextra-2009.14259-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 49 2009-07-21 15:24 /srv/ftp/testing/os/i686/texlive-latex3-2009.14222-1-any.pkg.tar.gz -> ../any/texlive-latex3-2009.14222-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 51 2009-07-21 15:51 /srv/ftp/testing/os/i686/texlive-pictures-2009.14223-1-any.pkg.tar.gz -> ../any/texlive-pictures-2009.14223-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 51 2009-07-21 15:51 /srv/ftp/testing/os/i686/texlive-pstricks-2009.13822-1-any.pkg.tar.gz -> ../any/texlive-pstricks-2009.13822-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 50 2009-07-21 15:24 /srv/ftp/testing/os/i686/texlive-science-2009.13945-1-any.pkg.tar.gz -> ../any/texlive-science-2009.13945-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 41 2009-07-21 15:24 /srv/ftp/testing/os/x86_64/mkinitcpio-0.5.26-1-any.pkg.tar.gz -> ../any/mkinitcpio-0.5.26-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 54 2009-07-21 15:51 /srv/ftp/testing/os/x86_64/texlive-bibtexextra-2009.14259-1-any.pkg.tar.gz -> ../any/texlive-bibtexextra-2009.14259-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 49 2009-07-21 15:24 /srv/ftp/testing/os/x86_64/texlive-latex3-2009.14222-1-any.pkg.tar.gz -> ../any/texlive-latex3-2009.14222-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 51 2009-07-21 15:51 /srv/ftp/testing/os/x86_64/texlive-pictures-2009.14223-1-any.pkg.tar.gz -> ../any/texlive-pictures-2009.14223-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 51 2009-07-21 15:51 /srv/ftp/testing/os/x86_64/texlive-pstricks-2009.13822-1-any.pkg.tar.gz -> ../any/texlive-pstricks-2009.13822-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 50 2009-07-21 15:24 /srv/ftp/testing/os/x86_64/texlive-science-2009.13945-1-any.pkg.tar.gz -> ../any/texlive-science-2009.13945-1-any.pkg.tar.gz
On Tue, Jul 21, 2009 at 3:00 PM, Firmicus<Firmicus@gmx.net> wrote:
Aaron Griffin a écrit :
Ok, cron is using the cleanup script from arch-new so we don't blow away the 'any' symlinks while testing this.
Good. Thanks
Please continue to try out any package support. When we get split support completed, I'll push this to /arch and the community server
Just added some more packages. Everything works as expected, but we still have that spurious error message from repo_unlock (probably from a "trapped" cleanup?) at the end. Strange...
[francois@gerolde ~]$ /arch-new/db-update testing ==> Processing 3 new/updated arch-independent packages for 'testing'... Checked out revision 46920. Validating package arch (any) texlive-bibtexextra Checking SVN for texlive-bibtexextra Validating package arch (any) texlive-pictures Checking SVN for texlive-pictures Validating package arch (any) texlive-pstricks Checking SVN for texlive-pstricks Updating DB for testing-i686 ==> Copying DB file from 'testing'... ==> Processing 3 new/updated packages for repository 'testing'... Checked out revision 46920. Copying new files to '/srv/ftp/testing/os/any' and symlinking Updating DB for testing-x86_64 ==> Copying DB file from 'testing'... ==> Processing 3 new/updated packages for repository 'testing'... Checked out revision 46920. Copying new files to '/srv/ftp/testing/os/any' and symlinking error: repo lock doesn't exist... something went terribly wrong! <<<<<<<<<<<<<<<<<<<
But as I said, all files and symlinks are there:
[francois@gerolde ~]$ ls -Rl /srv/ftp/testing/os/*/*-any.* -rw-r--r-- 1 thomas ftp-extra 16067 2009-07-20 19:07 /srv/ftp/testing/os/any/mkinitcpio-0.5.26-1-any.pkg.tar.gz -rw-r--r-- 1 francois ftp-extra 1400167 2009-07-21 15:51 /srv/ftp/testing/os/any/texlive-bibtexextra-2009.14259-1-any.pkg.tar.gz -rw-r--r-- 1 francois ftp-extra 59983 2009-07-20 13:15 /srv/ftp/testing/os/any/texlive-latex3-2009.14222-1-any.pkg.tar.gz -rw-r--r-- 1 francois ftp-extra 825965 2009-07-21 15:51 /srv/ftp/testing/os/any/texlive-pictures-2009.14223-1-any.pkg.tar.gz -rw-r--r-- 1 francois ftp-extra 1019771 2009-07-21 15:51 /srv/ftp/testing/os/any/texlive-pstricks-2009.13822-1-any.pkg.tar.gz -rw-r--r-- 1 francois ftp-extra 386464 2009-07-20 06:00 /srv/ftp/testing/os/any/texlive-science-2009.13945-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 41 2009-07-21 15:24 /srv/ftp/testing/os/i686/mkinitcpio-0.5.26-1-any.pkg.tar.gz -> ../any/mkinitcpio-0.5.26-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 54 2009-07-21 15:51 /srv/ftp/testing/os/i686/texlive-bibtexextra-2009.14259-1-any.pkg.tar.gz -> ../any/texlive-bibtexextra-2009.14259-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 49 2009-07-21 15:24 /srv/ftp/testing/os/i686/texlive-latex3-2009.14222-1-any.pkg.tar.gz -> ../any/texlive-latex3-2009.14222-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 51 2009-07-21 15:51 /srv/ftp/testing/os/i686/texlive-pictures-2009.14223-1-any.pkg.tar.gz -> ../any/texlive-pictures-2009.14223-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 51 2009-07-21 15:51 /srv/ftp/testing/os/i686/texlive-pstricks-2009.13822-1-any.pkg.tar.gz -> ../any/texlive-pstricks-2009.13822-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 50 2009-07-21 15:24 /srv/ftp/testing/os/i686/texlive-science-2009.13945-1-any.pkg.tar.gz -> ../any/texlive-science-2009.13945-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 41 2009-07-21 15:24 /srv/ftp/testing/os/x86_64/mkinitcpio-0.5.26-1-any.pkg.tar.gz -> ../any/mkinitcpio-0.5.26-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 54 2009-07-21 15:51 /srv/ftp/testing/os/x86_64/texlive-bibtexextra-2009.14259-1-any.pkg.tar.gz -> ../any/texlive-bibtexextra-2009.14259-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 49 2009-07-21 15:24 /srv/ftp/testing/os/x86_64/texlive-latex3-2009.14222-1-any.pkg.tar.gz -> ../any/texlive-latex3-2009.14222-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 51 2009-07-21 15:51 /srv/ftp/testing/os/x86_64/texlive-pictures-2009.14223-1-any.pkg.tar.gz -> ../any/texlive-pictures-2009.14223-1-any.pkg.tar.gz lrwxrwxrwx 1 francois ftp-extra 51 2009-07-21 15:51 /srv/ftp/testing/os/x86_64/texlive-pstricks-2009.13822-1-any.pkg.tar.gz -> ../any/texlive-pstricks-2009.13822-1-any.pkg.tar.gz lrwxrwxrwx 1 ftp ftp-extra 50 2009-07-21 15:24 /srv/ftp/testing/os/x86_64/texlive-science-2009.13945-1-any.pkg.tar.gz -> ../any/texlive-science-2009.13945-1-any.pkg.tar.gz
Added some output to the lock/unlock functions to see if it helps illustrate the issue better I also fiddled with the cleanup function in db-update a bit, so it only calls repo_unlock on an error. Didnt commit the changes, but lets see if that helps
Added some output to the lock/unlock functions to see if it helps illustrate the issue better
I also fiddled with the cleanup function in db-update a bit, so it only calls repo_unlock on an error. Didnt commit the changes, but lets see if that helps
I think I fixed it. The thing is that through trap cleanup 0 the function cleanup() is called automatically at the end of the script. And since cleanup in turn calls repo_unlock $reponame $current_arch the latter fails because $current_arch is "x86_64" and we have already unlocked each repo before the end of the for loop, resulting in a spurious error message. Replacing repo_unlock by rm -f $TMPDIR/.repolock.$reponame.* in cleanup() solves this. (It might be actually better to have a function delete_repolocks() in db-function that does this...) I have also moved a few lines down within the if [ -n "$ANYPKGS" ]; then <...> fi construct. Otherwise one would get the message ==> Processing 0 new/updated arch-independent packages for '$reponame'... even when no package with arch=any is present in staging See attached patch... Gute Nacht, F
participants (2)
-
Aaron Griffin
-
Firmicus