[arch-dev-public] dbscripts pkg pools

[Aaron Griffin]

On Wed, Apr 28, 2010 at 12:21 PM, Pierre Schmitz <pierre at archlinux.de> wrote:
> On Wed, 28 Apr 2010 11:56:53 -0500, Aaron Griffin
> <aaronmgriffin at gmail.com> wrote:
>> Hello all,
>> I've had these patches sitting around for a bit:
>> http://code.phraktured.net/cgit.cgi/dbscripts/log/?h=pkgpools
>>
>> I'd like to see if someone else has the time to test them.
>>
>> I have tested them with the following procedure:
>> * created ./testing/{staging,myrepo,packages,package-cleanup}/ with
>> necessary subdirs
>> * modified config to point to these paths
>> * commented out all the SVN malarky in
> db-add/db-remove/ftpdir-cleanup/etc
>> * changed get_repos_for_host() to return "myrepo"
>>
>> Testing:
>> * Add a package to myrepo
>>    - Package file in packages/os/*/
>>    - Relative symlink in myrepo/os/*/
>>    - In myrepo DB
>> * Remove package from repo 'foo'
>>    - Removed from myrepo DB
>> * Run cron-jobs/ftpdir-cleanup
>>    - Symlink is removed from myrepo/os/*/
>>    - Package file moved from packages/os/*/ to package-cleanup/
>>
>> Anyone else want to verify this, or should I push it to master?
>
> Thanks for your heads-up. That was on my todo list but atm I am not really
> able to catch up. But those patches are a lot more readable than those from
> your working branch which virtually changed every line of code. I'll have a
> closer look at those soon.

Yeah, there's two patches in the working branch that are totally
unrelated, so I dumped them for the purposes of these changes.

> For now:
> * Does this also work with the old layout/packages? (especially the
> cleanup and move scripts)

No. What will have to happen is that all packages will need to be
moved to /srv/ftp/packages/os/*/ and symlinked to where they actually
end up (with relative symlinks, for the purposes of rsync). This can
be scripted away fairly quickly.

> * I still don't know how to handle the community and community-testing
> repo. We cannot really put those packages into the same folder as
> core,extra,testing. That could break the cleanup-scripts (different svn)
> and definitely will break the rsync process from sigurd to gerolde.

Hmm, if the new DB scripts are placed on there, it will have it's own
/srv/ftp/packages dir and relative symlinks. This is going to be very
hard to rsync in.

One possibility is to move all of community to a subtree in our ftp
layout, i.e. /srv/ftp/repos/other/{community, ...}. This would be a
fairly drastic change, but would make this easier.

> * Do all packages have the $arch within their filename? (community might
> be a problem here as they are not using db-scripts for that long)

What is this referencing? Is this related to the ARCHES loop I added?