On Thu, Jan 22, 2009 at 7:35 PM, Aaron Griffin wrote:

On Thu, Jan 22, 2009 at 7:00 PM, Aaron Griffin wrote:

...

On Thu, Jan 22, 2009 at 6:21 PM, Pierre Schmitz wrote:

...

In addition to this we'll need to update our devtools and db-scripts to handle those packages right. extrapkg should be quite simple; archrelease and repo- add should work just fine; only the db-scripts will really fail because they cannot guess the svn-dir just from the package name anymore.

Hmmm, this one I will have to think about. The devtools changes should be straightforward, but the db-scripts changes will not. Any ideas as to how we can convert a packagename to an svn-dir name? I'd rather not scan the whole repo...

Best I can do after some initial hacking. Are there any assumptions we can make based on package name to trim the list so we don't have to check all PKGBUILDs?

This is done without a checkout of the full repo, using svn list and svn cat...

$ ./pkgfind 3ddesktop #first package in the repos Repo scan complete in 51ms Package -> 3ddesktop/

$ ./pkgfind bash Repo scan complete in 3809ms Package -> bash/

$ ./pkgfind libpano13 #median package in the repos Repo scan complete in 44170ms Package -> libpano13/

$ ./pkgfind zsync #2nd to last package in the repos Repo scan complete in 90997ms Package -> zsync/

These numbers are just going to grow...

Optimization: If we assume the first letters match... $ ./pkgfind libpano13 Repo scan complete in 5555ms Package -> libpano13/ $ ./pkgfind zvbi Repo scan complete in 599ms Package -> zvbi/ $ ./pkgfind zsync Repo scan complete in 539ms Package -> zsync/ $ ./pkgfind yasm Repo scan complete in 235ms Package -> yasm/ $ ./pkgfind perlxml Repo scan complete in 3473ms Package -> perlxml/

Allan McRae wrote:

Aaron Griffin wrote:

...

On Thu, Jan 22, 2009 at 7:35 PM, Aaron Griffin wrote:

...

On Thu, Jan 22, 2009 at 7:00 PM, Aaron Griffin wrote:

...

On Thu, Jan 22, 2009 at 6:21 PM, Pierre Schmitz wrote:

...

In addition to this we'll need to update our devtools and db-scripts to handle those packages right. extrapkg should be quite simple; archrelease and repo- add should work just fine; only the db-scripts will really fail because they cannot guess the svn-dir just from the package name anymore.

Hmmm, this one I will have to think about. The devtools changes should be straightforward, but the db-scripts changes will not. Any ideas as to how we can convert a packagename to an svn-dir name? I'd rather not scan the whole repo...

Best I can do after some initial hacking. Are there any assumptions we can make based on package name to trim the list so we don't have to check all PKGBUILDs?

This is done without a checkout of the full repo, using svn list and svn cat...

$ ./pkgfind 3ddesktop #first package in the repos Repo scan complete in 51ms Package -> 3ddesktop/

$ ./pkgfind bash Repo scan complete in 3809ms Package -> bash/

$ ./pkgfind libpano13 #median package in the repos Repo scan complete in 44170ms Package -> libpano13/

$ ./pkgfind zsync #2nd to last package in the repos Repo scan complete in 90997ms Package -> zsync/

These numbers are just going to grow...

Optimization: If we assume the first letters match...

$ ./pkgfind libpano13 Repo scan complete in 5555ms Package -> libpano13/

$ ./pkgfind zvbi Repo scan complete in 599ms Package -> zvbi/

$ ./pkgfind zsync Repo scan complete in 539ms Package -> zsync/

$ ./pkgfind yasm Repo scan complete in 235ms Package -> yasm/

$ ./pkgfind perlxml Repo scan complete in 3473ms Package -> perlxml/

Thomas suggested using symlinks in the svn directory? From his email: e.g. mysql/trunk mysql/repos/mysql/extra-{i686,x86_64} mysql/repos/mysql-clients/extra-{i686,x86_64} mysql/repos/mysql-libs/extra-{i686,x86_64} mysql-clients -> mysql mysql-libs -> mysql

That should solve most of the trouble here.

I suppose I should clarify how I thought this would work. The extrapkg etc. scripts just loop over ${pkgname[*]}, and commit it to the "main" directory, being whatever the package puts it in. The packager would need to make the symlinks for their other packages, much like we currently have to set up pkgname/{trunk,repo} for any new package. Then there should not need to be any changes in the db-scripts (I think...). Allan

Aaron Griffin wrote:

On Thu, Jan 22, 2009 at 8:03 PM, Allan McRae wrote:

...

Allan McRae wrote:

...

Aaron Griffin wrote:

...

On Thu, Jan 22, 2009 at 7:35 PM, Aaron Griffin wrote:

...

On Thu, Jan 22, 2009 at 7:00 PM, Aaron Griffin wrote:

...

On Thu, Jan 22, 2009 at 6:21 PM, Pierre Schmitz wrote:

> In addition to this we'll need to update our devtools and db-scripts > to handle > those packages right. extrapkg should be quite simple; archrelease and > repo- > add should work just fine; only the db-scripts will really fail > because they > cannot guess the svn-dir just from the package name anymore. > > Hmmm, this one I will have to think about. The devtools changes should be straightforward, but the db-scripts changes will not. Any ideas as to how we can convert a packagename to an svn-dir name? I'd rather not scan the whole repo...

Best I can do after some initial hacking. Are there any assumptions we can make based on package name to trim the list so we don't have to check all PKGBUILDs?

This is done without a checkout of the full repo, using svn list and svn cat...

$ ./pkgfind 3ddesktop #first package in the repos Repo scan complete in 51ms Package -> 3ddesktop/

$ ./pkgfind bash Repo scan complete in 3809ms Package -> bash/

$ ./pkgfind libpano13 #median package in the repos Repo scan complete in 44170ms Package -> libpano13/

$ ./pkgfind zsync #2nd to last package in the repos Repo scan complete in 90997ms Package -> zsync/

These numbers are just going to grow...

Optimization: If we assume the first letters match...

$ ./pkgfind libpano13 Repo scan complete in 5555ms Package -> libpano13/

$ ./pkgfind zvbi Repo scan complete in 599ms Package -> zvbi/

$ ./pkgfind zsync Repo scan complete in 539ms Package -> zsync/

$ ./pkgfind yasm Repo scan complete in 235ms Package -> yasm/

$ ./pkgfind perlxml Repo scan complete in 3473ms Package -> perlxml/

Thomas suggested using symlinks in the svn directory? From his email: e.g. mysql/trunk mysql/repos/mysql/extra-{i686,x86_64} mysql/repos/mysql-clients/extra-{i686,x86_64} mysql/repos/mysql-libs/extra-{i686,x86_64} mysql-clients -> mysql mysql-libs -> mysql

That should solve most of the trouble here.

I suppose I should clarify how I thought this would work. The extrapkg etc. scripts just loop over ${pkgname[*]}, and commit it to the "main" directory, being whatever the package puts it in. The packager would need to make the symlinks for their other packages, much like we currently have to set up pkgname/{trunk,repo} for any new package. Then there should not need to be any changes in the db-scripts (I think...).

Yes, the db-scripts are the complication.

For a given package, foo, the db-scripts check for svn-packages/foot/repos/$REPO-$ARCH to ensure that the package should be in this repo and that the version in svn is the same as the package.

Because we can have svn-packages/foo/trunk/PKGBUILD build a package named bar, the above isn't as simple as it once was. We NOW need to scan through all PKGBUILDs when we look for the proper version for a package named "bar"

I am missing something here. Why can't it just check for svn-packages/bar/repos/$REPO-$ARCH once the symlinks are added in svn? Maybe have extrapkg check that the symlinks are created and do it automatically if not. Allan

Aaron Griffin wrote:

On Thu, Jan 22, 2009 at 8:51 PM, Allan McRae wrote:

...

Aaron Griffin wrote:

...

On Thu, Jan 22, 2009 at 8:03 PM, Allan McRae wrote:

...

Allan McRae wrote:

...

Aaron Griffin wrote:

...

On Thu, Jan 22, 2009 at 7:35 PM, Aaron Griffin wrote:

> On Thu, Jan 22, 2009 at 7:00 PM, Aaron Griffin > > wrote: > > > >> On Thu, Jan 22, 2009 at 6:21 PM, Pierre Schmitz >> wrote: >> >> >> >>> In addition to this we'll need to update our devtools and db-scripts >>> to handle >>> those packages right. extrapkg should be quite simple; archrelease >>> and >>> repo- >>> add should work just fine; only the db-scripts will really fail >>> because they >>> cannot guess the svn-dir just from the package name anymore. >>> >>> >>> >> Hmmm, this one I will have to think about. The devtools changes >> should >> be straightforward, but the db-scripts changes will not. Any ideas as >> to how we can convert a packagename to an svn-dir name? I'd rather >> not >> scan the whole repo... >> >> >> > Best I can do after some initial hacking. Are there any assumptions we > can make based on package name to trim the list so we don't have to > check all PKGBUILDs? > > This is done without a checkout of the full repo, using svn list and > svn > cat... > > $ ./pkgfind 3ddesktop #first package in the repos > Repo scan complete in 51ms > Package -> 3ddesktop/ > > $ ./pkgfind bash > Repo scan complete in 3809ms > Package -> bash/ > > $ ./pkgfind libpano13 #median package in the repos > Repo scan complete in 44170ms > Package -> libpano13/ > > $ ./pkgfind zsync #2nd to last package in the repos > Repo scan complete in 90997ms > Package -> zsync/ > > These numbers are just going to grow... > > > > Optimization: If we assume the first letters match...

$ ./pkgfind libpano13 Repo scan complete in 5555ms Package -> libpano13/

$ ./pkgfind zvbi Repo scan complete in 599ms Package -> zvbi/

$ ./pkgfind zsync Repo scan complete in 539ms Package -> zsync/

$ ./pkgfind yasm Repo scan complete in 235ms Package -> yasm/

$ ./pkgfind perlxml Repo scan complete in 3473ms Package -> perlxml/

Thomas suggested using symlinks in the svn directory? From his email: e.g. mysql/trunk mysql/repos/mysql/extra-{i686,x86_64} mysql/repos/mysql-clients/extra-{i686,x86_64} mysql/repos/mysql-libs/extra-{i686,x86_64} mysql-clients -> mysql mysql-libs -> mysql

That should solve most of the trouble here.

I suppose I should clarify how I thought this would work. The extrapkg etc. scripts just loop over ${pkgname[*]}, and commit it to the "main" directory, being whatever the package puts it in. The packager would need to make the symlinks for their other packages, much like we currently have to set up pkgname/{trunk,repo} for any new package. Then there should not need to be any changes in the db-scripts (I think...).

Yes, the db-scripts are the complication.

For a given package, foo, the db-scripts check for svn-packages/foot/repos/$REPO-$ARCH to ensure that the package should be in this repo and that the version in svn is the same as the package.

Because we can have svn-packages/foo/trunk/PKGBUILD build a package named bar, the above isn't as simple as it once was. We NOW need to scan through all PKGBUILDs when we look for the proper version for a package named "bar"

I am missing something here. Why can't it just check for svn-packages/bar/repos/$REPO-$ARCH once the symlinks are added in svn? Maybe have extrapkg check that the symlinks are created and do it automatically if not.

Oh no, *I* missed it... the symlinks are top level. I get it now.

That could be because of the example I cut from Thomas' email. The extra folder level in the repos directory is unneed because the PKGBUILD and source are the same for all the split packages. It should be like: mysql/repos/extra-{i686,x86_64} mysql-clients -> mysql mysql-libs -> mysql Allan