[arch-dev-public] Divide and Konquer - Splitting KDE packages

[Allan McRae]

Aaron Griffin wrote:
> On Thu, Jan 22, 2009 at 8:51 PM, Allan McRae <allan at archlinux.org> wrote:
>   
>> Aaron Griffin wrote:
>>     
>>> On Thu, Jan 22, 2009 at 8:03 PM, Allan McRae <allan at archlinux.org> wrote:
>>>
>>>       
>>>> Allan McRae wrote:
>>>>
>>>>         
>>>>> Aaron Griffin wrote:
>>>>>
>>>>>           
>>>>>> On Thu, Jan 22, 2009 at 7:35 PM, Aaron Griffin
>>>>>> <aaronmgriffin at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> On Thu, Jan 22, 2009 at 7:00 PM, Aaron Griffin
>>>>>>> <aaronmgriffin at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> On Thu, Jan 22, 2009 at 6:21 PM, Pierre Schmitz <pierre at archlinux.de>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> In addition to this we'll need to update our devtools and db-scripts
>>>>>>>>> to handle
>>>>>>>>> those packages right. extrapkg should be quite simple; archrelease
>>>>>>>>> and
>>>>>>>>> repo-
>>>>>>>>> add should work just fine; only the db-scripts will really fail
>>>>>>>>> because they
>>>>>>>>> cannot guess the svn-dir just from the package name anymore.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> Hmmm, this one I will have to think about. The devtools changes
>>>>>>>> should
>>>>>>>> be straightforward, but the db-scripts changes will not. Any ideas as
>>>>>>>> to how we can convert a packagename to an svn-dir name? I'd rather
>>>>>>>> not
>>>>>>>> scan the whole repo...
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> Best I can do after some initial hacking. Are there any assumptions we
>>>>>>> can make based on package name to trim the list so we don't have to
>>>>>>> check all PKGBUILDs?
>>>>>>>
>>>>>>> This is done without a checkout of the full repo, using svn list and
>>>>>>> svn
>>>>>>> cat...
>>>>>>>
>>>>>>> $ ./pkgfind 3ddesktop #first package in the repos
>>>>>>> Repo scan complete in 51ms
>>>>>>> Package -> 3ddesktop/
>>>>>>>
>>>>>>> $ ./pkgfind bash
>>>>>>> Repo scan complete in 3809ms
>>>>>>> Package -> bash/
>>>>>>>
>>>>>>> $ ./pkgfind libpano13 #median package in the repos
>>>>>>> Repo scan complete in 44170ms
>>>>>>> Package -> libpano13/
>>>>>>>
>>>>>>> $ ./pkgfind zsync #2nd to last package in the repos
>>>>>>> Repo scan complete in 90997ms
>>>>>>> Package -> zsync/
>>>>>>>
>>>>>>> These numbers are just going to grow...
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> Optimization: If we assume the first letters match...
>>>>>>
>>>>>> $ ./pkgfind libpano13
>>>>>> Repo scan complete in 5555ms
>>>>>> Package -> libpano13/
>>>>>>
>>>>>> $ ./pkgfind zvbi
>>>>>> Repo scan complete in 599ms
>>>>>> Package -> zvbi/
>>>>>>
>>>>>> $ ./pkgfind zsync
>>>>>> Repo scan complete in 539ms
>>>>>> Package -> zsync/
>>>>>>
>>>>>> $ ./pkgfind yasm
>>>>>> Repo scan complete in 235ms
>>>>>> Package -> yasm/
>>>>>>
>>>>>> $ ./pkgfind perlxml
>>>>>> Repo scan complete in 3473ms
>>>>>> Package -> perlxml/
>>>>>>
>>>>>>
>>>>>>             
>>>>> Thomas suggested using symlinks in the svn directory?
>>>>> From his email: e.g. mysql/trunk
>>>>> mysql/repos/mysql/extra-{i686,x86_64}
>>>>> mysql/repos/mysql-clients/extra-{i686,x86_64}
>>>>> mysql/repos/mysql-libs/extra-{i686,x86_64}
>>>>> mysql-clients -> mysql
>>>>> mysql-libs -> mysql
>>>>>
>>>>> That should solve most of the trouble here.
>>>>>
>>>>>
>>>>>           
>>>> I suppose I should clarify how I thought this would work.   The extrapkg
>>>> etc. scripts just loop over ${pkgname[*]}, and commit it to the "main"
>>>> directory, being whatever the package puts it in.  The packager would
>>>> need
>>>> to make the symlinks for their other packages, much like we currently
>>>> have
>>>> to set up pkgname/{trunk,repo} for any new package.  Then there should
>>>> not
>>>> need to be any changes in the db-scripts (I think...).
>>>>
>>>>         
>>> Yes, the db-scripts are the complication.
>>>
>>> For a given package, foo, the db-scripts check for
>>> svn-packages/foot/repos/$REPO-$ARCH to ensure that the package should
>>> be in this repo and that the version in svn is the same as the
>>> package.
>>>
>>> Because we can have svn-packages/foo/trunk/PKGBUILD build a package
>>> named bar, the above isn't as simple as it once was. We NOW need to
>>> scan through all PKGBUILDs when we look for the proper version for a
>>> package named "bar"
>>>
>>>       
>> I am missing something here.  Why can't it just check for
>> svn-packages/bar/repos/$REPO-$ARCH once the symlinks are added in svn?
>>  Maybe have extrapkg check that the symlinks are created and do it
>> automatically if not.
>>     
>
> Oh no, *I* missed it... the symlinks are top level. I get it now.
>   

That could be because of the example I cut from Thomas' email. The extra 
folder level in the repos directory is unneed because the PKGBUILD and 
source are the same for all the split packages. It should be like:

mysql/repos/extra-{i686,x86_64}
mysql-clients -> mysql
mysql-libs -> mysql
 
Allan