[pacman-dev] About makepkg dlagents

[Ciprian Dorin, Craciun]

On Mon, Oct 26, 2009 at 9:16 AM, Allan McRae <allan at archlinux.org> wrote:
> Ciprian Dorin, Craciun wrote:
>>
>>    About download-agents for makepkg. (This is why I've bothered the
>> developers of pacman this morning with compilation issues... :))
>>
>>    So I wanted to be able to make PKGBUILD and makepkg extract
>> sources for Git repositories. And after asking on the arch-general, I
>> was pointed to the following bug:
>>        http://bugs.archlinux.org/task/7816
>>
>>    As a consequence I've started hacking and commed up with the
>> following patches (put at the end). (I've ended up with four pathes
>> from git format-patch... Should I put only the diff between maint
>> branch and my branch?)
>>
>>    Also below is the description on the bug-tracker:
>>
>> ~~~~
>> I've made some hacking staring from pacman Git repository. My work is
>> available at (branch patches/git-dlagent):
>> http://gitorious.org/~ciprian.craciun/pacman/ciprian-craciun-patches
>>
>> In summary:
>> * added a script: /usr/bin/git-dlagent that takes exactly two
>> arguments: URL and output.
>> * updated /etc/makepkg.conf to include the following URL's: git://,
>> git+file://, git+http://, git+https://, git+ssh://, git+rsync;
>>
>
> <snip>
>
> Hi,
>
> Thanks for tackling this.  There are some interesting ideas in your patch.

    I'm glad you've loked over my proposal. (I was woried for a while. :) )


> From my point of view, this is far too complex for inclusion into makepkg.
>  The addition of six new download URLs to makepkg (and that is just for git
> and not cvs,svn,hg,...) and requiring a download script just seems
> overboard.

    Its unfortunately that my proposal is not accepted in the current
form, but I can't (completley) agree with the reasons:

    * "the addition of six new download URLs to makepkg config": I see
why this is a problem -- there would be a lot of boiler-plate
configuration code in there -- but this is not a problem of the
extensions, but of the architecture of the configuration file; why
couldn't we just source a config file for each download agent? or
perhaps split the config file in multiple config files: this has other
benefits also, by focusing on concerns: dlagent parameters, compile
parameters, etc.

    * "requiring a download script": today makepkg already depends on
curl / wget, rsync, or scp; why would another executable (or script in
this case) be a problem?


> Taking a look at git PKGBUILDs on the AUR and the git prototype distributed
> with the Arch from the ABS package, what is really wanted is to reduce this:
>
> --start--
> _gitroot="GITURL"
> _gitname="MODENAME"
>
> build() {
>  cd "$srcdir"
>  msg "Connecting to GIT server...."
>
>  if [ -d $_gitname ] ; then
>   cd $_gitname && git pull origin
>   msg "The local files are updated."
>  else
>   git clone $_gitroot
>  fi
>
>  msg "GIT checkout done or server timeout"
>  msg "Starting make..."
>
>  rm -rf "$srcdir/$_gitname-build"
>  git clone "$srcdir/$_gitname" "$srcdir/$_gitname-build"
>  cd "$srcdir/$_gitname-build"
> --end--
>
> with a "source=(git...)" line.  I'm sure that you have covered some edge
> cases that can not be handled very easily in this fashion and I see some
> added functionality in your changes.  However, I am not sure we need that
> complexity in 99% of use cases.

    Before answering, please allow me a to present my context.
Although I'm not an user of either ArchLinux or Pacman for too long
(only about a month), I've been srugling with build systems on other
distributions. At one moment in time I've even created my own build
system in PLT Scheme (now I'm trying to port the ideas over Python).

    So based on what I've seen a build system has the following
responsabilities:
    * compile-time dependency checking; (like ./configure does, but
without the actual configuration;)
    * build environment preparation; (this includes downloading needed
source code, checking it, patching, etc.;)
    * configuring the build according to the package specification;
(invoking ./configure;)
    * building the package; (make;)
    * creating a deployable package;

    Now getting back with the problem where we want to reduce the
snipet of code you've highlighted. We can see that the purpose of that
code is to "prepare the build environment", more exactly fetching the
source code, but we are forced to showell it inside the "build stage"
function.

    So the reason why I've created the patch is because I want to
tread source code comming from Git just like any other source-code
bundle, without beeing forced to put it inside the build function.


> Your patch has pointed out some interesting issues.  Especially the issue of
> checksums and how they are handled when the source is not a real source
> file.
>
> Allan

    So once again thank you for your feedback.

    Ciprian.