[pacman-dev] The great VCS packages overhaul of 2015

[Matthew Monaco]

On 03/12/2012 09:58 PM, Allan McRae wrote:
> It is well know that the VCS package support in makepkg is subpar...
> The number of bugs about this is into double figures.  There have also
> been some patches submitted attempting to be able to use the source=()
> array in PKGBUILDs to specify a VCS repo and remove a lot of the
> repetitive crap involved for repo checkout/updating in the current
> PKGBUILDs.
> 
> To get the ball rolling again, I think we should pick one VCS system and
> flesh out what we need and what the prototype PKGBUILD would look like.
>  Then we can move on to the other VCS systems and finally implement it.
>  I guess by the time all the bikeshedding is finished, this will all be
> done by 2015 and hence the subject line.  :)
> 
> 
> I am going to start with git.
> 
> Current:
> pkgver = the date the package was built
> _gitroot = the url of the git repo
> _gitname = the name of the directory we check out the repo into...
> 
> According to the man page, _gitname is supposed to be a branch or tag
> but that is all lies.
> 
> 
> What would we like to have for a flexible git package implementation?
>  - url of the repo
>  - be able to specify the branch/tag/commit to use (and appropriate
> combinations)
>  - a decent pkgver
> 
> 
> 1) URL:
> There were previous patches to the mailing list that never really got
> finished, but I think we were fairly happy with this syntax:
> 
> source=(git://projects.archlinux.org/pacman.git)
> source=(git@@http://projects.archlinux.org/git/pacman.git)
> 
> Does it make any sense to allow the "::" syntax here?  i.e.
> source=(git@@dirname::http://projects.archlinux.org/git/pacman.git)
> 
> where dirname is the name of the directory it checked out into?   I am
> thinking we should probably do the checkout into $vcsdir=$startdir/vcs,
> so this would only be needed if
> #1 - we supported multiple VCS checkouts in one PKGBUILD there were two
> that wanted to used the same name...
> #2 - another source file conflicted
> 
> #2 is readily dealt with and I am not sure we should allow #1 (see
> below) so I would vote to skip it until a genuine need is shown.
> 
> 
> 2) Specifying commit to work with:
> I think that this is the difficult bit...   the syntax with the source
> array is already convoluted enough, so I do not think they should be
> added there.  So that suggests we go with assigning them to variables
> like _git_branch, _git_commit, _git_tag... etc.
> 
> But what if we have two git sources? For example, say pacman allowed
> building against an internal copy of libarchive if a folder named
> libarchive was found in its root directory. So:
> 
> source=(git://projects.archlinux.org/pacman.git
>         git://github.com/libarchive/libarchive)
> 

refs=(master
      v3.14.1)
?

Non-vcs sources shouldn't need a placeholder or anything like that.

Of course, instead of sources there could be a repos=() array or something like
that. This would alleviate the syntactic issue of trying to specify a repo,
(optional) dir to clone into, and ref to checkout in a succinct way.

repos=([dir::]url[::ref])

> build()
> {
>   cd $srcdir/pacman
>   ln -s ../libarchive
>   ./autogen.sh
> ...
> 
> It might seem a somewhat convoluted example, but (e.g.) gcc does allow
> in source tree building of many of its dependencies.  The question is
> should we consider this outside the realms of the reasonable and state
> one VCS repo should be one package.   I'd say 99.999% of VCS PKGBUILDs
> (at a lower bound) would never use two VCS sources... and the ones that
> do need this could do manual checking out of the non-main source within
> the PKGBUILD anyway.
> 
> 
> 3) pkgver
> Use output of "git describe" (with a s/-/_/) and fall back to "git
> rev-list HEAD | wc -l" (with a trailing commit id added) if there are no
> tags in the repo.
>

What if there are no tags in the repo, and for whatever reason the PKGBUILD
switches to a different branch? The number of commits just doesn't seem that stable.

Is it at all feasible to just use the build date for package ordering if it's a
vcs package? This way the pkgver can be the ref that was checked out:

  - the short hash
  - the tag
  - the branch (_short hash maybe OR only automatically bump the pkgrel)

> 
> PROTOTYPE:
> 
> pkgname=pacman-git
> pkgver=AUTO
> ...
> source=(git://projects.archlinux.org/users/allan/pacman.git)
> _git_branch="working"
> _git_commit="8c857343"
> 

Are these looking forward to other VCS's though? What scenario is more than one
ref needed? Makepkg would just be doing a checkout right?

> build()
> {
>   cd $srcdir/pacman
>   ./autogen.sh
>   ...
> }
> 
> 
> What makepkg does:
> 1) goes into $vcsdir, checks for the pacman directory
>    - if not present, do the git checkout
>    - if present, enter and do a "git pull" unless --holdver is specified
> 2) enters $srcdir, and does the appropriate clone of the repo in $vscdir
> to be at the required branch/tag/commit
> 3) starts build() etc...
> 

Seems like there's a lot of stuff that can be shared up front, but at some point
there are really two different types of VCS packages: those that should pull
automatically every time the pkg is built (branch), and those that use vcs for
obtaining the source, but are really meant to target a specific version
(tag,commit).

In general, I'll say that I really don't like makepkg manipulating my PKGBUILDs.
So it'd be cool if all of this produced sane package versions in the built
package, but didn't need to touch the PKGBUILD itself.

> 
> 
> Oh wow...  you are still reading this?   Well then, you are now
> qualified to comment on the proposal.  Please stick to just concerns
> with the git usage at the moment, unless you see something monumental
> that we will not be able to support using other VCS systems using this
> approach.
> 
> Allan
>