[pacman-dev] The great VCS packages overhaul of 2015

[Allan McRae]

On 13/03/12 15:36, Matthew Monaco wrote:
> On 03/12/2012 09:58 PM, Allan McRae wrote:

>> 2) Specifying commit to work with:
>> I think that this is the difficult bit...   the syntax with the source
>> array is already convoluted enough, so I do not think they should be
>> added there.  So that suggests we go with assigning them to variables
>> like _git_branch, _git_commit, _git_tag... etc.
>>
>> But what if we have two git sources? For example, say pacman allowed
>> building against an internal copy of libarchive if a folder named
>> libarchive was found in its root directory. So:
>>
>> source=(git://projects.archlinux.org/pacman.git
>>         git://github.com/libarchive/libarchive)
>>
> 
> refs=(master
>       v3.14.1)
> ?
> 
> Non-vcs sources shouldn't need a placeholder or anything like that.
> 
> Of course, instead of sources there could be a repos=() array or something like
> that. This would alleviate the syntactic issue of trying to specify a repo,
> (optional) dir to clone into, and ref to checkout in a succinct way.
> 
> repos=([dir::]url[::ref])

I would really prefer not to have a new array for the repo source.  Part
of the attraction of getting all this into the source array is that we
only have to look in one place to see where the source came from.

>> build()
>> {
>>   cd $srcdir/pacman
>>   ln -s ../libarchive
>>   ./autogen.sh
>> ...
>>
>> It might seem a somewhat convoluted example, but (e.g.) gcc does allow
>> in source tree building of many of its dependencies.  The question is
>> should we consider this outside the realms of the reasonable and state
>> one VCS repo should be one package.   I'd say 99.999% of VCS PKGBUILDs
>> (at a lower bound) would never use two VCS sources... and the ones that
>> do need this could do manual checking out of the non-main source within
>> the PKGBUILD anyway.
>>
>>
>> 3) pkgver
>> Use output of "git describe" (with a s/-/_/) and fall back to "git
>> rev-list HEAD | wc -l" (with a trailing commit id added) if there are no
>> tags in the repo.
>>
> 
> What if there are no tags in the repo, and for whatever reason the PKGBUILD
> switches to a different branch? The number of commits just doesn't seem that stable.

No tags in the repo is covered by the rev-list option.  I agree that
these numbers are not particularly stable (with switching branches etc),
but I would assume that someone providing a repo with a VCS package will
not be branch hopping too often and for local packaging it really does
not matter.

> Is it at all feasible to just use the build date for package ordering if it's a
> vcs package? This way the pkgver can be the ref that was checked out:
> 
>   - the short hash
>   - the tag
>   - the branch (_short hash maybe OR only automatically bump the pkgrel)

I would really like to move away from the build date having anything to
do with package ordering as it is entirely non-deterministic about what
is in a package.  Also, that would require pacman to know which packages
were built from a VCS style PKGBUILD and all the extra tracking to
implement that...

>>
>> PROTOTYPE:
>>
>> pkgname=pacman-git
>> pkgver=AUTO
>> ...
>> source=(git://projects.archlinux.org/users/allan/pacman.git)
>> _git_branch="working"
>> _git_commit="8c857343"
>>
> 
> Are these looking forward to other VCS's though? What scenario is more than one
> ref needed? Makepkg would just be doing a checkout right?

This might be me not understanding some of the subtleties of git, but
this is what I see as options for building from a git repo...

1) build from master HEAD
2) build from a branch HEAD
3) build from a given commit/tag

So, assuming that the git repo is checked out from upstream in $vcsdir,
the command needed for each of these is:

1) git clone --branch master $vcsdir     [*]
2) git clone --branch <branch> $vcsdir
3) git clone $vcsdir; git checkout <ref>

[*] this fails if the default branch upstream is not master, but does
allow you to either do some work in the copy in $vcsdir or have your
PKGBUILD git source point to a local copy of a git repo that you
actually do work in...


Anyway, from what I can see from those three use cases is that to be
flexible here, we need to be able to specify a branch or a ref
(commit/tag).  I'm not sure how to differentiate the two without
assigning them to different variables.

And while it is not necessary, I would find it useful to have the branch
specified even when the commit/tag is used more for documentation than
anything else.


>> build()
>> {
>>   cd $srcdir/pacman
>>   ./autogen.sh
>>   ...
>> }
>>
>>
>> What makepkg does:
>> 1) goes into $vcsdir, checks for the pacman directory
>>    - if not present, do the git checkout
>>    - if present, enter and do a "git pull" unless --holdver is specified
>> 2) enters $srcdir, and does the appropriate clone of the repo in $vscdir
>> to be at the required branch/tag/commit
>> 3) starts build() etc...
>>
> 
> Seems like there's a lot of stuff that can be shared up front, but at some point
> there are really two different types of VCS packages: those that should pull
> automatically every time the pkg is built (branch), and those that use vcs for
> obtaining the source, but are really meant to target a specific version
> (tag,commit).

There is quite some overlap there...   I quite often have commits on my
local pacman working branch that I do not want running on my live system
just yet.  So my pacman-git package gets built from my working branch
with the last commit I am happy with optionally specified.

> In general, I'll say that I really don't like makepkg manipulating my PKGBUILDs.
> So it'd be cool if all of this produced sane package versions in the built
> package, but didn't need to touch the PKGBUILD itself.

I tend to agree about makepkg not adjusting PKGBUILDs.  Although, it
does have the advantage that after building a package, you can "makepkg
--source" and send that archive to someone and they will be able to
build the exact same package as you using the --holdver flag (or at
least will be able to when --holdver works...).


Allan