[aur-dev] Git repositories for AUR packages

William Giokas 1007380 at gmail.com
Wed Jun 4 16:16:40 EDT 2014


On Mon, Jun 02, 2014 at 07:08:43PM +0200, Lukas Fleischer wrote:
> Hi list,
> 
> The next major AUR release will come with Git integration. In this
> email, I am going to summarize previous ideas and make things a bit more
> concrete (sketch implementation details etc.)
> 
> If you are new to this topic, please read [1] and [2] first.
> 
> Our plan is to create one Git repository per package base. Each of the
> Git repositories contains a PKGBUILD and zero or more additionally
> required files, so you will no longer need to build a source tarball.
> There will be a pkgbuild-introspection utility to generate and write
> .AURINFO metadata without automatically putting that file inside the
> tarball. Florian had the idea to use a shared Git object database for
> space efficiency. 
> 
> When creating a new account, users will be able to upload an SSH public
> key. This key will be stored in the AUR database and can also be changed
> using the profile edit page. Using this public key, users will be able
> to push to repositories they maintain using Git over SSH. For
> authentication, a custom script doing a lookup in the AUR user database
> will be used as AuthorizedKeysCommand. If a matching key is found, that
> script prints the corresponding authorized_keys line with a forced
> command that invokes a wrapper around the Git shell. That wrapper in
> turn does authorization by checking maintainership and then calls
> git-shell(1).

I am very much for this (and [1] is one of my mails from more than a year
ago). I would recommend working with gitolite so that we can keep as
much of the code we need maintained upstream, as gitolite has already
proven its security and its efficiency. Also, it may allow other users
to use gitolite with database backends like the AUR without using the
AUR itself.

I actually think that the space estimates that I had before (~5M per
repo) would be way high with a shared object cache, especially because
that could be packed very efficiently. I would recommend having a hard
limit on the size of these repos, however, and something around 5M
should be good to help prevent people from uploading binary bits, as
those would probably explode repos and things.

From what I have seen, most of the heavy lifting that we would need to
do has already been done by things like cgit and gitolite, so really it
would only be a matter of getting the authentication in place for the
mass of users the AUR has.

> Using Git hooks, the .AURINFO metadata of a package is parsed when
> pushing and the AUR package database is updated accordingly. Hooks will
> also take care of checking the tree objects for huge files etc. The
> receive.denyNonFastForwards configuration option will be enabled to
> prevent users from rewriting the history.

This was the main concern I had when writing [1], as we wold have to
parse pkgbuilds in some fashion. With the .AURINFO files now in use,
this is much less of a concern.

> In order to submit new packages, you will be able to generate empty Git
> repositories via the AUR web interface. During the transition period,
> all existing source tarballs will be converted to bare repositories with
> one initial commit whose tree equals the tarball contents.
> 
> Instead of the source tarball download link and the PKGBUILD preview,
> there will be a public clone URL on the package details page. In a
> second step, cgit will be configured to provide a web-based interface to
> all the repositories. Then, links to tarball snapshots and to the
> PKGBUILD preview will be added to the details page as a replacement for
> the "Download tarball" and "View PKGBUILD" links (all this is supposed
> to happen before the first release).
> 
> As soon as all this is set up, we can add support for multiple
> maintainers and related features.

This should be able to be done very similarly to how the official
packages are set up, however instead of branches there would be
completely separate repositories. One thing that I think should not be
done at all is having a scheme like

  git://.../username/package

This would force users to change their upstream URLs based on
maintainership, which changes quite a lot on the AUR, but this is
something that was probably already thought of. I'd suspect that we
would still have to do something like

  git://.../pa/package

but there may be a way around that, even.

> If there are any questions or suggestions regarding this setup, please
> feel free to ask/reply.

I think this is an amazing idea, and I have some good plans for things
like pre-commit hooks for local repos that will auto-generate the
.AURINFO file for users without them even having to worry about that.

If there's any way I can help, I'd be more than willing!

[1] https://mailman.archlinux.org/pipermail/aur-dev/2013-March/002411.html
[2] https://mailman.archlinux.org/pipermail/aur-dev/2014-January/002592.html

Thanks,

-- 
William Giokas | KaiSforza | http://kaictl.net/
GnuPG Key: 0x73CD09CF
Fingerprint: F73F 50EF BBE2 9846 8306  E6B8 6902 06D8 73CD 09CF
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mailman.archlinux.org/pipermail/aur-dev/attachments/20140604/e011ad34/attachment.asc>


More information about the aur-dev mailing list