On Mon, Jun 02, 2014 at 07:08:43PM +0200, Lukas Fleischer wrote:
Hi list,
The next major AUR release will come with Git integration. In this email, I am going to summarize previous ideas and make things a bit more concrete (sketch implementation details etc.)
If you are new to this topic, please read [1] and [2] first.
Our plan is to create one Git repository per package base. Each of the Git repositories contains a PKGBUILD and zero or more additionally required files, so you will no longer need to build a source tarball. There will be a pkgbuild-introspection utility to generate and write .AURINFO metadata without automatically putting that file inside the tarball. Florian had the idea to use a shared Git object database for space efficiency.
When creating a new account, users will be able to upload an SSH public key. This key will be stored in the AUR database and can also be changed using the profile edit page. Using this public key, users will be able to push to repositories they maintain using Git over SSH. For authentication, a custom script doing a lookup in the AUR user database will be used as AuthorizedKeysCommand. If a matching key is found, that script prints the corresponding authorized_keys line with a forced command that invokes a wrapper around the Git shell. That wrapper in turn does authorization by checking maintainership and then calls git-shell(1).
I am very much for this (and [1] is one of my mails from more than a year ago). I would recommend working with gitolite so that we can keep as much of the code we need maintained upstream, as gitolite has already proven its security and its efficiency. Also, it may allow other users to use gitolite with database backends like the AUR without using the AUR itself. I actually think that the space estimates that I had before (~5M per repo) would be way high with a shared object cache, especially because that could be packed very efficiently. I would recommend having a hard limit on the size of these repos, however, and something around 5M should be good to help prevent people from uploading binary bits, as those would probably explode repos and things. From what I have seen, most of the heavy lifting that we would need to do has already been done by things like cgit and gitolite, so really it would only be a matter of getting the authentication in place for the mass of users the AUR has.
Using Git hooks, the .AURINFO metadata of a package is parsed when pushing and the AUR package database is updated accordingly. Hooks will also take care of checking the tree objects for huge files etc. The receive.denyNonFastForwards configuration option will be enabled to prevent users from rewriting the history.
This was the main concern I had when writing [1], as we wold have to parse pkgbuilds in some fashion. With the .AURINFO files now in use, this is much less of a concern.
In order to submit new packages, you will be able to generate empty Git repositories via the AUR web interface. During the transition period, all existing source tarballs will be converted to bare repositories with one initial commit whose tree equals the tarball contents.
Instead of the source tarball download link and the PKGBUILD preview, there will be a public clone URL on the package details page. In a second step, cgit will be configured to provide a web-based interface to all the repositories. Then, links to tarball snapshots and to the PKGBUILD preview will be added to the details page as a replacement for the "Download tarball" and "View PKGBUILD" links (all this is supposed to happen before the first release).
As soon as all this is set up, we can add support for multiple maintainers and related features.
This should be able to be done very similarly to how the official packages are set up, however instead of branches there would be completely separate repositories. One thing that I think should not be done at all is having a scheme like git://.../username/package This would force users to change their upstream URLs based on maintainership, which changes quite a lot on the AUR, but this is something that was probably already thought of. I'd suspect that we would still have to do something like git://.../pa/package but there may be a way around that, even.
If there are any questions or suggestions regarding this setup, please feel free to ask/reply.
I think this is an amazing idea, and I have some good plans for things like pre-commit hooks for local repos that will auto-generate the .AURINFO file for users without them even having to worry about that. If there's any way I can help, I'd be more than willing! [1] https://mailman.archlinux.org/pipermail/aur-dev/2013-March/002411.html [2] https://mailman.archlinux.org/pipermail/aur-dev/2014-January/002592.html Thanks, -- William Giokas | KaiSforza | http://kaictl.net/ GnuPG Key: 0x73CD09CF Fingerprint: F73F 50EF BBE2 9846 8306 E6B8 6902 06D8 73CD 09CF