[aur-dev] Using git as a backend for the AUR

[William Giokas]

All,

So in my spare time I was thinking about the AUR and how it could be
better. Back in January I commented on a bug[1] about integrating the
AUR and git to have a powerful, robust backend for the AUR. I think that
the original idea of creating one massive repository was inherently
flawed for most users, as it requires that the entire repository be
cloned, and that is no trivial task for people with only moderate
internet speeds. Were it similar to the way the official repositories
git setup is, then it would be okay for fetching, but using single
repositories for each project would make access control much less of a
nightmare.

At the moment I have no experience with PHP or messing with the AUR, but
I have been playing around with git and using it to track packages and
the like, as well as some minimal experience with access control and
git.

Currently, this is kind of the roadmap I see in my head:

* Set up access controls for the AUR users to access over ssh using
  keys. While I could see this being somewhat arduous, it shouldn't be
  terribly hard to automate or set up. Something like gitolite or
  whatever should make it simpler.
* Each repository on the server would contain a single package (if
  someone decides to do a split package on the AUR it would contain the
  whole set of packages), allowing for multiple users to have push
  access and maintain the packages.
* The repositories would be limited to 5M, unless given special
  permission (certain kernel packages with lots of massive configuration
  files) which should make enough room for the needed files, while
  helping to enforce the idea of not putting retrievable sources in the
  source tarball. I did some quick math using the number of packages on
  the AUR right now, and if every repo used that 5M limit, it would
  require a bit over 200G. Granted, I doubt that this limit would be
  reached for most packages. 
* With the repository set up, give maintainers a week or so to upload
  history for their packages (in case they already keep their repos in
  this set up), and new packages would get put directly into a new repo.
* Once the week for transition is over, begin moving all of the current
  packages in the AUR to the git format.

This basically concludes setting up the git stuff, which I have done on
a local machine on a very small scale (5 packages) in the past. One
thing I was thinking of was the use of `makepkg -S` tarballs still being
available. cryptocrack came up with a good point, that just simply
copying things over could overwrite the .git directory in the designated
directory. I don't think that would be too hard to get around, but I'm no
expert. I wrote a quick and dirty script that extracts and commits to
the respective repository. Currently, almost no packages use a
.changelog file, but there could be some minimal parsing done of a
packages .changelog file to craft a git commit message from a tarball. 
Obviously there needs to be some kind of security check, but that could be
done by the frontend using the system we have now for source tarballs.

At the moment, I am unsure exactly how the AUR parses the PKGBUILDs it
receives, but I'm sure that this could easily be adapted to the git
system.

One major advantage to having the AUR managed this way is that it would
allow users to check for updates on the AUR without the need for complex
helpers outside of git. Secondly, it would make mirroring the AUR, or
just parts of the AUR extremely easy. In case something happens to the
AUR, people can still get their packages and maintainers can still
update them very easily using git. It would also still allow people to
grab tarballs of the current master branch. This way git is not a
requirement to use the AUR, it just makes things a butt ton easier.
Also, it would allow users to see the entire source of the package, not
just the PKGBUILD, similar to how the official repository works, just
that things would be in distinct git repositories not branches.

I would really like to see this kind of thing come to fruition, and if
anyone would be willing to help, then please say so.


Pardon my scattered thoughts,
-- 
William Giokas | KaiSforza
GnuPG Key: 0x73CD09CF
Fingerprint: F73F 50EF BBE2 9846 8306  E6B8 6902 06D8 73CD 09CF

[1]: https://bugs.archlinux.org/task/23010
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mailman.archlinux.org/pipermail/aur-dev/attachments/20130316/b925c923/attachment-0001.asc>