[aur-dev] Using git as a backend for the AUR
danielwallace at gtmanfred.com
Sun Mar 17 13:14:01 EDT 2013
On Sun, Mar 17, 2013 at 12:43:01PM +0100, Jelle van der Waa wrote:
> On 16/03/13 23:10, William Giokas wrote:
> > All,
> > So in my spare time I was thinking about the AUR and how it could be
> > better. Back in January I commented on a bug about integrating the
> > AUR and git to have a powerful, robust backend for the AUR. I think that
> > the original idea of creating one massive repository was inherently
> > flawed for most users, as it requires that the entire repository be
> > cloned, and that is no trivial task for people with only moderate
> > internet speeds. Were it similar to the way the official repositories
> > git setup is, then it would be okay for fetching, but using single
> > repositories for each project would make access control much less of a
> > nightmare.
> > At the moment I have no experience with PHP or messing with the AUR, but
> > I have been playing around with git and using it to track packages and
> > the like, as well as some minimal experience with access control and
> > git.
> > Currently, this is kind of the roadmap I see in my head:
> > * Set up access controls for the AUR users to access over ssh using
> > keys. While I could see this being somewhat arduous, it shouldn't be
> > terribly hard to automate or set up. Something like gitolite or
> > whatever should make it simpler.
> > * Each repository on the server would contain a single package (if
> > someone decides to do a split package on the AUR it would contain the
> > whole set of packages), allowing for multiple users to have push
> > access and maintain the packages.
> The AUR currently has ~ 41076 packages, which would bring us to an equal
> amount of git repo's. This doesn't seem very efficient to me.
> With svn you can put it all in one repo and checkout a single package
> without checking out the whole repo, which is how
> [core],[extra],[community] work.
> I know Exherbo uses Git for their repo's but they seem to split it up
> into categories .
> Git seems to have sparse checkout for this in 1.7, but I don't know how
> it works 
git clone --single-branch git://projects.archlinux.org/svntogit/packages.git -b packages/linux
... then make one for community.git :)
and inside, you can do stuff like...
git fetch origin packages/bash
git checkout -b packages/bash FETCH_HEAD
and checkout the other branches
> > * The repositories would be limited to 5M, unless given special
> > permission (certain kernel packages with lots of massive configuration
> > files) which should make enough room for the needed files, while
> > helping to enforce the idea of not putting retrievable sources in the
> > source tarball. I did some quick math using the number of packages on
> > the AUR right now, and if every repo used that 5M limit, it would
> > require a bit over 200G. Granted, I doubt that this limit would be
> > reached for most packages.
> > * With the repository set up, give maintainers a week or so to upload
> > history for their packages (in case they already keep their repos in
> > this set up), and new packages would get put directly into a new repo.
> > * Once the week for transition is over, begin moving all of the current
> > packages in the AUR to the git format.
> > This basically concludes setting up the git stuff, which I have done on
> > a local machine on a very small scale (5 packages) in the past. One
> > thing I was thinking of was the use of `makepkg -S` tarballs still being
> > available. cryptocrack came up with a good point, that just simply
> > copying things over could overwrite the .git directory in the designated
> > directory. I don't think that would be too hard to get around, but I'm no
> > expert. I wrote a quick and dirty script that extracts and commits to
> > the respective repository. Currently, almost no packages use a
> > .changelog file, but there could be some minimal parsing done of a
> > packages .changelog file to craft a git commit message from a tarball.
> > Obviously there needs to be some kind of security check, but that could be
> > done by the frontend using the system we have now for source tarballs.
> > At the moment, I am unsure exactly how the AUR parses the PKGBUILDs it
> > receives, but I'm sure that this could easily be adapted to the git
> > system.
> > One major advantage to having the AUR managed this way is that it would
> > allow users to check for updates on the AUR without the need for complex
> > helpers outside of git. Secondly, it would make mirroring the AUR, or
> > just parts of the AUR extremely easy. In case something happens to the
> > AUR, people can still get their packages and maintainers can still
> > update them very easily using git. It would also still allow people to
> > grab tarballs of the current master branch. This way git is not a
> > requirement to use the AUR, it just makes things a butt ton easier.
> > Also, it would allow users to see the entire source of the package, not
> > just the PKGBUILD, similar to how the official repository works, just
> > that things would be in distinct git repositories not branches.
> > I would really like to see this kind of thing come to fruition, and if
> > anyone would be willing to help, then please say so.
> > Pardon my scattered thoughts,
> Although I like the idea, I really think it needs more thoughts ;). So
> thanks for bringing it up.
>  http://git.exherbo.org/summer/
>  http://schacon.github.com/git/RelNotes/1.7.0.txt
Archlinux Trusted User (gtmanfred)
Georgia Institute of Technology
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 490 bytes
Desc: not available
More information about the aur-dev