[aur-dev] Using git as a backend for the AUR

[William Giokas]

On Sun, Mar 17, 2013 at 01:44:12PM +0100, Kwpolska wrote:
> On 16/03/13 23:10, William Giokas wrote:
> >  The repositories would be limited to 5M
> 
> Measuring by current size of contents or total size?  The latter would
> be very unpleasant for packages modified often or packages with binary
> files (eg. dropbox/dropbox.png; binary files in git aren’t using
> deltas AFAIK).

Binaries (even images) should not be hosted in the AUR. This is simply a
waste of space with the plethora of hosting services around. Someone on
aur-general mentioned using a 1-2M limit, which actually seems more
sane.

> On Sun, Mar 17, 2013 at 12:43 PM, Jelle van der Waa <jelle at vdwaa.nl> wrote:
> > The AUR currently has ~ 41076 packages, which would bring us to an equal
> > amount of git repo's. This doesn't seem very efficient to me.
> 
> Well, why do you think it is inefficient?  Maybe because of the amount
> of repos?  GitHub hosts (as of now) 5,700,361 repos[0], big and small.
>  That number doesn’t include gists, and there are a lot of them.  So,
> we would be around six million, at the very least.

While I agree with you, even having 41K git repos would take some more
thought. 

> Because of the repo size?  Let me disprove you.
> 
> Currently, the AUR stores stuff in 41078 tarballs (tar+gz), plus 41078
> PKGBUILD files stored without any compression on the server (eg.
> https://aur.archlinux.org/packages/pk/pkgbuilder/PKGBUILD ).  The
> PKGBUILD and the tarball weigh a total of 1241 bytes[1].
> 
> Now, we make a local git repo (git init --bare), and get the
> aforementioned PKGBUILD into it from a downstream clone (git
> {add,commit,push}).  After removing the sample hooks (unnecessary,
> although the AUR may make good use of such hooks[2]), we end up with
> 843 bytes[2].  After an update (pkgver+md5sums) we get 1518 bytes[3].
> This is a small price for a complete PKGBUILD modification history.
> 
> If somebody wants numbers for Mercurial, it gets 1262 and 1725
> bytes[4], respectively.  I couldn’t set SVN up easily, so I don’t have
> any numbers.
> 
> [0] https://github.com/search; displayed if you are lucky enough
> [1] makepkg --source; wc -c PKGBUILD pkgbuilder*.src.tar.gz
> [2] For example, Heroku uses post-update hooks to trigger a build of
> the website.  GitHub has an awesome Service Hook system, where you can
> have a bazillion services do stuff after you push (eg. unit tests via
> Travis CI)
> [3] wc -c config description HEAD objects/*/* refs/heads/master
> [4] wc -c 00changelog.i branch dirstate hgrc last-message.txt requires
> undo.bookmarks undo.branch undo.desc undo.dirstate
> store/00changelog.i store/00manifest.i store/fncache store/phaseroots
> store/undo store/undo.phaseroots store/data/* cache/branchheads-served
> 
> > With svn you can put it all in one repo and checkout a single package
> > without checking out the whole repo, which is how
> > [core],[extra],[community] work.
> 
> svn is ugly and human-unfriendly.  Nobody likes it.  And I couldn’t
> get it to work properly, as mentioned above.
> 
> > Git seems to have sparse checkout for this in 1.7, but I don't know how it works.
> 
> http://jasonkarns.com/blog/subdirectory-checkouts-with-git-sparse-checkout/

Thank you,
-- 
William Giokas | KaiSforza
GnuPG Key: 0x73CD09CF
Fingerprint: F73F 50EF BBE2 9846 8306  E6B8 6902 06D8 73CD 09CF
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mailman.archlinux.org/pipermail/aur-dev/attachments/20130317/2fc12871/attachment-0001.asc>