On Fri, Nov 12, 2010 at 3:56 PM, csgeek <csgeek@archlinux.us> wrote:

Somewhat related question....

I've seen a few places deploy git based build systems.

funtoo (re-implementation of gentoo's portage but git based) and the arch based that was linked above.

I'm just curious.. at some point wouldn't that just create excessive history that's not needed?  Do you really care about PKGBUILD of something that's 2..3.... 5 years out of date?

what's excessive? your talking about 95%+ highly compressible text files, with highly differentiable change sets and relatively slow updates (vs. an active source code tree); i don't see any problems. it won't accrue much at all imo, even after several years (decades?), but i don't have numbers to back that up. however it is possible to use grafting and other techniques to prune/drop history, and still work with the repo. git and the data model will evolve too; last time i checked there was specifications/steps toward integrating v4/v5 packfile format. git does however, need a good library to bind to; the code right now is... ehm... yeah...

Then again.. if we could use something like that.. to checkout an old version of a package, that would be well worth it.  One thing that was available in gentoo that I miss in arch is the ability to have 2-3 version of a package available.

yeah, you could get any PKGBUILD you needed, with ease, and diff then against anything else... maybe even, dare i say, p2p share the immutable objects around... C Anthony

On Sat, Nov 13, 2010 at 12:35 AM, Ray Rashif <schiv@archlinux.org> wrote:

On 13 November 2010 06:13, C Anthony Risinger <anthony@extof.me> wrote:

...

On Fri, Nov 12, 2010 at 3:56 PM, csgeek <csgeek@archlinux.us> wrote:

...

Somewhat related question....

I've seen a few places deploy git based build systems.

funtoo (re-implementation of gentoo's portage but git based) and the arch based that was linked above.

I'm just curious.. at some point wouldn't that just create excessive history that's not needed?  Do you really care about PKGBUILD of something that's 2..3.... 5 years out of date?

what's excessive?  your talking about 95%+ highly compressible text files, with highly differentiable change sets and relatively slow updates (vs. an active source code tree); i don't see any problems. it won't accrue much at all imo, even after several years (decades?), but i don't have numbers to back that up.

however it is possible to use grafting and other techniques to prune/drop history, and still work with the repo.  git and the data model will evolve too; last time i checked there was specifications/steps toward integrating v4/v5 packfile format.

git does however, need a good library to bind to; the code right now is... ehm... yeah...

...

Then again.. if we could use something like that.. to checkout an old version of a package, that would be well worth it.  One thing that was available in gentoo that I miss in arch is the ability to have 2-3 version of a package available.

yeah, you could get any PKGBUILD you needed, with ease, and diff then against anything else...

maybe even, dare i say, p2p share the immutable objects around...

Binary packages are not version-controlled.

i wasn't implying they were; the question/answers seem to be about PKGBUILDs and related. that said it doesn't mean they couldn't be. we history pruning and various other techniques, such a system could be constructed where users are not required to have every package ever built, only what they wanted, but other peers could maintain the history as far back as they wanted. when all peers drop a point in history, it is lost (but at that point it would seem no one cares) the binary packs themselves could also be unversioned, and only their (very strong) fingerprint maintained in the history.

You can retrieve older versions of PKGBUILDs from our Subversion repository. You can diff between revisions as well. For something so straightforward, GIT is no more helpful.

that was an obvious example. git's data model is very simple and flexible, svn is a convoluted piece of <insert>. with git i can target specific branches for any repository; i can munge history with ease and finesse; i can share object repositories with unrelated projects; i can use the hooking system to create automatic development/staging/production platforms for my team. i could go on for much longer, but this is a well trodden path. git gives you the flexibility to create that which has not yet been discovered.

PKGBUILDs don't get multiple commits by multiple contributors to the extent software projects do, where GIT would make sense.

they will in time my friend, if i get my way :-) this is a goal i am actively pursuing. git makes sense in all capacities; you don't need to be a kernel or display server to realize the extent it can simplify your development life. i use it every day. it rocks. C Anthony

On Sat, Nov 13, 2010 at 6:30 PM, Allan McRae <allan@archlinux.org> wrote:

On 14/11/10 03:23, C Anthony Risinger wrote:

...

On Sat, Nov 13, 2010 at 12:35 AM, Ray Rashif<schiv@archlinux.org>  wrote:

...

PKGBUILDs don't get multiple commits by multiple contributors to the extent software projects do, where GIT would make sense.

they will in time my friend, if i get my way :-) this is a goal i am actively pursuing.  git makes sense in all capacities; you don't need to be a kernel or display server to realize the extent it can simplify your development life.  i use it every day.  it rocks.

We have massive git proponents on the development team and even they admit that SVN fits our needs for package management far better than git.

in what other ways besides the below?

The major issue is checking out an individual folder/package which is very annoying to do in git.  Yes, I know it can be done in some hackish way, but with SVN that works out of the box.  Or we could take the Fedora approach of one repo per package (lets just say no to that...).

hmm, yes, as git ultimately tracks only a single tree object per commit, this object is usually the top level tree for the entire project. however, there are some solutions: ) submodules, the obvious but cumbersome case ) sparse checkout in 1.7.0. there are a couple gotchas IIRC... you still need to pull the whole repo, or a depth of it. additionally, the full path to the directory would be used on checkout (and i dont know if SVN does this, i'm not as fluent there) ) use multiple repos :-), but with a small hook that updates a master repo containing all packages. use a shared object cache ) place all packages into their own branch, with independent history (ie. different origin commits or a common "null" origin). this is my favorite... the latter could be useful, specifically because packages do not cross over with each other (they are all independent "directories", now and forever). ponder this (branch names within a single repo): [README] [master] [core] [extra] [community] [testing] [core/kernel26] [core/xz] ..... ..... [extra/xorg-server] ..... `README` will be the default branch; it will contain instructions on how to work with the repository. anyone cloning the repo would see this first, and only this. `master` is all repositories, all packages. `core`, `extra`, etc. are each their respective repositories, and all respective packages. the remaining are individual packages. now, you could just clone the repo, and get everything (equivalent of a "super" ABS)... or, you could do targeted fetches to get only a single package. you could use the same fetch to pull any particular repo. checking out `master` yields a directory full of repos -- full of packages. checking out a repo yields a a directory full of packages. checking out a package yields only that package, no excess structures or directories. exporting one of the package branches, at any revision, would produce an installable package. all this would need to work is a simple post-receive hook to perform a sync between master, the repo branches, and package branches. this is bi-directional (you could update from any branch, and changes would propagate to the correct places). all that is required in the "up" direction is to read the tree from updated packages, merge the SHA1s into the trees for appropriate repo branches, and last merge all updated trees into the master branch. in the down direction, you would read the top-level tree, pull the SHA1 for changed directories (which correspond to a named branch), and update. git has the tools to do this easily, and could be done using (possibly multiple for efficiency) index files (an inefficient working copy not needed). i actually think it would be a very simple hook; i could probably whip it up soonish if this sounds interesting. the only other thing you might need is a tiny pre-receive hook to drop a lock file, and deny others until the sync is complete (rare), just in case. is this what your describing? i wrote more than i meant to, but meh... imo, none of the options are hackish; i think they just fine, detailing git's flexible nature, with the last being particularly elegant. git gives you the tools to think up your own design + flow, without binding you to any predetermined methods.

Also, we do not need "fancy" stuff like branching and hooks etc.  The current system works very well.  We would need some major motivation to switch and there is nothing that even makes me remotely think a switch is worth it.

i'm sure the current system does work well, and probably will continue to do so for some time. one thing i've noted, is none of the shortcomings with git i have ever encountered (including the above) have anything to do with the data model; they are nothing but limitations in the top level UI, or the protocol. for example, i wish shallow repos were much more functional (sparse checkouts could be truly sparse [like SVN i think]). i also wish i could fetch a SHA1 range from a remote, or a single SHA1 and all deps... anyway, in my particular case (the "if i get my way" comment :-), my package manager/distribution system i like to sink hours into is not even capable of a singular point; it is to expect that everyone can contribute from anywhere, and all peers are equal. _you_ decide which peers are significant to you; in this respect, every user gets to choose their own devs and TU's, and effectively build their own distro. C Anthony