[arch-dev-public] Git
Hi, As I gather, we all like git better than svn, for a long list of reasons. Are there any objections to switching over from svn to git for repositories for the official packages? Yes, this can not be done in a heartbeat. The tools and documentation needs to be updated and the workflow needs to be tested, but are there objections to starting the transition process? -- Best regards, Alexander Rødseth xyproto / TU
On Sun, Sep 29, 2013 at 9:00 PM, Alexander Rødseth <rodseth@gmail.com> wrote:
As I gather, we all like git better than svn, for a long list of reasons. Are there any objections to switching over from svn to git for repositories for the official packages?
I'm strongly in favor of git, and would be happy to work on the transition if we decide to make the switch.
Yes, this can not be done in a heartbeat. The tools and documentation needs to be updated and the workflow needs to be tested, but are there objections to starting the transition process?
If we are going to do this change, let's spend some time on discussing how we want the new layout to be like. I think we can simplify stuff a lot compared to current svn. -t
Am 29.09.2013 21:07, schrieb Tom Gundersen:
On Sun, Sep 29, 2013 at 9:00 PM, Alexander Rødseth <rodseth@gmail.com> wrote:
As I gather, we all like git better than svn, for a long list of reasons. Are there any objections to switching over from svn to git for repositories for the official packages?
I'm strongly in favor of git, and would be happy to work on the transition if we decide to make the switch.
Yes, this can not be done in a heartbeat. The tools and documentation needs to be updated and the workflow needs to be tested, but are there objections to starting the transition process?
If we are going to do this change, let's spend some time on discussing how we want the new layout to be like. I think we can simplify stuff a lot compared to current svn.
-t
We already did, but no time to implement yet: https://wiki.archlinux.org/index.php/User:Pierre/pkgdbgit -- Pierre Schmitz, https://pierre-schmitz.com
Hi, Thanks for the feedback! As a summary: * The reception of the idea of introducing git varies from strongly in favor to no objection (or implicitly in favor). There is only one suggestion that svn might be easier for our use. * At least one person is happy to work on the transition (Tom Gundersen). * I see that Florian Pritz has made great strides with regards to the tooling, ref. the thread with the subject "Rewriting dbscripts (packagers should read this!)" * The ideas here about using one git repo per package (as suggested by Pierre and Tom) is exactly what has been used in the other thread. * The ideas about using one branch per package or packages as submodules seems to have been disregarded. * The ideas about supporting on a host like github or gitorious have only had a positive reception so far, but someone needs to write a few words about specifically how a service like github or gitorious could be used together with the new tooling (possibly giving references to how Arch Linux ARM are running things). Ideas for "next action" once the tooling is in place are also needed. I imagine that it mainly is a question of someone with access to the servers flipping the right switches. -- Best regards, Alexander Rødseth xyproto / TU
The following text was sent to me by Kevin Mihelich @ Arch Linux ARM, with permission to post it here as well: We poll the svntogit repo regularly to bring up updates to packages that are made by Arch maintainers, and build everything as-is. For packages that require modifications (configure settings, patches, etc.) to build on ARM systems, we take the package and put it into our github repo in the same spot under core/extra/community with the needed changes. Our build system software then recognizes that the same package is in github and svntogit under the same repo, and uses the github version over svntogit. We also maintain two separate repos out of our github, one called 'alarm' for packages we create, and an 'aur' repo for some of the highly requested AUR packages (since many ARM systems lack the ability to build packages in timely manner, or at all). Like has been mentioned, github's ability to clone and submit pull requests has been a huge feature for us, as it allows contributions and fixes to packages from people that don't want to be a maintainer. It allows us to quickly review the changes, and if everything is fine it get merged and built automatically by the build system -- a huge time saver for us. Additionally, you can dig into commits or existing files and comment on particular lines, creating separate conversation threads on points of issue that need to be resolved. Now, if I may, here's how your decisions on how you set up a git repo in place of svn affect us. Right now, with the svntogit providing a merged repo of all the packages in two places (core/extra in one, community in another) allows me to simply poll the two git repos for changes and operate on that changeset. This means that when packages get added, removed, or modified, I'm able to see everything that takes place. If each package was to be split into its own git repo, I would need to devise some (potentially ugly) solution to scrape all of the current repos from somewhere, match them against what I know to exist, add packages (repos) that just got created and delete packages (repos) that just got removed, then further correlate which pacman repo each belongs to, if repos have changed, etc. It would also require that I either find some way to determine which package git repos changed and only poll them, or potentially have to poll the ~4500 git repos that would exist individually. In short, I don't know how I would accomplish this off the top of my head, and it could be a massive pain to implement. My two cents toward the repo structuring would be to follow the same structure that is in place now with svntogit: have a packages.git that contains core and extra, which Arch developers have access to change, and a community.git that contains community, which devs and TUs have access to change. Unless there is some overbearing need to individually delegate write access per package, individual git repos per package seems like it would be more of an administrative headache than it is worth (aside from destroying my current development model for Arch Linux ARM). Should you decide to use a service like github, a combined package repo also means that community-generated pull requests are also combined, allowing more people to be notified of changes, submit their input, or merge changes when someone is on vacation. A combined repo also means that you can delegate a 'testing' branch to contain all the testing versions of all the packages, allowing one to push/pull to all of the testing packages in one move, or merge a large set of changes back to stable, without having to go through and do the same operation on multiple git repos (I'm specifically thinking of things like the Gnome and KDE package sets). One could potentially create a 'gnome-testing' branch, do all the changes and builds based on that branch, then when it's ready merge that branch back into stable and you're done.
On Fri, Oct 4, 2013 at 6:15 PM, Alexander Rødseth <rodseth@gmail.com> wrote:
Now, if I may, here's how your decisions on how you set up a git repo in place of svn affect us. Right now, with the svntogit providing a merged repo of all the packages in two places (core/extra in one, community in another) allows me to simply poll the two git repos for changes and operate on that changeset. This means that when packages get added, removed, or modified, I'm able to see everything that takes place. If each package was to be split into its own git repo, I would need to devise some (potentially ugly) solution to scrape all of the current repos from somewhere, match them against what I know to exist, add packages (repos) that just got created and delete packages (repos) that just got removed, then further correlate which pacman repo each belongs to, if repos have changed, etc. It would also require that I either find some way to determine which package git repos changed and only poll them, or potentially have to poll the ~4500 git repos that would exist individually. In short, I don't know how I would accomplish this off the top of my head, and it could be a massive pain to implement.
To clarify, the individual git repos are just for tracking individual package development. There still would be a central git repo maintained by our dbscripts which reflects the status of all our pacman repos. Either by using submodules pointing at tagged commits in the individual git repos, or, if submodules don't work out well, by copying the state of those tags, maybe using git-subtree.
My two cents toward the repo structuring would be to follow the same structure that is in place now with svntogit: have a packages.git that contains core and extra, which Arch developers have access to change, and a community.git that contains community, which devs and TUs have access to change. Unless there is some overbearing need to individually delegate write access per package, individual git repos per package seems like it would be more of an administrative headache than it is worth (aside from destroying my current development model for Arch Linux ARM). Should you decide to use a service like github, a combined package repo also means that community-generated pull requests are also combined, allowing more people to be notified of changes, submit their input, or merge changes when someone is on vacation. A combined repo also means that you can delegate a 'testing' branch to contain all the testing versions of all the packages, allowing one to push/pull to all of the testing packages in one move, or merge a large set of changes back to stable, without having to go through and do the same operation on multiple git repos (I'm specifically thinking of things like the Gnome and KDE package sets). One could potentially create a 'gnome-testing' branch, do all the changes and builds based on that branch, then when it's ready merge that branch back into stable and you're done.
When moving changes from testing to stable, the individual git repos are not touched at all. They only track package versions, not which pacman repos they are in (if any). Moving happens inside dbscripts, which only needs to update the pacman repos and the central git repo mentioned above.
What about handling mass-edits? Right now I feel the approach of having one repo for core/extra and one for community with branches for testing would be better, my only issue with this being that I wouldn't tag each package release in it and I don't really see the need for it. I am unconvinced on the advantages of having separate repos for each package.
On Mon, Oct 07, 2013 at 09:47:18AM +0200, Massimiliano Torromeo wrote:
What about handling mass-edits?
Right now I feel the approach of having one repo for core/extra and one for community with branches for testing would be better, my only issue with this being that I wouldn't tag each package release in it and I don't really see the need for it.
I am unconvinced on the advantages of having separate repos for each package.
I think the following approach would be a very flexible one: main repo's: * core with branches: * staging * testing * master (stable) * extra with branches: * staging * testing * master (stable) * community with branches: * staging * testing * master (stable) All packages would be submodules of the specific repo's, this means if you want to do mass editing you can do it from within the main repo and push all your submodules from there. With some scripting this can be done in an easy way. The tagging of every package would be pointless in this scenario because you can just indicate it in the commit and people wanting to see the whole history can check the tags and commits in the package specific repo's. Depending on the repo and branch we could detect how the package must be build. Which could make it as simple as just having a build-i686 and build-x86_64, those can detect repo/branch and determine what pacman configuration must be used, with *-testing, with *-staging. The biggest problem with git is the removal of submodules, but soon [1] it will be improved. [1] https://github.com/git/git/commit/95c1641 -- Ike
On Mon, Oct 07, 2013 at 09:47:18AM +0200, Massimiliano Torromeo wrote:
What about handling mass-edits?
Really, how often do you do this? Why would your workflow be so different? Instead of 'svn up', you'd 'git clone'. Instead of 'svn commit', you'd write a for loop with 'git commit'. The actual editing remains exactly the same.
Right now I feel the approach of having one repo for core/extra and one for community with branches for testing would be better, my only issue with this being that I wouldn't tag each package release in it and I don't really see the need for it.
The fact is, git does not perform well with wide trees. In your world, each branch would be fully divergent from the root. This leads to bloat and poor performance. Switching branches would, over time, become increasingly slow as you'd need to rebuild the entire tree. In essence, you'd be poorly mimicking subtress, submodules, or repo-for-each, but with a restricted feature set. Some git operations cease to make sense in this sort of world -- tagging is less obvious, and branching would be weird since you're in a restricted "namespace". d
On 30 September 2013 03:00, Alexander Rødseth <rodseth@gmail.com> wrote:
Hi,
As I gather, we all like git better than svn, for a long list of reasons. Are there any objections to switching over from svn to git for repositories for the official packages?
Yes, this can not be done in a heartbeat. The tools and documentation needs to be updated and the workflow needs to be tested, but are there objections to starting the transition process?
I don't think there are serious objections. Just that it needs a ton of work. Anyone motivated enough should pick up from where e.g. Pierre left off, and finalize a complete proposal that those not bothered can look at and just ack. -- GPG/PGP ID: C0711BF1
participants (8)
-
Alexander Rødseth
-
Dave Reisner
-
Ike Devolder
-
Jan Alexander Steffens
-
Massimiliano Torromeo
-
Pierre Schmitz
-
Rashif Ray Rahman
-
Tom Gundersen