[arch-dev-public] Getting to New Arch SCM Layout and Repo Management
All, Arch needs to move forward. The question: how? Moving forward at high speed is desirable, but we need first make sure we're not pointed towards a cliff and that our safety cables are attached and in good working order. I am setting my sights on exactly one aspect of this problem: SCM and repo management and getting us from where we are into the new system we all want to have. BACKGROUND For the purists, there is a sentiment I would like to acknowledge, because it's one that's been bugging me. I believe that the most often overlooked and underrated "Arch Way" design choice was to support i686 only. A single common denominator makes everything a lot simpler-- we just solve one problem and we solve it very well. Development resources are not spread thin and are highly focused on the same end. I am not the purist I once was. There's a vital and important segment of the community that needs 64-bit support, we can't just drop i686 support, and anyone has to acknowledge that i686 won't be around forever. To survive in the long haul, Arch will need to support more than just the i686 architecture. With some weight then, knowing we're setting out *against* the Arch Way, but along a trail that *must* be blazed, we consider the landscape. GOAL Using an open design and implementation process, we will implement a new system for managing Arch repositories and PKGBUILD materials. I will provide a box with Internet access for this project we can use as our home. It will be completely separate from the main Arch servers. If our system proves itself, it will be presented to the main Arch development team as the new standard for repo management. Main design choices I am making (to be amended as we go along): 1) The system will be able to manage an arbitrary number of repositories for multiple architectures efficiently. 2) Use of the system will not require a shell account on the server. 3) Multiple instances of the system will be easily deployable on a single server. 4) It will be trivially easy to create and destroy repositories, to copy or move packages between repositories, or to place a package in more than one repository. 5) It will be easy for developers with SCM access to efficiently incorporate patches and input from community members. 6) The system will allow anyone with SCM access to register as a maintainer for a package. 7) All functions of the system will be primarily accessible from a simple commandline interface. 8) The system will be itself deployed as an Arch package and be easy to set up. 9) The system will be sufficiently documented in a functional, if not beautiful, fashion. SCHEDULE By May 15: Anyone with another SCM system or layout to suggest should do so in the Arch-repos group. Your goal is to persuade Jason your system is the best one. Discussion of all those systems will then take place over the following two weeks. By May 7: I will bring 7 key people on board to work on this project. Drop me an email if you're interested in joining the team! Initially, I know we will need: SYSADMIN - An existing Arch developer who will manage the Linux box I provide and the accounts for the SCM system. This person should have at least a small amount of time each day to handle management tasks related to the project. PUBLICIST - I am committed to organizing this project, and will continue to post status to the arch-dev-public list for all to see. However, not everyone looks here. We need someone who will deliver the news about this project and the status as it progresses to the forums, the IRC channel, and the other mailing lists. This won't be a time-consuming job, but will require someone who has knowledge and facility with all the communications channels used in the Arch community and who can focus on getting the word out. DEVELOPERS - There will be some straight-up development work needed to get us from the SCM system and layout to an actual management system for repos. We may benefit from some work already done by others (pacman3, AUR, main site repo management tools) but we will need to do some original coding here. I will be somewhat involved in the design and coding, but will choose three (3) people to help with development. ANYONE is a potential candidate.. send me and email and tell me why you'd make a good candidate. Focus on good ideas you have for the system, languages you're comfortable in, how much of Getting Things Done (GTD) you've implemented for yourself, and amount of time you'll have to give to the project in June and July. COMMUNITY MEMBERS - I will choose two (2) members of the community who specifically are NOT developers or TUs (yet!) but who are interested in helping maintain the packages in our new system. These people will NOT be given SCM access, so as to help us evaluate how easy it is for community members to contribute and for developers to incorporate their contributions. Please contact me directly if you're interested in this job. These two people will specifically be identified as having the time to spend time helping us design our system in May and test our system during the month of July. On May 20: I will ask Jason to choose the best SCM system and layout he's seen. If at that point it does not meet this project's goals, we will minimally modify it to meet them. May 20-31: The developers and I will figure out the number, kind, and properties of the tools we will develop and divide up the work. This is the real remaining design step. JUNE: During June, the developers and I will build the system and get it ready for initial deployment. The source will be open to for anyone to observe and contribute, but we 4 will have commit access. JULY: On July 1, we will launch the system and put some real packages in it. At that time, we will need MAINTAINERS to join the project. We will deploy the system with REAL packages and REAL maintainers, just a small number of them. Any developer or TU wishing to participate will be allowed to contribute *AT MOST 4 PACKAGES OF RELATIVELY SMALL SIZE* to the test system and they will begin to be maintained in our system. It will be up to the individual developer or TU to decide if they will propagate the PKGBUILD changes from our system back into the official repos. AUGUST: In August, we will evaluate how we have done. If all has gone well, we will hand our system to the Arch development community with a recommendation to enhance and deploy it for main site and AUR repo management. I hope you will all consider joining me for this potentially exciting project! Let's get something done. - P
On 4/27/07, Paul Mattal
All,
Arch needs to move forward. The question: how? Moving forward at high speed is desirable, but we need first make sure we're not pointed towards a cliff and that our safety cables are attached and in good working order.
I am setting my sights on exactly one aspect of this problem: SCM and repo management and getting us from where we are into the new system we all want to have.
BACKGROUND For the purists, there is a sentiment I would like to acknowledge, because it's one that's been bugging me. I believe that the most often overlooked and underrated "Arch Way" design choice was to support i686 only. A single common denominator makes everything a lot simpler-- we just solve one problem and we solve it very well. Development resources are not spread thin and are highly focused on the same end.
I am not the purist I once was. There's a vital and important segment of the community that needs 64-bit support, we can't just drop i686 support, and anyone has to acknowledge that i686 won't be around forever. To survive in the long haul, Arch will need to support more than just the i686 architecture.
With some weight then, knowing we're setting out *against* the Arch Way, but along a trail that *must* be blazed, we consider the landscape.
GOAL Using an open design and implementation process, we will implement a new system for managing Arch repositories and PKGBUILD materials.
I will provide a box with Internet access for this project we can use as our home. It will be completely separate from the main Arch servers. If our system proves itself, it will be presented to the main Arch development team as the new standard for repo management.
Main design choices I am making (to be amended as we go along): 1) The system will be able to manage an arbitrary number of repositories for multiple architectures efficiently.
2) Use of the system will not require a shell account on the server.
3) Multiple instances of the system will be easily deployable on a single server.
4) It will be trivially easy to create and destroy repositories, to copy or move packages between repositories, or to place a package in more than one repository.
5) It will be easy for developers with SCM access to efficiently incorporate patches and input from community members.
6) The system will allow anyone with SCM access to register as a maintainer for a package.
7) All functions of the system will be primarily accessible from a simple commandline interface.
8) The system will be itself deployed as an Arch package and be easy to set up.
9) The system will be sufficiently documented in a functional, if not beautiful, fashion.
SCHEDULE By May 15: Anyone with another SCM system or layout to suggest should do so in the Arch-repos group. Your goal is to persuade Jason your system is the best one. Discussion of all those systems will then take place over the following two weeks.
By May 7: I will bring 7 key people on board to work on this project. Drop me an email if you're interested in joining the team! Initially, I know we will need:
SYSADMIN - An existing Arch developer who will manage the Linux box I provide and the accounts for the SCM system. This person should have at least a small amount of time each day to handle management tasks related to the project.
PUBLICIST - I am committed to organizing this project, and will continue to post status to the arch-dev-public list for all to see. However, not everyone looks here. We need someone who will deliver the news about this project and the status as it progresses to the forums, the IRC channel, and the other mailing lists. This won't be a time-consuming job, but will require someone who has knowledge and facility with all the communications channels used in the Arch community and who can focus on getting the word out.
DEVELOPERS - There will be some straight-up development work needed to get us from the SCM system and layout to an actual management system for repos. We may benefit from some work already done by others (pacman3, AUR, main site repo management tools) but we will need to do some original coding here. I will be somewhat involved in the design and coding, but will choose three (3) people to help with development. ANYONE is a potential candidate.. send me and email and tell me why you'd make a good candidate. Focus on good ideas you have for the system, languages you're comfortable in, how much of Getting Things Done (GTD) you've implemented for yourself, and amount of time you'll have to give to the project in June and July.
COMMUNITY MEMBERS - I will choose two (2) members of the community who specifically are NOT developers or TUs (yet!) but who are interested in helping maintain the packages in our new system. These people will NOT be given SCM access, so as to help us evaluate how easy it is for community members to contribute and for developers to incorporate their contributions. Please contact me directly if you're interested in this job. These two people will specifically be identified as having the time to spend time helping us design our system in May and test our system during the month of July.
On May 20: I will ask Jason to choose the best SCM system and layout he's seen. If at that point it does not meet this project's goals, we will minimally modify it to meet them.
May 20-31: The developers and I will figure out the number, kind, and properties of the tools we will develop and divide up the work. This is the real remaining design step.
JUNE: During June, the developers and I will build the system and get it ready for initial deployment. The source will be open to for anyone to observe and contribute, but we 4 will have commit access.
JULY: On July 1, we will launch the system and put some real packages in it. At that time, we will need MAINTAINERS to join the project.
We will deploy the system with REAL packages and REAL maintainers, just a small number of them. Any developer or TU wishing to participate will be allowed to contribute *AT MOST 4 PACKAGES OF RELATIVELY SMALL SIZE* to the test system and they will begin to be maintained in our system. It will be up to the individual developer or TU to decide if they will propagate the PKGBUILD changes from our system back into the official repos.
AUGUST: In August, we will evaluate how we have done. If all has gone well, we will hand our system to the Arch development community with a recommendation to enhance and deploy it for main site and AUR repo management.
I hope you will all consider joining me for this potentially exciting project! Let's get something done.
Paul, this is great. Thanks for spearheading the organization on this. I am already up to my eyeballs in things to do, otherwise I'd be in your role. Needless to say, I will be participating in all this, so don't fear!
Aaron Griffin wrote:
SYSADMIN - An existing Arch developer who will manage the Linux box I provide and the accounts for the SCM system. This person should have at least a small amount of time each day to handle management tasks related to the project.
DEVELOPERS - There will be some straight-up development work needed to get us from the SCM system and layout to an actual management system for repos. We may benefit from some work already done by others (pacman3, AUR, main site repo management tools) but we will need to do some original coding here. I will be somewhat involved in the design and coding, but will choose three (3) people to help with development. ANYONE is a potential candidate.. send me and email and tell me why you'd make a good candidate. Focus on good ideas you have for the system, languages you're comfortable in, how much of Getting Things Done (GTD) you've implemented for yourself, and amount of time you'll have to give to the project in June and July.
I'd volunteer for the above two. On the other hand, you named quite a few advantages over the old layout. I guess you don't really know what it should look like in detail yet eh?
Cheers, -T
Alexander Baldeck wrote:
Aaron Griffin wrote:
SYSADMIN - An existing Arch developer who will manage the Linux box I provide and the accounts for the SCM system. This person should have at least a small amount of time each day to handle management tasks related to the project.
DEVELOPERS - There will be some straight-up development work needed to get us from the SCM system and layout to an actual management system for repos. We may benefit from some work already done by others (pacman3, AUR, main site repo management tools) but we will need to do some original coding here. I will be somewhat involved in the design and coding, but will choose three (3) people to help with development. ANYONE is a potential candidate.. send me and email and tell me why you'd make a good candidate. Focus on good ideas you have for the system, languages you're comfortable in, how much of Getting Things Done (GTD) you've implemented for yourself, and amount of time you'll have to give to the project in June and July.
I'd volunteer for the above two. On the other hand, you named quite a few advantages over the old layout. I guess you don't really know what it should look like in detail yet eh?
I'm not sure exactly what it will look like yet.. I certainly have some ideas, but I want to enter the design process with a truly open mind. Many of the goals are things I know I need in a system and are things I can imagine a system implementing well in a limited timeframe. I have observed and participated in the thinking process in the arch-repos google group and have faith that whatever that process yields will be a really good consensus on what is the best course of forward action on the SCM piece. I know phrakture has already done some thinking and some implementing on the repo management piece, and I'll want to see what he has done and what advice he has to offer. Thanks for volunteering interest! Right now, the immediate need may be for a sysadmin. I'm hoping to get our "home base" up and running this week and then there'll be plenty of setup to do on it. - P
Paul Mattal wrote:
Thanks for volunteering interest! Right now, the immediate need may be for a sysadmin. I'm hoping to get our "home base" up and running this week and then there'll be plenty of setup to do on it.
- P
Well, I have to present some skills here too as others already did. Sysadmin stuff goes without saying, there's probably not too much that isn't generic that would concern the admin. So just throw stuff at me as it comes in if you'd like. My help as a dev could involve skills like: * Bash scripting * C/C++ coding * Python coding (admitting I'm not too great at it yet) * PHP (probably not needed) Cheers, -U
Paul Mattal wrote:
PUBLICIST - I am committed to organizing this project, and will continue to post status to the arch-dev-public list for all to see. However, not everyone looks here. We need someone who will deliver the news about this project and the status as it progresses to the forums, the IRC channel, and the other mailing lists. This won't be a time-consuming job, but will require someone who has knowledge and facility with all the communications channels used in the Arch community and who can focus on getting the word out.
Sign me up, Paul. I'm in all the right places for that one. T.
Tom K wrote:
Paul Mattal wrote:
PUBLICIST - I am committed to organizing this project, and will continue to post status to the arch-dev-public list for all to see. However, not everyone looks here. We need someone who will deliver the news about this project and the status as it progresses to the forums, the IRC channel, and the other mailing lists. This won't be a time-consuming job, but will require someone who has knowledge and facility with all the communications channels used in the Arch community and who can focus on getting the word out.
Sign me up, Paul. I'm in all the right places for that one.
Great! Welcome aboard! - P
I only really have one suggestion.
JUNE: During June, the developers and I will build the system and get it ready for initial deployment. The source will be open to for anyone to observe and contribute, but we 4 will have commit access.
JULY: On July 1, we will launch the system and put some real packages in it. At that time, we will need MAINTAINERS to join the project.
We will deploy the system with REAL packages and REAL maintainers, just a small number of them. Any developer or TU wishing to participate will be allowed to contribute *AT MOST 4 PACKAGES OF RELATIVELY SMALL SIZE* to the test system and they will begin to be maintained in our system. It will be up to the individual developer or TU to decide if they will propagate the PKGBUILD changes from our system back into the official repos.
AUGUST: In August, we will evaluate how we have done. If all has gone well, we will hand our system to the Arch development community with a recommendation to enhance and deploy it for main site and AUR repo management.
Be careful not to make this system too big. The main site and the AUR have somewhat different requirements. Don't let yourself get bogged down having to implement all those features. Jason
Jason Chu wrote:
Be careful not to make this system too big. The main site and the AUR have somewhat different requirements. Don't let yourself get bogged down having to implement all those features.
Yes, this issue is already on my radar. I'm specifically carving out JUST the SCM and repo management piece, and not focusing on many of the other features that the AUR provides. The system we create will hopefully be a common building block for replacing both systems going forward but will not itself be sufficient to replace either. The goal is to create something lightweight that is simple and flexible enough that using it just to handle these two pieces (SCM and repo management) will just make sense for 99% of use cases where you have groups of people contributing to the development and management of groups of packages. I imagine that a generic web interface will follow on to this project, but one is not intended for this initial project. With the SCM and repo pieces solved, a web interface can be a cleanly separated next project. - P
participants (5)
-
Aaron Griffin
-
Alexander Baldeck
-
Jason Chu
-
Paul Mattal
-
Tom K