On Mon, Dec 22, 2008 at 1:52 PM, Ghost1227 wrote:

+1 Great idea! (one of those why didn't that ever occur to me things)... My only question is, Allan, if you've been thinking of this for a while, why are you hesitating to implement it? And what can we as TUs do to help see this come to fruition/make the transition easier?

I imagine he's been hesitant since it's a pretty massive task. As long as nobody pulls a bfinch on us we could probably make this idea reality. -- Callan Barrett

Ghost1227 wrote:

Callan Barrett wrote:

...

On Mon, Dec 22, 2008 at 1:52 PM, Ghost1227 wrote:

...

+1 Great idea! (one of those why didn't that ever occur to me things)... My only question is, Allan, if you've been thinking of this for a while, why are you hesitating to implement it? And what can we as TUs do to help see this come to fruition/make the transition easier?

I imagine he's been hesitant since it's a pretty massive task. As long as nobody pulls a bfinch on us we could probably make this idea reality.

Has that term been coined as an official TU catchphrase? "Pulling a bfinch?" :P

Lets not... My main thinking in the past was to unify the tools used for the "official" and community repos. That either involved fixing the community/AUR integration to use the new official devtools or split community and the AUR completely. If we split them, then we (being devs and TUs) need to decide whether the community packages get shown on the main Arch package page or are kept separate. As Callan put it, a big change with lots of planning needed... Allan

On 22/12/2008, at 2:14 PM, Allan McRae wrote:

If we split them, then we (being devs and TUs) need to decide whether the community packages get shown on the main Arch package page or are kept separate.

Considering many people have asked for this already, I'd say it would be a good move to have it on the main Arch site. In addition to that, there have been many discussions about (and even work towards) rewriting AUR, and I'm sure this task would be easier if [community] was out of the way (support for binary and PKGBUILD repositories, vs just PKGBUILD repositories). The only downside to this that I see is the loss of the political separation of community from the official repositories. There always seemed to be a wall between the two, with community being an "unsupported" repository. If this repository shows up on the main website, it would seem as though it is officially supported. I'm not sure how that affects the developers.

On Tue, Dec 23, 2008 at 12:54:38AM +0900, Callan Barrett wrote:

On Tue, Dec 23, 2008 at 12:38 AM, Sebastian Nowicki wrote:

...

The only downside to this that I see is the loss of the political separation of community from the official repositories. There always seemed to be a wall between the two, with community being an "unsupported" repository. If this repository shows up on the main website, it would seem as though it is officially supported. I'm not sure how that affects the developers.

Personally, I see that as a massive upside. The wall between TUs and normal developers can be really annoying. Community is already officially supported anyway since it's enabled by default now, if TUs are treating community as though it's not official they're treating it wrong.

Well, the TUs don't really have control over Arch Linux defaults. I think the idea behind community is that it's a bit of a testing grounds for future official packagers. So quality and usefulness of the repo is important but not as important as core or extra. Community is the bridge between unsupported and extra. I believe that correlation should remain pretty explicit as it is now. If community is brought on as another official repo, then the distinction between extra and community is eliminated. Why not just add those packages to extra then? What we really need is a system that can adapt to any type of repo, source based, or binary based. AUR is probably the closest to achieving that, but it has a number of limitations. We'd overcome that by designing a new system. We should be using the same tools for the repos, but I think community should remain a distinct part of AUR.

eliott wrote:

well this thread pretty much died. Bummer. It seemed like such a good idea to me. :/

I think I will just blame Loui for killing it. Loui! Idea killer! boo. hiss!!

;)

I think holidays happened. So here is what I think needs done before this goes ahead: 1) official TU discussion period/vote 2) devtools/db-script support for community, ABS update (I can handle all that) 3) prepare web stuff for switch (AUR and main Arch site depending on what is happening with this.). 3a) actually decide what is happening with web stuff. 4) do the cvs -> svn migration. There still should be the scripts from the main repo migration lying around 5) have a drink to celebrate being finished. I'm going to start at #5... Anyone want to go ahead with organizing #1? Allan

On Wed, Jan 07, 2009 at 12:18:08AM +0100, Xavier wrote:

On Mon, Dec 29, 2008 at 11:44 PM, Loui Chang wrote:

...

Well, the TUs don't really have control over Arch Linux defaults.

I think the idea behind community is that it's a bit of a testing grounds for future official packagers. So quality and usefulness of the repo is important but not as important as core or extra.

Community is the bridge between unsupported and extra. I believe that correlation should remain pretty explicit as it is now. If community is brought on as another official repo, then the distinction between extra and community is eliminated. Why not just add those packages to extra then?

The distinction is exactly the same as now. community repo is managed by a community of Trusted User, while extra is managed by arch developers. It is still a bridge between unsupported and extra. The only difference is that on the implementation level, it would be closer to extra, while now it is closer to unsupported. But on the usage level, it is always in the middle. And community can always be a testing ground for future official packagers : as eliott said, it is even easier to switch from a technical point of view if community is managed just like core/extra.

Well with that, I was answering the desires of some people to have community thrown in the same lot with core and extra. Like having packages listed from the main website rather than aur.archlinux.org for example.

On Wed, Jan 7, 2009 at 6:10 AM, Allan McRae wrote:

Loui Chang wrote:

...

On Wed, Jan 07, 2009 at 12:18:08AM +0100, Xavier wrote:

...

On Mon, Dec 29, 2008 at 11:44 PM, Loui Chang wrote:

...

Well, the TUs don't really have control over Arch Linux defaults.

I think the idea behind community is that it's a bit of a testing grounds for future official packagers. So quality and usefulness of the repo is important but not as important as core or extra.

Community is the bridge between unsupported and extra. I believe that correlation should remain pretty explicit as it is now. If community is brought on as another official repo, then the distinction between extra and community is eliminated. Why not just add those packages to extra then?

The distinction is exactly the same as now. community repo is managed by a community of Trusted User, while extra is managed by arch developers. It is still a bridge between unsupported and extra. The only difference is that on the implementation level, it would be closer to extra, while now it is closer to unsupported. But on the usage level, it is always in the middle. And community can always be a testing ground for future official packagers : as eliott said, it is even easier to switch from a technical point of view if community is managed just like core/extra.

Well with that, I was answering the desires of some people to have community thrown in the same lot with core and extra. Like having packages listed from the main website rather than aur.archlinux.org for example.

I would like to hear Aaron's opinion on whether [community] packages should appear on the main web page rather than the AUR if/when we go for the single repo-tools route. The advantage of having the [community] packages shown on the main page is less maintenance. It also separates the AUR from any repo (which I personally think is a good thing...)

The disadvantage, is [community] becomes "overly official".

I don't really have a problem with it. If community updates begin to dwarf updates for other repos, we may simply have to list more packages, or add multiple RSS feeds, but from a general standpoint, I don't really know any cons here.

On Sun, Dec 21, 2008 at 10:15 PM, Allan McRae wrote:

eliott wrote:

...

Hello!

Long time reader, first time poster (in quite a while). I was bored today, so I read through some of the mailing list archive. Whew!

Anyway, I had one of those crazy ideas that is, well, crazy. As this is the internet and I have some free time, allow me to subject you all to my crazy idea! Hooray!

*coughs*

Why not move towards separating the duties of the current AUR system from the duties of the TU system. I propose making the TUs 'associate arch devs', and having them use the main arch repository system, but in a reduced security/permission role. Something like TUs getting access to community only, like they have now..but using the same dev tools system as the mainline devs, having community just be 'another repo' in the main site interface (just like extra and core).

Sure it will take a bit of effort, but it would streamline some processes for sure!

Detriments: - TUs wouldn't be as autonomous as they historically have been. This could be good or bad, but putting it in the 'bad' category, because people fear change..and this is a change. - It would take _some amount_ more than 0 value of effort to get the TUs into the mainline dev system.

Benefits: - Less tools to support, as the TUs would then use the same dev tools as devs, but with limited permissions. - Process streamline - Clearer path for migration from 'associate dev' to 'dev'. Just flip a bit in the permissions for the devtools, and they can now move packages to extra or core, etc. - Decouple community from the AUR. The AUR would then 'be its own bag'. Leaving end user voting as a guide for moving packages into community (or not..whatever), then packages in community would disappear from the aur when they were adopted into community. - Greater visibility for community repository. It is already included by default, so it should be treated as such... a vital component to the overall arch ecosystem. - Future AUR rewrites or refactoring would be greatly simplified. - The existing TU toolchain and server daemon can be jettisoned into the abyss, to live with other lovecraftian horrors.

So there you have it. Talk amongst yourselves. I will be over here..hiding in the shrubbery. o.O

I have been thinking about this for a while. Another advantage is that [community] can easily have a [community-testing] repo or even just use [testing] directly. And given we currently can given permissions for only using [extra] or [core], this would be fairly simple to do.

I've been thinking of this as well. It would require some design effort from the server side though, but wouldn't be too hard. Do we need full shell accounts for these users? yes: Can be done in a minimal amount of time with very little changes to the dbscripts. Management headache with regard to TUs vote new people in periodically. no: Would require a restricted sftp process, or some daemon for file upload. Less management hassle, more code writing. One of the good things here is that we could very easily allow some TUs who have the gusto to commit to the main repo SVN (even temporarily) to do changes like fixing licenses and the like. We can give permissions to commit to the repos without permissions to upload to the repos. As I stated on the "backend rewrite" thread - I am raring to go and anyone who wants to help out with the backend tools should contact me offlist (things in my inbox take higher priority, usually). Anyway, thanks for the mail cactus. It's always helpful to see ideas thrown around. I've thought about this for some time, but feared the TUs would be generally against it.

On Mon, Dec 22, 2008 at 10:15 PM, Allan McRae wrote:

Aaron Griffin wrote:

...

I've been thinking of this as well. It would require some design effort from the server side though, but wouldn't be too hard. Do we need full shell accounts for these users?

yes: Can be done in a minimal amount of time with very little changes to the dbscripts. Management headache with regard to TUs vote new people in periodically.

yes: People can stage their rebuilds.

...

no: Would require a restricted sftp process, or some daemon for file upload. Less management hassle, more code writing.

no: keeps updating packages the same as now (upload and forget). No manual db-script running.

For TUs who want to know how the main repo update procedure works, see http://wiki.archlinux.org/index.php/DeveloperWiki:HOWTO_Be_A_Packager . The main difference to current [community] updating is that packages are uploaded to a staging area which requires you to run and extra script to actually push them to the repos.

If we are going to have a testing repo for community (which I think is overdue and this change would make easy to implement), I think TUs will need full shell access in order to move the packages from the testing repo to the main one.

Quick aside: Do we need ANOTHER testing repo for community, or could we use the same one?

On Tue, 23 Dec 2008, Allan McRae wrote:

Aaron Griffin wrote:

...

On Mon, Dec 22, 2008 at 10:15 PM, Allan McRae wrote:

...

Aaron Griffin wrote:

...

I've been thinking of this as well. It would require some design effort from the server side though, but wouldn't be too hard. Do we need full shell accounts for these users?

yes: Can be done in a minimal amount of time with very little changes to the dbscripts. Management headache with regard to TUs vote new people in periodically.

yes: People can stage their rebuilds.

yes: For people who maintain packages in both core/extra and community, everything would be centralized. I guess the community section of bug tracker would also be merged with the regular ArchLinux one. For this reason, it's a big +1 from me (note I'm not a TU anymore).

...

...

...

no: Would require a restricted sftp process, or some daemon for file upload. Less management hassle, more code writing.

no: keeps updating packages the same as now (upload and forget). No manual db-script running.

For TUs who want to know how the main repo update procedure works, see http://wiki.archlinux.org/index.php/DeveloperWiki:HOWTO_Be_A_Packager . The main difference to current [community] updating is that packages are uploaded to a staging area which requires you to run and extra script to actually push them to the repos.

If we are going to have a testing repo for community (which I think is overdue and this change would make easy to implement), I think TUs will need full shell access in order to move the packages from the testing repo to the main one.

Not really. I guess the main use of the community-testing repo would be when revbuild are done. In this case, the dev who moves the rebuild from testing to core/extra could move the rebuilds out of community-testing at the same time.

...

...

Quick aside: Do we need ANOTHER testing repo for community, or could we use the same one?

Decision pending... I have alway thought the switch to SVN would easily allow a testing repo for community. But that was when I was thinking that community repo would still be entirely separate. If the repos are going to be all in the same place, there is little point creating another repo.

Allan

I think it all boils down to what would be the easiest/best solution in terms of permissions handling. If we don't give them access to the server, then a separate community-testing that would be updated with a daemon like community might be better. We also want the TU to only be able to put only community package in testing, modify them in testing if needed and then move them back to community. If they have access to the server to run the db scripts and we can set proper permissions on the testing repo then we could just have one testing repo. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

Abhishek Dasgupta wrote:

2008/12/23 Eric Bélanger :

...

I think it all boils down to what would be the easiest/best solution in terms of permissions handling. If we don't give them access to the server, then a separate community-testing that would be updated with a daemon like community might be better. We also want the TU to only be able to put only community package in testing, modify them in testing if needed and then move them back to community. If they have access to the server to run the db scripts and we can set proper permissions on the testing repo then we could just have one testing repo.

Why not merge extra and community, and give permissions accordingly? Even now there are quite a few packages in extra that are orphaned or have not been updated in ages. Some popular packages were moved to community recently, for example. Thus we need to shift around packages between repositories depending on who is maintaining it (dev or TU). If extra and community were merged, such problems would not be there. Some packages are more important, and permissions could be set to allow only certain people to write to that package directory.

AFAIK, it is a lot easier to give permissions for certain repos rather than parts of repos, though I could be wrong. Having the [community] repo set up like [core] and [extra] would mean that moving packages to/from community would involve running a single script, much like currently moving packages from [testing] to the main repos. Allan

On Tue, 23 Dec 2008, Allan McRae wrote:

Abhishek Dasgupta wrote:

...

2008/12/23 Eric Bélanger :

...

I think it all boils down to what would be the easiest/best solution in terms of permissions handling. If we don't give them access to the server, then a separate community-testing that would be updated with a daemon like community might be better. We also want the TU to only be able to put only community package in testing, modify them in testing if needed and then move them back to community. If they have access to the server to run the db scripts and we can set proper permissions on the testing repo then we could just have one testing repo.

Why not merge extra and community, and give permissions accordingly? Even now there are quite a few packages in extra that are orphaned or have not been updated in ages. Some popular packages were moved to community recently, for example. Thus we need to shift around packages between repositories depending on who is maintaining it (dev or TU). If extra and community were merged, such problems would not be there. Some packages are more important, and permissions could be set to allow only certain people to write to that package directory.

AFAIK, it is a lot easier to give permissions for certain repos rather than parts of repos, though I could be wrong. Having the [community] repo set up like [core] and [extra] would mean that moving packages to/from community would involve running a single script, much like currently moving packages from [testing] to the main repos.

Allan

I was thinking the same thing. If we merge extra and community, we'll need to set permissions on a package-by-package basis instead of a repo-by-repo basis. Better keep them separate. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

On Tue, Dec 23, 2008 at 4:22 PM, Abhishek Dasgupta wrote:

2008/12/23 Eric Bélanger :

...

I was thinking the same thing. If we merge extra and community, we'll need to set permissions on a package-by-package basis instead of a repo-by-repo basis. Better keep them separate.

Either way, if it is decided that community should move to the devtools backend, how hard would it be to decouple the AUR from community?

-- Abhishek

No too hard, there's no issue with that. You would basically stop the daemon running that adds packages to the database and clear out anything currently in community. There's not a whole lot of community specific code in the AUR. -- Callan Barrett

On Tue, Dec 23, 2008 at 4:43 PM, Aaron Griffin wrote:

Additionally, the AUR is going to need to be able to import or use info from the main package DBs in this case. This gives us a lot of advantages, but forces us to couple some things. Unless, of course, you wanted to go the route the main site went: parse a db.tar.gz file and put the data into the DBs.

I suppose I'm missing something here. If we moved community to a normal repo why would the AUR have any notion of what the main repos are and what they contain? I'm thinking of stuff like checking for duplicates packages but I don't know if that's what you're talking about. -- Callan Barrett

2008/12/23 Aaron Griffin :

I was thinking in terms of linking deps properly on the package pages. Not sure how that's done now.

Though, the "checking for duplicate packages" might be a good idea too 8)

If the main site had a function like this: http://archlinux.org/pkg/pacman which would take the user to the pacman info page (i686), then the linking of dependencies on the package pages would be trivial; or we can just find out which repo a package is in by parsing the db.tar.gz and linking accordingly. Either way, it isn't much of a problem. -- Abhishek

2008/12/23 Aaron Griffin :

This works, but needs the repo. Unless that's what you're saying here (that the repo is needed) http://archlinux.org/packages/core/i686/pacman/

What I meant was, we could do away with parsing db.tar.gz in AUR for linking dependencies if there was a way to directly go to the package page without needing the repository name. -- Abhishek

2008/12/22, Abhishek Dasgupta :

Why not merge extra and community, and give permissions accordingly?

-1 I think that is a wrong idea. Actually, the community repo is too big to merge extra and community. Those repos are and always should be separate, imho. My 2 cents. -- Arch Linux Developer (voidnull) AUR & Pacman Italian Translations Microdia Developer http://www.archlinux.it

I love the idea as well. :) If it's easy to implement, like Allan pointed out, I say we go for it!