On Tue, Dec 2, 2008 at 9:01 PM, Aaron Griffin wrote:

On Tue, Dec 2, 2008 at 5:39 PM, Allan McRae wrote:

...

This starts the official discussion period for the addition of rules governing the addition of packages to [community]. As this is essentially a bylaw change, we will use that voting procedure: 5 days discussion, 7 days voting, quorum of 75% required.

[proposal]

* Only "popular" packages may enter the repo, as defined by 1% usage from pkgstats or 10 votes on the AUR.

* Automatic exceptions to this rule are: - i18n packages - accessibility packages - drivers - dependencies of packages who satisfy the definition of popular, including makedeps and optdeps - packages that are part of a collection and are intended to be distributed together, provided a part of this collection satisfies the definition of popular

* Any additions not covered by the above criteria must first be proposed on the aur-general mailing list, explaining the reason for the exemption (e.g. renamed package, new package). The agreement of three other TUs is required for the package to be accepted into [community]. Proposed additions from TUs with large numbers of "non-popular" packages are more likely to be rejected.

* TUs are strongly encouraged to move packages they currently maintain from [community] if they have low usage. No enforcement will be made, although resigning TUs packages may be filtered before adoption can occur.

[end proposal]

I throw something in here during the official discussion period, directed at all the people saying "omg these metrics suck" (regarding pkgstats and votes).

The fact is, it's come to all our attention that we need _some_ way to control packages in community. These are the only metrics we have at the moment. The above bylaw, no matter what the actual metric used, is a decent one. Modifying the metric at a later time can and should be done, but for now these are the only metrics we have.

Simply put: some structure here is needed. If this is all we have right now, we should do it, rather than say "screw it, let's stick with this freeform thing we've been doing". If we find these metrics to be wanting, we can change it later. And seriously, how hard is it to go onto IRC or the forums and say "I want to put Foo in community, and need 8 more votes please!"

Well Aaron, even proponents of this proposal have said that the statistics and/or votes are a problem as far as accuratcy. Heck a simple examination shows that the voting system has over two orders of magnitude for a variable and NO basis for determining the shape of the possible distribution of votes , so no understanding of what then constitutes a mean either. AND you have again said it is needed to be doing this. So for the past several weeks many have been asking why. Would you PLEASE explain why this is needed. In detail, and with some statistics that can be verified and have a good basis in compilation. As it would be NICE to know exactly WHY this is NEEDED. And how this change accomplishes such a needed change. Best regards; Bob Finch

On Fri, Dec 5, 2008 at 09:56, bardo wrote:

. Don't put any hard limits at all, just encourage contructive discussion and communication between TUs themselves, the community and devs. I personally trust all of you TUs and I think that, by better defining what our goals are and how we want our repo to be, we can solve this problem without much hassle.

Corrado Primier

First, thank you for the well thought-out non-flame post. There hasn't been enough of that lately. I think that regardless of how the vote goes, that we should work on improving this. Obviously, I can't see inside peoples personal emails, but I see next to no communication between TUs except for the crowd that hangs out on IRC. I don't really have anything constructive to add regarding how we should improve it, but I think it's important to do so. Now, regarding the proposal. I think that we should edit it so that the current proposal is for a three month probationary period during which the proposed changes will be used as rules, and at the end of which time we can vote again on whether to make the change permanant. I don't know if I make that clear, but if anyone has a question, please ask. I think this would address some of the concerns of such changes being difficult to undo at a later date, as it should give us an easy out if the changes prove to be ineffectual or overly onerous. --Daenyth

On Fri, Dec 5, 2008 at 10:13, Callan Barrett wrote:

But why? Why make 2 definite sets of votes when it's just as easy to bring this up again if it's not working out? This just seems like an out for people who don't want this to happen, if you don't like it you should vote no on it.

-- Callan Barrett

I'm in favor of the current proposal, but it's been brought up before, by bfinch I believe, that it might become difficult to remove this at a later date. He never explained why it would be difficult, but I thought that this was a decent compromise to alleviate such concerns. I could be way off base though on this, though. I haven't really given the idea a lot of thought.

On Fri, Dec 5, 2008 at 8:56 AM, bardo wrote:

On Wed, Dec 3, 2008 at 12:39 AM, Allan McRae wrote:

...

This starts the official discussion period for the addition of rules governing the addition of packages to [community]. As this is essentially a bylaw change, we will use that voting procedure: 5 days discussion, 7 days voting, quorum of 75% required.

Since this is an official discussion period, I'll state my opinions in a (hopefully!) brief form. I've been thinking about the whole thing for quite a while now, evaluating all opposing opinions, and I have made up my mind. Any reply is welcome.

I *won't* let the discussion end up in a flame, so don't expect an answer to provocative e-mails. I have seen, on both sides of the fence, a lot of people falling in the quicksands of 'a priori' arguments. I'm trying to address the problem and the solutions I can think of from a rational point of view.

My succinct conclusions: . Everybody agreed that we weren't able to find a statistically relevant way to calculate package usage. . Imposing limits, albeit relaxed, on TUs will, in my opinion, demotivate many of them. I'll be the first to feel demotivated. . We are using as a reference the pkgstats results that came from very few days of usage. If we really want to put arbitrary limits, then they should be discussed after *at least* one month of pkgstats running. I already wonder: did results change from when we started the discussion? . Even if we restrict package uploading in some way, this won't mean we'll have solved the resource problem: it will only be postponed. If we reduce package intake by 10% (aka: more than it was proposed) it will only take ~11% more time to hog the server resources again, supposing all packages have the same weight. It was demonstrated that the least used packages are pretty small, though. Not a real solution, imho. . With this vote we don't consider that we are *not* totally independent from Arch, even though we theorically are. As Aaron rightly stated, we also need to consider their opinion about the matter. A TU-only vote does exactly the opposite, until an official proposal comes from them.

These are the reasons why I intend to REJECT the proposal.

My (hopefully constructive) alternative proposals: . Ask someone who knows well what s/he's taking about (a statistician) if s/he can come up with a way to get better numbers from the means we have, with the constraints we have, then decide how to act. . Work with widely used third-party applications (aurvote, yaourt) to better exploit the mean of voting, and get more useful numbers. . Don't put any hard limits at all, just encourage contructive discussion and communication between TUs themselves, the community and devs. I personally trust all of you TUs and I think that, by better defining what our goals are and how we want our repo to be, we can solve this problem without much hassle.

This is something that was brought up by Thayer in a personal email to me, and I just wanted to throw this out there. This is my summary of _his_ thoughts, and not my opinion exactly. What we're trying to address here is not packages getting _into_ community, but package _rot_ (ed: I *love* the term "package rot"). That is, it's not the volume of packages going into community that's the issue, it's the fact that none of them leave. A package that was popular a month ago may not be popular today. Thayer proposed adding flagging for packages that would indicate: a) that it doesn't follow packaging standards and b) that it is "rotting" or no longer useful. This would do a few things: clean up the repo, keep TU motivation up, and still allow us to get new-fangled packages in the community repo. The thing I like about this is that a lot of times, something new comes out (say dwm) and people are all "omg I want to try it!" but want it to be easy. That initial popularity seems, theoretically, enough to move it into community. But the popularity of a lot of these new-fangled apps tends to wane quickly. It makes sense to have a package like this in community as soon as one can put it there, but we need checks and balances to decide if it should be removed. Again, this is not my idea, and I'm not saying "omg let's do this instead". I'm just throwing it out there to try to get real discussion going.

On Fri, Dec 5, 2008 at 6:26 PM, Aaron Griffin wrote:

What we're trying to address here is not packages getting _into_ community, but package _rot_ (ed: I *love* the term "package rot"). That is, it's not the volume of packages going into community that's the issue, it's the fact that none of them leave. A package that was popular a month ago may not be popular today. Thayer proposed adding flagging for packages that would indicate: a) that it doesn't follow packaging standards and b) that it is "rotting" or no longer useful. This would do a few things: clean up the repo, keep TU motivation up, and still allow us to get new-fangled packages in the community repo.

And in the same spirit, it would be wonderful if the community repository was accessible at some point from the archlinux.org package search like it happens with archlinux.de, i am a long time fan of. Pierres and anyone else who have worked on have done a wonderful job. That was part of what i meant by "being official" in my previous mails. BTW Bardo's mail was excellent and right to the point. Greg

Aaron Griffin wrote:

...

My succinct conclusions: . Everybody agreed that we weren't able to find a statistically relevant way to calculate package usage. . Imposing limits, albeit relaxed, on TUs will, in my opinion, demotivate many of them. I'll be the first to feel demotivated. . We are using as a reference the pkgstats results that came from very few days of usage. If we really want to put arbitrary limits, then they should be discussed after *at least* one month of pkgstats running. I already wonder: did results change from when we started the discussion?

<snip>

...

better exploit the mean of voting, and get more useful numbers. . Don't put any hard limits at all, just encourage contructive discussion and communication between TUs themselves, the community and devs. I personally trust all of you TUs and I think that, by better defining what our goals are and how we want our repo to be, we can solve this problem without much hassle.

<snip>

Thayer proposed adding flagging for packages that would indicate: a) that it doesn't follow packaging standards and b) that it is "rotting" or no longer useful. This would do a few things: clean up the repo, keep TU motivation up, and still allow us to get new-fangled packages in the community repo.

The thing I like about this is that a lot of times, something new comes out (say dwm) and people are all "omg I want to try it!" but want it to be easy. That initial popularity seems, theoretically, enough to move it into community. But the popularity of a lot of these new-fangled apps tends to wane quickly. It makes sense to have a package like this in community as soon as one can put it there, but we need checks and balances to decide if it should be removed.

i like this idea, but i have a concern. i think better statistics are essential for a successful long-term solution to this problem. however, i think a technical fix will only go so far; there are *social* problems that will limit any technical fix. example 1: votes are unreliable because voting users are not a representative sample. example 2: flagging packages as anything doesn't guarantee any course of action, in any of the repositories. bitlbee, in [extra] is 0.2.1 versions behind the upstream release. it has been flagged-out-of-date. the maintainer has been contacted individually. an updated pkgbuild has been sent to the maintainer and posted on the forums. but it's only getting older in [extra]. on the one hand we have users not availing themselves of the participation mechanisms that are available. on the other hand there are maintainers not responding to those users who do follow-through. so, while the aaron/thayer amendment to the proposal (or is it a separate proposal?) provides a couple useful new statistical measures, i don't see that it would actually generate better statistics. several ideas have been floated to solve this particular problem, like download statistics. those all need more consideration and development, though. it seems to me that there won't be any real consensus on a concrete proposal to regulate [community] until there's a mechanism for generating accurate, reliable usages statistics. i would anticipate a close vote; given the furor that's surrounded this proposal, i would also anticipate a lot of bad feeling on both sides arising from a close, binding vote. if i were a tu, i'd move to table this proposal and form a working group to study the social and technical problems of generating good usage statistics. it would put off a resolution to the resource consumption problems, but i feel that, sometimes, "now" is not better than "better." in the meantime, it seems that moving all the games out of [community] and into [games] is one concrete and non-controversial way to take some load off the server. sorry to go on at length, especially as i'm not even a tu. yrs, kludge

replying in-line here:

CCing Jeff on here so he remembers this. :) The rest is a response to the actual content.

oh, damn, sorry... if already bugged the hell out of him. that wasn't meant as a poke, just an example!

...

so, while the aaron/thayer amendment to the proposal (or is it a separate proposal?) provides a couple useful new statistical measures, i don't see that it would actually generate better statistics. several ideas have been floated to solve this particular problem, like download statistics. those all need more consideration and development, though.

If others think it is a good idea, then they must at least help out with the work.

granted. and i don't think i have the skills to make that happen. but i think there are others who've proposed alternatives who might.

...

it seems to me that there won't be any real consensus on a concrete proposal to regulate [community] until there's a mechanism for generating accurate, reliable usages statistics. i would anticipate a close vote; given the furor that's surrounded this proposal, i would also anticipate a lot of bad feeling on both sides arising from a close, binding vote.

if i were a tu, i'd move to table this proposal and form a working group to study the social and technical problems of generating good usage statistics. it would put off a resolution to the resource consumption problems, but i feel that, sometimes, "now" is not better than "better."

This sounds like way more bureaucracy (man, I can never spell that word) than we need or want. And with regard to your now/better point: I think it'd a good general rule that 75% now is better than 100% a year from now

<rant type="anarcho_nerd"> avoiding beauraucracy (neither can i) is essential, so that the system doesn't start running the users. but sometimes effecting social organizations/algorithms is necessary to keep some users from running other users. i think the latter is one of the underlying-if-unspoken concerns afloat in this discussion. that's why i'm pimping formal consensus processes here. they take serious engagement, commitment to collaboration, trust, and time. my experience, though, tells me that the benefits--collective buy-in and global conflict resolution--outweigh the overhead. arch is getting bigger and bigger social groups need to emerge *some* form of internal structure, or they fracture into smaller groups that can be self-managing without those structures. <rant type="anarcho_nerd"> dog, we're gonna have to change the subject-line *again*! -kludge

Hi, all, I generally just lurk on this list (as I'm not a TU), but figured I might throw in a couple of cents to this discussion, since there's some talk about statistics, and I have a strong statistical background. Oh, and as it turns out, "a few cents" ends up being something like a dollar. Sorry for the long post, but I'm trying to be thorough. Some of the points that have already been made: (Yes, everybody hates a recap. Deal.) - The current votes system is insufficient as a valid means of determining how many people *really* use a package, because not all users vote. - pkgstats hasn't been around long enough to have generated as much information as folks would like. (And, again, not everybody has installed/used it). - A system for reliably working out what packages have a sufficiently large user base in Arch to justify inclusion in [community] would be a very nice thing to have. My thought is basically that the best way to get decent usage stats for a given package is to implement a pkg-download counter (yes, I know this has been suggested). Unfortunately, this raises some technical issues. Particularly (read: off the top of my head), we'd need a mechanism to account for fringe cases such as people who, say, accidentally remove or break a PKGBUILD and re-download five minutes later, and the (probably/hopefully much more rare) people who would repeatedly download a PKGBUILD in order to artificially inflate its usage statistics. These could probably be taken care of with, say, an IP-address recorder (but that of course raises privacy issues of its own) or requiring users to login in order to download (which I *don't* think is a good idea; it has wa-a-a-a-ay too many obvious drawbacks). It would also be useful not to double-count people who are merely updating to a new version, and to have some means of keeping track of people who have *removed* packages. Pkgstats is a good start in the direction of keeping reliable statistics. Some ideas for making that more prevalent are: - Including a new, well-publicized (so people know they can turn it off) function in pacman that would (optionally, default=yes) to automagically send in installed/removed pkgs (and I suppose a complete list the first time it's run) to something like Debian's popularity-contest server. - Supplementing pkgstats data, where lacking, with data from Debian's popularity-contest server. - Adding a cronjob for pkgstats (again, enabled by default). - Adding a feature to pkgstats that would also *save* the full list it has sent in (preferably to a compressed file, but whatever), and then next time it's run, check against the saved list--packages that have been removed are removed from this master list, and packages that have been added are, well, added to the list. Something like a diff would then be sent in. - Adding pkgstats to the install CD as an optional, but useful, package. So. That's all I can think of on this for now, but if folks would like statistics advice, I'm on this and a couple arch lists, and feel free to CC me on the email or whatever just to get my attention. If someone has specific questions about something or about something I said here, ask! And I'd be glad to help with implementing...whatever gets decided. Thanks for reading this far! Ivy P.S.: For what it's worth, I agree that the actual vote should be tabled until there are one or more solid, researched proposals (which needn't take more than a week or two!). (And not to downplay the actual suggestion in question!) And I'd be glad to help out with that, too, though again I'm not a TU. (-: -- If I Ever Become An Evil Villainess... 46. If an advisor says to me "My liege, the heroine is but one person. What can one person possibly do?" I will reply "This," and kill the advisor.

On Sun, Dec 7, 2008 at 8:59 AM, Aaron Griffin wrote:

Functionality such as this will never ever make it into pacman. Ever. I know I'm speaking for Dan here, but I'm confident he feels the same. Pacman will never ever upload anything such as usage statistics. Pacman is a package manager. External scripts can and should handle this.

I'm unsure why other solutions even come up at all. Barring impossible and invasive solutions such as monitoring every single one of our mirrors or forcing people to upload statistics I'd like someone to come up with a "better" solution than what we currently have in voting and pkgstats. Also keep in mind that voting for packages isn't meant to be seen as a "this percentage of users use this package", it's a vote. If you like a package you vote for it, if you'd like a package to go into community you vote for it. It doesn't mean that whoever voted represents X amount of users overall (I guess in this case it also means we're coming up with a fairly arbitrary number for how many votes are required but as long as it's thought out properly it wouldn't matter). It's perfectly valid for what we need it for. -- Callan Barrett