[arch-dev-public] Possible signoff addendum
One of the things I was thinking about, because sometimes signoffs can stagnate.... Should we have a time-based limit? That is, if a package has been in testing for X days, with no complaints, but no signoffs, yet, can we consider it fully functional, assuming the packager is comfortable with that. I'd like opinions on this. I'm trying to solve the issue of the big ol' backlog of pending-signoff packages. Cheers, Aaron :wq
Aaron Griffin wrote:
One of the things I was thinking about, because sometimes signoffs can stagnate....
Should we have a time-based limit?
That is, if a package has been in testing for X days, with no complaints, but no signoffs, yet, can we consider it fully functional, assuming the packager is comfortable with that.
I'd like opinions on this. I'm trying to solve the issue of the big ol' backlog of pending-signoff packages.
I quite like this idea. A "no bug report after given period of time = no problems" strategy should work as the community can do a lot more testing in total than we can. Allan
2008/8/31 Aaron Griffin <aaronmgriffin@gmail.com>:
One of the things I was thinking about, because sometimes signoffs can stagnate....
Should we have a time-based limit?
That is, if a package has been in testing for X days, with no complaints, but no signoffs, yet, can we consider it fully functional, assuming the packager is comfortable with that.
I'd like opinions on this. I'm trying to solve the issue of the big ol' backlog of pending-signoff packages.
I can imagine this causing people to stop signing off altogether ("meh, it'll be signed off automatically in a couple weeks"). This could lead to people not testing altogether or to a "packages must spend X days in testing" philosophy in the long run. This could knock the 'bleeding edge' off the Arch. I think its better to find a way to encourage people to do more signing off. Some kind of incentive or motivation. Some ideas: - policy to do at least two signoffs for every package submitted to testing (bump it to three or four in times where testing is backlogged) - count the number of signoffs people do on the web based signoff and reward the big signoffers with recognition and the non-signoffers with negative recognition (possibly just a list of top three and bottom three signoffers on the dashboard or signoffs page). This sounds a bit big brotherish... - somehow flag packages that have been in testing for an unreasonable amount of time. Possibly sorting the signoffs list by length in testing, listing exceptionally long-time testing packages in the dashboard, e-mailing the list periodically with a list of such packages, etc. Another alternative: - Create a web based interface to allow community users to signoff on packages. We could have a policy of X number of user signoffs is equal to one developer signoff. Developers can be trusted to test it better (or not? ;-), but arch members can contribute too. In that case we still need some incentive to the end users to bother doing this. There aren't a lot of people like dolby and cactus who seem to actually like working really hard with as little recognition as possible. Dusty
On Sun, Aug 31, 2008 at 11:11 AM, Dusty Phillips <buchuki@gmail.com> wrote:
Another alternative:
- Create a web based interface to allow community users to signoff on packages. We could have a policy of X number of user signoffs is equal to one developer signoff. Developers can be trusted to test it better (or not? ;-), but arch members can contribute too. In that case we still need some incentive to the end users to bother doing this. There aren't a lot of people like dolby and cactus who seem to actually like working really hard with as little recognition as possible.
I actually thought about this a while back. It was in terms of "each package needs 10 points, developer signoffs are 5 points, TU signoffs are 3 points, user signoffs are one point" It might be a useful way to find active community members too. If you could think of a way to creatively implement something like this, I'd be all for it, assuming other people are. But yeah, I think you may be right about the "encouraging people not to sign off at all" part. It would make people lazy.
Aaron Griffin wrote:
On Sun, Aug 31, 2008 at 11:11 AM, Dusty Phillips <buchuki@gmail.com> wrote:
Another alternative:
- Create a web based interface to allow community users to signoff on packages. We could have a policy of X number of user signoffs is equal to one developer signoff. Developers can be trusted to test it better (or not? ;-), but arch members can contribute too. In that case we still need some incentive to the end users to bother doing this. There aren't a lot of people like dolby and cactus who seem to actually like working really hard with as little recognition as possible.
I actually thought about this a while back. It was in terms of "each package needs 10 points, developer signoffs are 5 points, TU signoffs are 3 points, user signoffs are one point"
It might be a useful way to find active community members too. If you could think of a way to creatively implement something like this, I'd be all for it, assuming other people are.
But yeah, I think you may be right about the "encouraging people not to sign off at all" part. It would make people lazy.
It's not necessarily a bad thing to have the signoffs come from testers who are not developers-- it saves time for developers to be developing if others are testing. I know I test when I can, but usually get around to it less when I have a backlog of developing to do already. So it might be a better division of labor to bring actual testers on board who like testing and who help us out by doing that. With the web interface, we should be able to give them just access to signoffs. Just a thought. - P
On Tue, 2008-09-02 at 08:02 -0400, Paul Mattal wrote:
It's not necessarily a bad thing to have the signoffs come from testers who are not developers-- it saves time for developers to be developing if others are testing.
I know I test when I can, but usually get around to it less when I have a backlog of developing to do already.
So it might be a better division of labor to bring actual testers on board who like testing and who help us out by doing that. With the web interface, we should be able to give them just access to signoffs.
Just a thought.
- P
+1, Agree.
On Tue, 2008-09-02 at 08:02 -0400, Paul Mattal wrote:
It's not necessarily a bad thing to have the signoffs come from testers who are not developers-- it saves time for developers to be developing if others are testing.
I know I test when I can, but usually get around to it less when I have a backlog of developing to do already.
I would be happy to test more packages, but I am hesitant to open up the testing repo on my system after having some bad encounters in the past. I've also got an arch in virtualbox, but that doesn't really help for hardware-related stuff like the recent 3945 wifi update. How do you other devs manage your test packages? Do you use separate test machines or are ya just livin' on the edge? Any best practises you'd like to share?
On Fri, Sep 12, 2008 at 1:39 PM, Thayer Williams <thayer@archlinux.org> wrote:
On Tue, 2008-09-02 at 08:02 -0400, Paul Mattal wrote:
It's not necessarily a bad thing to have the signoffs come from testers who are not developers-- it saves time for developers to be developing if others are testing.
I know I test when I can, but usually get around to it less when I have a backlog of developing to do already.
I would be happy to test more packages, but I am hesitant to open up the testing repo on my system after having some bad encounters in the past.
I've also got an arch in virtualbox, but that doesn't really help for hardware-related stuff like the recent 3945 wifi update.
How do you other devs manage your test packages? Do you use separate test machines or are ya just livin' on the edge? Any best practises you'd like to share?
I just run testing. I haven't had anything break that wasn't my own fault in ages
On Fri, Sep 12, 2008 at 1:39 PM, Thayer Williams <thayer@archlinux.org> wrote:
On Tue, 2008-09-02 at 08:02 -0400, Paul Mattal wrote:
It's not necessarily a bad thing to have the signoffs come from testers who are not developers-- it saves time for developers to be developing if others are testing.
I know I test when I can, but usually get around to it less when I have a backlog of developing to do already.
I would be happy to test more packages, but I am hesitant to open up the testing repo on my system after having some bad encounters in the past.
I've also got an arch in virtualbox, but that doesn't really help for hardware-related stuff like the recent 3945 wifi update.
How do you other devs manage your test packages? Do you use separate test machines or are ya just livin' on the edge? Any best practises you'd like to share?
Testing hasn't broken my machine beyond a quick 5 minute fix once in the last 6 months. I think we are getting a lot better at self-testing before blindly pushing packages. It isn't like you have to -Syu every day either- just take a glance at the ML and see if there are any "OMG broken" messages there before upgrading. -Dan
Thayer Williams schrieb:
I would be happy to test more packages, but I am hesitant to open up the testing repo on my system after having some bad encounters in the past.
I've also got an arch in virtualbox, but that doesn't really help for hardware-related stuff like the recent 3945 wifi update.
How do you other devs manage your test packages? Do you use separate test machines or are ya just livin' on the edge? Any best practises you'd like to share?
I use testing on all my machines. I occasionally hold back on a few updates when I know they might break and I have no time to fix it. I usually know how to fix stuff though.
2008/9/2 Aaron Griffin <aaronmgriffin@gmail.com>:
On Sun, Aug 31, 2008 at 11:11 AM, Dusty Phillips <buchuki@gmail.com> wrote:
Another alternative:
- Create a web based interface to allow community users to signoff on packages. We could have a policy of X number of user signoffs is equal to one developer signoff. Developers can be trusted to test it better (or not? ;-), but arch members can contribute too. In that case we still need some incentive to the end users to bother doing this. There aren't a lot of people like dolby and cactus who seem to actually like working really hard with as little recognition as possible.
I actually thought about this a while back. It was in terms of "each package needs 10 points, developer signoffs are 5 points, TU signoffs are 3 points, user signoffs are one point"
It might be a useful way to find active community members too. If you could think of a way to creatively implement something like this, I'd be all for it, assuming other people are.
The obvious implementation would be to give people the option to sign up for an account and to give them signoff permission (as Paul suggested). This is easy to implement but it means non-devs have to have access to dev.archlinux.org or we have to break some of the caching on the public site (which would make cactus weep). It also means yet another fricking user account that people have to sign up for. A less invasive option would be to add a 'signoff this package' link to testing packages just like there is currently a 'flag out of date' link. But it would be hard to police if some asshat wanted to come through and click the signoff button 50 times even though the package was thoroughly busted. There have been some bug reports suggesting linking packages to flyspray tickets (or vice versa), and I have a personal vendetta against flyspray so another option would be to create a new package-based bug tracker and port existing bugs and user accounts from flyspray. Then anyone with a 'Arch Linux KISS Bugtracker' account would also have the ability to signoff on packages. So people could actively log in and either say "yo this package works so good I want it to go live (ie: signoff)" or "wtf this package is broken, fix it fix it whine whine whine (ie: attach a bug to the package)". The question isn't so much which of these to do as how much time do I have to commit to it (within epsilon of 0), how many other awesome people want to submit patches (approximately 2), and how soon do we need it (two months back?) Dusty PS: sorry for not replying sooner, that little bit of time I have for Arch is mostly being eaten up by schwag lately.
On Thu, 4 Sep 2008, Dusty Phillips wrote:
2008/9/2 Aaron Griffin <aaronmgriffin@gmail.com>:
On Sun, Aug 31, 2008 at 11:11 AM, Dusty Phillips <buchuki@gmail.com> wrote:
Another alternative:
- Create a web based interface to allow community users to signoff on packages. We could have a policy of X number of user signoffs is equal to one developer signoff. Developers can be trusted to test it better (or not? ;-), but arch members can contribute too. In that case we still need some incentive to the end users to bother doing this. There aren't a lot of people like dolby and cactus who seem to actually like working really hard with as little recognition as possible.
I actually thought about this a while back. It was in terms of "each package needs 10 points, developer signoffs are 5 points, TU signoffs are 3 points, user signoffs are one point"
It might be a useful way to find active community members too. If you could think of a way to creatively implement something like this, I'd be all for it, assuming other people are.
The obvious implementation would be to give people the option to sign up for an account and to give them signoff permission (as Paul suggested). This is easy to implement but it means non-devs have to have access to dev.archlinux.org or we have to break some of the caching on the public site (which would make cactus weep). It also means yet another fricking user account that people have to sign up for.
A less invasive option would be to add a 'signoff this package' link to testing packages just like there is currently a 'flag out of date' link. But it would be hard to police if some asshat wanted to come through and click the signoff button 50 times even though the package was thoroughly busted.
There have been some bug reports suggesting linking packages to flyspray tickets (or vice versa), and I have a personal vendetta against flyspray so another option would be to create a new package-based bug tracker and port existing bugs and user accounts from flyspray. Then anyone with a 'Arch Linux KISS Bugtracker' account would also have the ability to signoff on packages. So people could actively log in and either say "yo this package works so good I want it to go live (ie: signoff)" or "wtf this package is broken, fix it fix it whine whine whine (ie: attach a bug to the package)".
The question isn't so much which of these to do as how much time do I have to commit to it (within epsilon of 0), how many other awesome people want to submit patches (approximately 2), and how soon do we need it (two months back?)
Dusty
PS: sorry for not replying sooner, that little bit of time I have for Arch is mostly being eaten up by schwag lately.
This sounds complicated. I think we could improve signoff by being more organized. One of the current problem is that we have packages that very few devs uses (wireless drivers, file syatem tools, raid, lvm, etc). As we don't know how many dev uses these, at every signoff thread we are probably waiting for signoffs that will never come. And the users probably assume that eventually a dev will signoff. So Aaron has to come and give his "untested" signoff just to get things moving. To improve this, we should make a list of these low-usage packages with the name of the devs that actually uses them (and for which architecture). This might even encourage signoff if a dev knows he's the only one who can signoff a package. If there are packages that not enough dev use to do the signoff, we could do one of two options: 1) Make an exceptions for these packages. We move them to core once the devs who use them have signed off. We could also force them to remain in testing for a minimal time (few days) to be more safe. This is equivalent to what we do now when Aaron gives his "untested" signoff except we don't wait for a few weeks for signoffs that will never come. 2) We involve the community at large. In the signoff thread, we explicitely state that we need community signoffs and for which architecture. Users signoff by replying directly to the dev who started the signoff thread or to the arch-general ML. If a few users signoff, then we move the package to core. The points system is not pratical. If no dev use a package, then we would need signoff from 7 TU or 20 users or any combination. This is a lot of signoff. If very few devs use a package, then it's probably the case for the community as well so it might take a long time to get all these signoffs. The signoff threads are on the arch-dev-public ML which is intented for Arch developpement. I assume that users subscribed to this list are more technical oriented and have Arch at heart. So if they signoff a package, then they probably know how to test it, have tested it and verified that it work as intended. Anyway, we would have their email address in case there are problems. Eric -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
participants (9)
-
Aaron Griffin
-
Allan McRae
-
Dan McGee
-
Dusty Phillips
-
Eduardo Romero
-
Eric Belanger
-
Paul Mattal
-
Thayer Williams
-
Thomas Bächler