[arch-dev-public] Switching the bugtracker to Bugzilla
It's time to switch our to something which is maintained and can be extended to our wishes. Flyspray isn't actively maintained, has had several security issues [1] [2]. It seems to be time to move on, and Bugzilla is one of the more active and maintained bugtrackers out there. Used by several big projects such as Gnome, LLVM and Mozilla. One of the benefits of moving would be the possibility of default assignees per 'component'. # Migration There are several options for migrating the bug history to Bugzilla and a few options are under debate. (input welcome) * No migration at all * Migrate open bugs * Migrate open bugs and auto-closing them * Migrate all bugs * Migrate all bugs and auto-closing them In either case, I believe it would be nice to "archive" the current bugtracker and make it read only. # User migration User migration should be possible as well, except migrating the password, a mass password reset would be wise. Since I'm not sure what kind of old hashing method / salt flyspray uses. # Migration Projects Bugzilla has a concept of products with components, so for all our packages we can create a component counterpart. It should be possible to auto-assign bugs with the pkgname <-> maintainer information from archweb. Possible products would be. # Products * Arch packages (core/extra or split this up) * Community packages (community) * Pacman * AURWeb * Keyring * Archweb (new) * Arch VM / Docker images (new) * Release engineering Input would be welcome, on what we should migrate from flyspray and what products we should define. [1] https://github.com/Flyspray/flyspray/releases/tag/v1.0-rc6 [2] https://github.com/Flyspray/flyspray/releases/tag/v1.0-rc4 -- Jelle van der Waa
Quoting Jelle van der Waa (2017-11-14 20:30:21)
There are several options for migrating the bug history to Bugzilla and a few options are under debate. (input welcome)
* No migration at all * Migrate open bugs * Migrate open bugs and auto-closing them * Migrate all bugs * Migrate all bugs and auto-closing them
In either case, I believe it would be nice to "archive" the current bugtracker and make it read only.
My first reaction is that it'd be nice to not have a bunch of old cruft around, but if we autoclose them and we could get the migrated bugs to have the same ID it would simplify having the old links still work with just a simple redirect to the new URL with the same ID, and it would mean that we wouldn't need to keep the current bugtracker around for the indefinite future.
# Products
* Arch packages (core/extra or split this up) * Community packages (community) * Pacman * AURWeb * Keyring * Archweb (new) * Arch VM / Docker images (new) * Release engineering
I feel like having core and extra as separate products would make searching and filling in the component field a *lot* easier, because that's pretty much my biggest annoyance with e.g. the GNOME bugzilla, their component lists are huge and it's really hard to browse through them. -- Sincerely, Johannes Löthberg PGP Key ID: 0x50FB9B273A9D0BB5 PGP Key FP: 5134 EF9E AF65 F95B 6BB1 608E 50FB 9B27 3A9D 0BB5 https://theos.kyriasis.com/~kyrias/
[2017-11-14 22:11:02 +0100] Johannes Löthberg via arch-dev-public:
My first reaction is that it'd be nice to not have a bunch of old cruft around,
For what it's worth: I completely agree. My choice would be to start over with a clean bug tracker and not migrate anything. Everyone who cares will create their account on the new tracker and every bug that matters will be recreated. And in the process we'll get rid of hundreds of very old bug no one cares about.
but if we autoclose them and we could get the migrated bugs to have the same ID it would simplify having the old links still work with just a simple redirect to the new URL with the same ID,
If that's straightforward to implement, sure, that'd be nice. But if not I don't think it's worth patching bugzilla for.
and it would mean that we wouldn't need to keep the current bugtracker around for the indefinite future.
We can convert the current tracker into a bunch of static pages. Personally I'd find that a very clean archival solution... Cheers. -- Gaetan
On 11/14/2017 02:30 PM, Jelle van der Waa wrote:
There are several options for migrating the bug history to Bugzilla and a few options are under debate. (input welcome)
* No migration at all * Migrate open bugs * Migrate open bugs and auto-closing them * Migrate all bugs * Migrate all bugs and auto-closing them
In either case, I believe it would be nice to "archive" the current bugtracker and make it read only.
We've never autoclosed bugs just because they are old, why start now *just because* we are migrating? Then too there are not-yet-implemented feature requests for pacman/aurweb/$project. I'd prefer we migrated all open bugs and left them open, possibly excluding old bugs filed under the kernel category as they have an alarming tendency to be upstream issues and/or hardware issues and anyway are questionably relevant considering the numerous updates. Possibly also extend that to also excluding open bugs that are assigned as upstream issues, as those usually have no real purpose other than to maybe track upstream and backport a fix as soon as one is available. If we have too many unresolved open bugs (which we do), they need to be triaged and dealt with (something I'd like to get around to doing and which I have already picked some low-hanging fruit).
# Migration Projects
Bugzilla has a concept of products with components, so for all our packages we can create a component counterpart. It should be possible to auto-assign bugs with the pkgname <-> maintainer information from archweb.
Possible products would be.
# Products
* Arch packages (core/extra or split this up) * Community packages (community) * Pacman * AURWeb * Keyring * Archweb (new) * Arch VM / Docker images (new) * Release engineering
Input would be welcome, on what we should migrate from flyspray and what products we should define.
netctl/mkinitcpio/devtools/dbscripts/namcap/pyalpm These are currently grouped together alongside core/extra as a component in the Arch Linux project, but if they don't each deserve their own project they can probably be components in an "Arch Projects" project. -- Eli Schwartz
On 2017-11-14 20:30, Jelle van der Waa wrote:
Used by several big projects such as Gnome, LLVM and Mozilla
GNOME will probably end up switching to Gitlab. (Not dismissing the fact that bugzilla is rather popular choice.)
# Migration
There are several options for migrating the bug history to Bugzilla and a few options are under debate. (input welcome)
As I said multiple times on IRC, I'm for starting from scratch. There are way too many inactive or/and incorrect bugs open, and honestly any effort to review that list is a waste of time. With no bugs open we can 1) pretend everything works fine 2) hopefully avoid zombie-bugs apocalypse that we have now. Flyspray could be mirrored with wget for read-only version.
Bugzilla has a concept of products with components, so for all our packages we can create a component counterpart. It should be possible to auto-assign bugs with the pkgname <-> maintainer information from archweb.
Packages come and go. Some never will have a bug reported because they just work fine or nobody uses them. Component per package sounds overboard unless it's going to be automated.
* Arch packages (core/extra or split this up) +1 for splitting.
* Archweb (new) * Arch VM / Docker images (new)
These two are already primarily developed on GitHub and I think bugs should be reported there as well. Bartłomiej
On 15/11/17 07:34, Bartłomiej Piotrowski wrote: >> There are several options for migrating the bug history to Bugzilla and a few options are under >> debate. (input welcome) > As I said multiple times on IRC, I'm for starting from scratch. There > are way too many inactive or/and incorrect bugs open, and honestly any > effort to review that list is a waste of time. With no bugs open we can > 1) pretend everything works fine 2) hopefully avoid zombie-bugs > apocalypse that we have now. Flyspray could be mirrored with wget for > read-only version. If you don't migrate pacman bugs, don't bother creating a pacman component. A
On 2017-11-14 22:59, Allan McRae wrote: > On 15/11/17 07:34, Bartłomiej Piotrowski wrote: >>> There are several options for migrating the bug history to Bugzilla and a few options are under >>> debate. (input welcome) >> As I said multiple times on IRC, I'm for starting from scratch. There >> are way too many inactive or/and incorrect bugs open, and honestly any >> effort to review that list is a waste of time. With no bugs open we can >> 1) pretend everything works fine 2) hopefully avoid zombie-bugs >> apocalypse that we have now. Flyspray could be mirrored with wget for >> read-only version. > > If you don't migrate pacman bugs, don't bother creating a pacman component. > > A > I mean packaging bugs here, not standalone projects. Bartłomiej
Em novembro 14, 2017 17:30 Jelle van der Waa escreveu:
It's time to switch our to something which is maintained and can be extended to our wishes. Flyspray isn't actively maintained, has had several security issues [1] [2].
Being the person responsible for writing our flyspray ansible's role and having spent an awful amount of time writing nginx redirects to overcome the fact that flyspray can't handle decent URL's, I might be a little biased against it.
It seems to be time to move on, and Bugzilla is one of the more active and maintained bugtrackers out there. Used by several big projects such as Gnome, LLVM and Mozilla. One of the benefits of moving would be the possibility of default assignees per 'component'.
I agree with the move to bugzilla. I advocated in the past for us to use something like gitlab, but we are using github and I think we should embrace it and make it "official". But I think neither gitlab nor github's issue trackers are that great. At least not for some of the things we do. We can use them just fine for our projects, though.
# Migration
There are several options for migrating the bug history to Bugzilla and a few options are under debate. (input welcome)
* No migration at all * Migrate open bugs * Migrate open bugs and auto-closing them * Migrate all bugs * Migrate all bugs and auto-closing them
In either case, I believe it would be nice to "archive" the current bugtracker and make it read only.
I'm for starting clean and making the current flyspray read-only.
# User migration
User migration should be possible as well, except migrating the password, a mass password reset would be wise. Since I'm not sure what kind of old hashing method / salt flyspray uses.
Agreed. Regards, Giancarlo Razzolini
On Tue, 14 Nov 2017 at 20:30:21, Jelle van der Waa wrote:
It's time to switch our to something which is maintained and can be extended to our wishes. Flyspray isn't actively maintained, has had several security issues [1] [2]. [...]
Great!
# Migration
There are several options for migrating the bug history to Bugzilla and a few options are under debate. (input welcome)
* No migration at all * Migrate open bugs * Migrate open bugs and auto-closing them * Migrate all bugs * Migrate all bugs and auto-closing them
In either case, I believe it would be nice to "archive" the current bugtracker and make it read only.
Doing no migration and having an archive of the current tickets sounds like a good idea to me -- I can only talk about the aurweb project and the packages projects, though -- maintainers of other components might disagree. After the migration, I would go through the open aurweb tickets on the Flyspray archive, create new tickets for the most important issues (should not be more than 10), fix/implement some of the easy stuff and close the tickets which I think are not worth fixing or implementing.
# User migration
User migration should be possible as well, except migrating the password, a mass password reset would be wise. Since I'm not sure what kind of old hashing method / salt flyspray uses.
Sounds good to me, although I do not see the benefit of doing this if we do not migrate tickets. If users need to reset their passwords, they can as well reregister. There aren't too many things to configure and the profile settings in Flyspray will likely not match what you can configure in Bugzilla anyways. Generally speaking, I am in favor of setting up a fresh Bugzilla instance and not doing any sort of migration. The idea of switching to a new bug tracker came up at least four years ago and my feeling is that all this migration work is what was holding it back for a long time. However, we should make sure that the old Flyspray URLs still work (and redirect to the Flyspray archive).
# Migration Projects
Bugzilla has a concept of products with components, so for all our packages we can create a component counterpart. It should be possible to auto-assign bugs with the pkgname <-> maintainer information from archweb.
Possible products would be.
# Products
* Arch packages (core/extra or split this up) * Community packages (community) * Pacman * AURWeb * Keyring * Archweb (new) * Arch VM / Docker images (new) * Release engineering
Sounds good to me. I would split [core] and [extra]. I would also call the packages projects "[core] packages", "[extra] packages" and "[community] packages"; unless there are technical limitations preventing the use of brackets. These brackets make it much clearer that the names are referring to pacman repositories; and to a novice Arch user, it may not be 100% clear what a "community package" or an "extra package" is. Best regards, Lukas
On 11/15/17 at 09:07am, Lukas Fleischer wrote:
On Tue, 14 Nov 2017 at 20:30:21, Jelle van der Waa wrote:
It's time to switch our to something which is maintained and can be extended to our wishes. Flyspray isn't actively maintained, has had several security issues [1] [2]. [...]
Great!
Thanks! Most of the credit goes to Bluewind for coming up with the idea and poking me about the status ;-) Note that Bugzilla, also has an REST API, which unlocks some more possibilities.
# Migration
There are several options for migrating the bug history to Bugzilla and a few options are under debate. (input welcome)
* No migration at all * Migrate open bugs * Migrate open bugs and auto-closing them * Migrate all bugs * Migrate all bugs and auto-closing them
In either case, I believe it would be nice to "archive" the current bugtracker and make it read only.
Doing no migration and having an archive of the current tickets sounds like a good idea to me -- I can only talk about the aurweb project and the packages projects, though -- maintainers of other components might disagree.
Ok, but it seems Allan would love to migrate all of his bugs, so we could see if that would be possible for aurweb as well. The biggest migration would be for packages.
After the migration, I would go through the open aurweb tickets on the Flyspray archive, create new tickets for the most important issues (should not be more than 10), fix/implement some of the easy stuff and close the tickets which I think are not worth fixing or implementing.
# User migration
User migration should be possible as well, except migrating the password, a mass password reset would be wise. Since I'm not sure what kind of old hashing method / salt flyspray uses.
Sounds good to me, although I do not see the benefit of doing this if we do not migrate tickets. If users need to reset their passwords, they can as well reregister. There aren't too many things to configure and the profile settings in Flyspray will likely not match what you can configure in Bugzilla anyways.
We currently have 12821 users in flyspray, of which there are ~ 50 with invalid email addresses and I'm not sure how many are actually 'active', since flyspray doesn't log the last logged in date.
Generally speaking, I am in favor of setting up a fresh Bugzilla instance and not doing any sort of migration. The idea of switching to a new bug tracker came up at least four years ago and my feeling is that all this migration work is what was holding it back for a long time. However, we should make sure that the old Flyspray URLs still work (and redirect to the Flyspray archive).
Setting up Bugzilla in Ansible, and figuring how to auto-create and sync products will still be some effort. And we have to make sure that Bugzilla, isn't as slow as it's showcased ;-) My todolist is here: https://wiki.archlinux.org/index.php/User:jelly/Bugzilla#TODO
* Release engineering
Sounds good to me. I would split [core] and [extra]. I would also call the packages projects "[core] packages", "[extra] packages" and "[community] packages"; unless there are technical limitations preventing the use of brackets. These brackets make it much clearer that the names are referring to pacman repositories; and to a novice Arch user, it may not be 100% clear what a "community package" or an "extra package" is.
I'm in favor, this should make it more clear where to report bugs, also with the automatic product selection. -- Jelle van der Waa
On 15.11.2017 09:07, Lukas Fleischer wrote:
Generally speaking, I am in favor of setting up a fresh Bugzilla instance and not doing any sort of migration. The idea of switching to a new bug tracker came up at least four years ago and my feeling is that all this migration work is what was holding it back for a long time.
That's not really true. The most important problem back then was that bugzilla only supported cgi and mod_perl. cgi performed really badly when I tested it and mod_perl didn't perform much better because it just hid the cost of compiling everything by using preforked workers. At least until the pool of preforked workers was used up and it needed to start new ones. In the end the server was able to compile bugzilla roughly 8 times per second using all cores and with a sufficient number of requests that also seemed to affect mod_perl. Nowadays bugzilla finally has psgi support where the perl code is compiled once and requests are sent to it. At least without data this leaves us with a few hundred requests per second so the base performance is much much better. Sadly it looks like the migration script, that once worked pretty well, no longer works out of the box. Jelle has been working on reviving it. Florian
Em novembro 15, 2017 6:07 Lukas Fleischer escreveu:
However, we should make sure that the old Flyspray URLs still work (and redirect to the Flyspray archive).
Even though flyspray URL's are not really flyspray's and are more nginx's [0], if there is no clash with bugzilla, I don't see any reason why we would not be able to redirect them to the flyspray archive. But what I have in mind for this is to have flyspray hosted under a different domain name. Something like old-bugs-meant-to-die.archlinux.org. Regards, Giancarlo Razzolini [0] https://git.archlinux.org/infrastructure.git/tree/roles/flyspray/templates/n...
On Tue, Nov 14, 2017 at 08:30:21PM +0100, Jelle van der Waa wrote:
[snip] # Products
* Arch packages (core/extra or split this up) * Community packages (community) * Pacman * AURWeb * Keyring * Archweb (new) * Arch VM / Docker images (new) * Release engineering
I think core and extra should be split up. There is also some work being done to help the communication between the testers, currently discussed at #archlinux-testing. I propose another component could be `testing` that encompasses bugs found from testing, community-testing and multilib-testing. It would make be a lot simpler for testers and others using the testing repos to find bugs and issues holding back package releases. -- Morten Linderud PGP: 9C02FF419FECBE16
On 11/16/17 at 02:01pm, Morten Linderud wrote:
On Tue, Nov 14, 2017 at 08:30:21PM +0100, Jelle van der Waa wrote:
[snip] # Products
* Arch packages (core/extra or split this up) * Community packages (community) * Pacman * AURWeb * Keyring * Archweb (new) * Arch VM / Docker images (new) * Release engineering
I think core and extra should be split up. There is also some work being done to help the communication between the testers, currently discussed at #archlinux-testing. I propose another component could be `testing` that encompasses bugs found from testing, community-testing and multilib-testing.
It would make be a lot simpler for testers and others using the testing repos to find bugs and issues holding back package releases.
Sounds good, I intent to start with a small PoC. But migration is a little bit more troublesome now, since I used an old db for conversion while the flyspray db changed it's schema. -- Jelle van der Waa
Sorry for being late to the party. On 2017-11-14 20:30:21 (+0100), Jelle van der Waa wrote:
It seems to be time to move on, and Bugzilla is one of the more active and maintained bugtrackers out there. Used by several big projects such as Gnome, LLVM and Mozilla. One of the benefits of moving would be the possibility of default assignees per 'component'. I guess the decision has already been made towards bugzilla, but if not, MantisBT [1] is another "only bug tracker" web application. It has quite improved over the recent versions concerning usability (nicely scalable on small screens) and is actively maintained. Default assignies can be made "by category" afaik. I can't say, if it fits all or any of the other requirements you're seeking though.
btw: I'm suggesting this not because I package mantisbt in AUR, but because I don't like using bugzilla too much ;-)
# Migration I would probably also opt for archiving and starting from scratch, if it indeed takes too long to sieve through and find relevant open bugs.
# User migration This would be great, but will be an ugly script job in any case ;) Additionally: If the relation to the user's bugs can be transferred, don't bother.
# Products Separating core/extra and a separate testing would be great. Although some "products" have found new bug tracking homes, I'd much rather see them unified under bugs.archlinux.org, than scattered.
Input would be welcome, on what we should migrate from flyspray and what products we should define. Sorry for probably adding more noise.
In any case, thanks for the work! Best, David [1] https://www.mantisbt.org/ -- https://sleepmap.de
participants (11)
-
Allan McRae
-
Bartłomiej Piotrowski
-
David Runge
-
Eli Schwartz
-
Florian Pritz
-
Gaetan Bisson
-
Giancarlo Razzolini
-
Jelle van der Waa
-
Johannes Löthberg
-
Lukas Fleischer
-
Morten Linderud