Re: [arch-general] [arch-dev-public] Switching the bugtracker to Bugzilla
Hello everyone, On 11/14/2017 10:34 PM, Bartłomiej Piotrowski wrote:
On 2017-11-14 20:30, Jelle van der Waa wrote:
Used by several big projects such as Gnome, LLVM and Mozilla
GNOME will probably end up switching to Gitlab. (Not dismissing the fact that bugzilla is rather popular choice.)
Bugzilla is old both in terms of look and feel and usability. It doesn't even support Markdown formatting[1], which could have been nice in order to be on part with the Markdown feature brought with the forthcoming aur web version. Contributing from a mobile device is rather complicated with bugzilla, even if I know some kind of non-maintained addon is available[2]. We can't see properly the pull requests being made to solve a specific problem and I know the Bugzilla's UI doesn't encourage them (just ask the Document Foundation about it). This is why, if I was in a position to provide a vote, I would have chosen a solution like gogs or Gitlab instead, which fixes the aforementioned problems and are well-maintained. On a side note, I'm currently in the process of rewriting the Gitlab install guide on the ArchLinux wiki, because I need it for corporate purpose and according to comments on the talk page, that article is hard to understand for newcomers. In the same process, gogs/Gitlabs would allow the opportunity to migrate the svn infra used for official packages to a real git-based infrastructure. svn-to-git, even if we have 'abs' [3] as syntactic sugar, adds unneeded cluttering. The current infra for official packages is blocking new contributions (I remember the cumbersome process it was for me to push 2 fixes to openjfx, patches via email is from an old age). Also, having an infra like gogs/Gitlab could allow us to have our Github ArchLinux account becoming the official read-only mirror for Arch Linux (yet another free backup). Synchronization scripts between both systems do already exist (I can ask at The Document Foundation if needed).
# Migration
There are several options for migrating the bug history to Bugzilla and a few options are under debate. (input welcome)
As I said multiple times on IRC, I'm for starting from scratch. There are way too many inactive or/and incorrect bugs open, and honestly any effort to review that list is a waste of time. With no bugs open we can 1) pretend everything works fine 2) hopefully avoid zombie-bugs apocalypse that we have now. Flyspray could be mirrored with wget for read-only version.
When I migrate something, I don't like to have anything legacy in an infrastructure. I usually migrate everything and do not leave a publicly accessible web app, especially if the latter isn't maintained any more, and has multiple security vulnerabilities. This doesn't prevent us to keep a complete offline backup copy if needed. Migrate everything doesn't prevent us to not close old bugs either. In this regard, it would be nice to have a policy to auto-close bugs that haven't received comments/reactions within x day. Any way, if a contributor has anything to add, he could reopen it if needed. Having a more strict bug triaging policy is usually good and reminds me a recent article I read on the Gitlab.com blog[4]. Just my two cents :) Ps.: I'm writing on arch-general, because I don't have write access to arch-dev-public, I'm not a TU (yet, but my application will come in the months to come, currently getting prepared to it), I'm just a Arch Linux user/contributor and have been since 2012 actually). [1] https://bugzilla.mozilla.org/show_bug.cgi?id=330707 [2] https://wiki.mozilla.org/Bugzilla:Addons#Handheld_Clients [3] https://wiki.archlinux.org/index.php/Arch_Build_System#Retrieve_PKGBUILD_sou... [4] https://about.gitlab.com/2017/10/26/triage-issues-gitmate/ -- William Gathoye <william@gathoye.be>
On 11/19/2017 09:08 AM, William Gathoye wrote:
Bugzilla is old both in terms of look and feel and usability. It doesn't even support Markdown formatting[1], which could have been nice in order to be on part with the Markdown feature brought with the forthcoming aur web version. Contributing from a mobile device is rather complicated with bugzilla, even if I know some kind of non-maintained addon is available[2]. We can't see properly the pull requests being made to solve a specific problem and I know the Bugzilla's UI doesn't encourage them (just ask the Document Foundation about it).
I cannot think why we are especially in need of markdown, I guess it is nice to have but I never regarded it as critical. If it is just to inline a patch I consider it a lot kinder to attach a patch that I can download rather than inline a patch that I have to convert from text on a webpage into a patch file.
This is why, if I was in a position to provide a vote, I would have chosen a solution like gogs or Gitlab instead, which fixes the aforementioned problems and are well-maintained. On a side note, I'm currently in the process of rewriting the Gitlab install guide on the ArchLinux wiki, because I need it for corporate purpose and according to comments on the talk page, that article is hard to understand for newcomers.
OTOH, gitlab is complicated to package and occasionally breaks, which would be embarrassing if we could not report bugs against the package because the package is broken and we don't have a bugtracker... And it's also a git repository server first and a rudimentary issue tracker second, so a "real" bugtracker would be preferable for the flexibility they offer.
In the same process, gogs/Gitlabs would allow the opportunity to migrate the svn infra used for official packages to a real git-based infrastructure. svn-to-git, even if we have 'abs' [3] as syntactic sugar, adds unneeded cluttering. The current infra for official packages is blocking new contributions (I remember the cumbersome process it was for me to push 2 fixes to openjfx, patches via email is from an old age).
We cannot migrate to git because no one has put in the work necessary to properly rewrite dbscripts. And if we had git for dbscripts, we would do quite well with our current cgit interface, which is... a git infrastructure. Don't forget that *the* standard git-based infrastructure is, well, git-send-email. And the reason people don't do that, nor do they attach patches for our svn repos, is mostly because reporting bugs is a lot easier than fixing them, and every target userbase is different. There are lots of other projects which get *lots* of contributions in the form of patches emailed or attached to bugzillas.
Also, having an infra like gogs/Gitlab could allow us to have our Github ArchLinux account becoming the official read-only mirror for Arch Linux (yet another free backup). Synchronization scripts between both systems do already exist (I can ask at The Document Foundation if needed).
I hereby volunteer to take the two minutes needed to write a synchronization script between cgit and and Github, should we desperately need one.
In this regard, it would be nice to have a policy to auto-close bugs that haven't received comments/reactions within x day. Any way, if a contributor has anything to add, he could reopen it if needed. Having a more strict bug triaging policy is usually good and reminds me a recent article I read on the Gitlab.com blog[4].
Hiding a bug that isn't getting fixed doesn't make it go away, and I have a visceral hatred of systems that do so. Filtering based on Feature Requests (which can potentially languish for years before someone is inspired to implement it, but are no less valid) and Bugs, which should never ever be closed unless they are either fixed or determined to be invalid, is pretty easy in basically any proper setup. In the very much less common event that a bug is actually waiting on a response from the reporter in order to determine whether it is valid, that is why we have a "Waiting for Response" category, which should only ever be manually set when it is determined that that is the correct response, and which I dubiously suppose could in theory be automated for closure but which isn't really a problem as we can handle doing so automatically. This whole mindset seems to come from a world which is dominated by developers of some project who don't have time to handle every bug report, are already resigned to only fixing the most visible bugs or the ones with the most insistent reporters who don't ever give up when a bugfix takes a long time due to the aforementioned issue of developers not having enough time to even look at bugs, let alone fix them, and want some way to sweep their trash under the rug so they don't have to see the warts. Just in case you were wondering how I feel about this. :) -- Eli Schwartz
On 19.11.2017 16:24, Eli Schwartz wrote:
We cannot migrate to git because no one has put in the work necessary to properly rewrite dbscripts.
Just to clarify: Gabriel Souza Franco (gbsf) has put quite a lot of work into this and some others have also worked on parts, but we haven't yet assembled all these parts or come up with a migration plan. It's a complex process and the people with sufficient access to deal with it are sadly quite busy. Additionally svn works well enough for our use case and other, more pressing, issues get priority. Git is certainly not forgotten and work will continue, but it may take a while until it's done. Florian
On 11/19/2017 11:04 AM, Florian Pritz via arch-general wrote:
On 19.11.2017 16:24, Eli Schwartz wrote:
We cannot migrate to git because no one has put in the work necessary to properly rewrite dbscripts.
Just to clarify: Gabriel Souza Franco (gbsf) has put quite a lot of work into this and some others have also worked on parts, but we haven't yet assembled all these parts or come up with a migration plan. It's a complex process and the people with sufficient access to deal with it are sadly quite busy. Additionally svn works well enough for our use case and other, more pressing, issues get priority. Git is certainly not forgotten and work will continue, but it may take a while until it's done.
Yes, this is a good point. :o I should have said "no one has put in all the work to finish". Apologies. -- Eli Schwartz
On 11/19/2017 04:24 PM, Eli Schwartz wrote:
OTOH, gitlab is complicated to package and occasionally breaks, which would be embarrassing if we could not report bugs against the package because the package is broken and we don't have a bugtracker...
I'll be using Gitlab professionally on Arch Linux. So as soon I become a TU (if it happen the Arch Linux community accepts me ;)) I think I'll help Sven-Hendrik Haase in this process. Packaging Gitlab as a single person is indeed a hard task.
This whole mindset seems to come from a world which is dominated by developers of some project who don't have time to handle every bug report, are already resigned to only fixing the most visible bugs or the ones with the most insistent reporters who don't ever give up when a bugfix takes a long time due to the aforementioned issue of developers not having enough time to even look at bugs, let alone fix them, and want some way to sweep their trash under the rug so they don't have to see the warts.
Just in case you were wondering how I feel about this.
Thanks Eli for your POV. This is why at The Document Foundation and Mattermost, we have a QA team (or at least one or two persons) which checks if the bugs are valid and put in cc the people that have the abilities to work on it. I know all of us have a life aside FOSS projects, but sometimes, even if the dev put in cc cannot actually work on the fix for the bug itself, he might answer a question in 2 minutes and help another one that will actually do the fix for him: bringing synergy in the process. From my understanding and the time I spent following this community (I'm subscribed to all Arch Linux public mailing lists and have been reading all the message every weekend since 2012), a true bug triager person is something I haven't seen in Arch yet (unless I'm wrong). On 11/19/2017 05:16 PM, Eli Schwartz wrote:
On 11/19/2017 11:04 AM, Florian Pritz via arch-general wrote:
Just to clarify: Gabriel Souza Franco (gbsf) has put quite a lot of work into this and some others have also worked on parts, but we haven't yet assembled all these parts or come up with a migration plan. It's a complex process and the people with sufficient access to deal with it are sadly quite busy. Additionally svn works well enough for our use case and other, more pressing, issues get priority. Git is certainly not forgotten and work will continue, but it may take a while until it's done.
Yes, this is a good point. :o
I should have said "no one has put in all the work to finish". Apologies.
Great. This is maybe a topic I'll be able to contribute to then as having to play with svn is something that annoys me during my daily life. Yet another point I'll be able to put in my application letter (email) when comes the time for me to apply to become a TU. :) Have a good end of weekend. Regards, -- William Gathoye <william@gathoye.be>
On Sun, 19 Nov 2017 17:58:55 +0100 William Gathoye <william@gathoye.be> wrote:
This is why at The Document Foundation and Mattermost, we have a QA team (or at least one or two persons) which checks if the bugs are valid and put in cc the people that have the abilities to work on it. I know all of us have a life aside FOSS projects, but sometimes, even if the dev put in cc cannot actually work on the fix for the bug itself, he might answer a question in 2 minutes and help another one that will actually do the fix for him: bringing synergy in the process.
From my understanding and the time I spent following this community (I'm subscribed to all Arch Linux public mailing lists and have been reading all the message every weekend since 2012), a true bug triager person is something I haven't seen in Arch yet (unless I'm wrong).
That is exactly what Eli and I do.
On 2017-11-19 17:58, William Gathoye wrote:
I'll be using Gitlab professionally on Arch Linux. So as soon I become a TU (if it happen the Arch Linux community accepts me ;)) I think I'll help Sven-Hendrik Haase in this process. Packaging Gitlab as a single person is indeed a hard task.
There is more to reliability of service than correct and reliable packaging. By any means Gitlab isn't "fire & forget" type of project and with my infra team hat on, I'm completely unwilling to spend my evenings or lunches on making sure it's running properly.
This is why at The Document Foundation and Mattermost, we have a QA team (or at least one or two persons) which checks if the bugs are valid and put in cc the people that have the abilities to work on it. I know all of us have a life aside FOSS projects, but sometimes, even if the dev put in cc cannot actually work on the fix for the bug itself, he might answer a question in 2 minutes and help another one that will actually do the fix for him: bringing synergy in the process.
From my understanding and the time I spent following this community (I'm subscribed to all Arch Linux public mailing lists and have been reading all the message every weekend since 2012), a true bug triager person is something I haven't seen in Arch yet (unless I'm wrong).
With all due respect, we are not a foundation or project that also sells membership or deployments/hosted service. We already have people responsible for bug triaging, some of packagers also chime in occasionally. It's being accomplished fully in free time, so I don't see the point of comparison. This includes finding people who can actually fix given bug if someone who should do that is missing. Forgive us for not doing as good job as others, but in my opinion, we're doing quite fine. Bartłomiej
On Sun, Nov 19, 2017 at 7:54 PM, Bartłomiej Piotrowski <bpiotrowski@archlinux.org> wrote:
On 2017-11-19 17:58, William Gathoye wrote:
I'll be using Gitlab professionally on Arch Linux. So as soon I become a TU (if it happen the Arch Linux community accepts me ;)) I think I'll help Sven-Hendrik Haase in this process. Packaging Gitlab as a single person is indeed a hard task.
There is more to reliability of service than correct and reliable packaging. By any means Gitlab isn't "fire & forget" type of project and with my infra team hat on, I'm completely unwilling to spend my evenings or lunches on making sure it's running properly.
This is why at The Document Foundation and Mattermost, we have a QA team (or at least one or two persons) which checks if the bugs are valid and put in cc the people that have the abilities to work on it. I know all of us have a life aside FOSS projects, but sometimes, even if the dev put in cc cannot actually work on the fix for the bug itself, he might answer a question in 2 minutes and help another one that will actually do the fix for him: bringing synergy in the process.
From my understanding and the time I spent following this community (I'm subscribed to all Arch Linux public mailing lists and have been reading all the message every weekend since 2012), a true bug triager person is something I haven't seen in Arch yet (unless I'm wrong).
With all due respect, we are not a foundation or project that also sells membership or deployments/hosted service. We already have people responsible for bug triaging, some of packagers also chime in occasionally. It's being accomplished fully in free time, so I don't see the point of comparison. This includes finding people who can actually fix given bug if someone who should do that is missing. Forgive us for not doing as good job as others, but in my opinion, we're doing quite fine.
Bartłomiej
I've refrained from commenting on this topic because I don't want to sound ungrateful people are taking the time to work on a fairly extensive migration off flyspray, but I'm not looking forward to Bugzilla (and I've contributed to a lot of Bugzilla-based projects in the past). It has the same mindset as Jira, making filing an issue a similar endeavour as filing taxes and creating artificial meta-work for both users and triagers. I strongly agree something like Gogs or Gitlab would be a much better path forward. Especially if, as Jelle was initially saying, the goal is for it to be "extended to our wishes". Furthermore, Gitlab has native support for federated login which we seriously could start using. Separate logins for bug tracker, BBS AUR, wiki, archweb and all the mailing lists is... eh. J. Leclanche
On 2017-11-19 20:11, Jerome Leclanche wrote:
I've refrained from commenting on this topic because I don't want to sound ungrateful people are taking the time to work on a fairly extensive migration off flyspray, but I'm not looking forward to Bugzilla (and I've contributed to a lot of Bugzilla-based projects in the past). It has the same mindset as Jira, making filing an issue a similar endeavour as filing taxes and creating artificial meta-work for both users and triagers. I strongly agree something like Gogs or Gitlab would be a much better path forward. Especially if, as Jelle was initially saying, the goal is for it to be "extended to our wishes". Furthermore, Gitlab has native support for federated login which we seriously could start using. Separate logins for bug tracker, BBS AUR, wiki, archweb and all the mailing lists is... eh.
J. Leclanche
This is example of wishful thinking and misunderstanding what our requirements are. Reporting anything on Bugzilla isn't different from using Flyspray, and we're far from "the most friendly distribution of the year" title anyway. Neither Gogs or Gitlab are primarily issue trackers and I hope you noticed that we're not discussing integrated code hosting solution. Calling Gogs extendable is overstatement. As far as I know, Gitlab supports external authentication providers only in the enterprise edition. Even if something has changed about it recently, somehow I doubt you're going to join #archlinux-devops tomorrow and say that you're eager to both maintain Gitlab and LDAP deployments, and then figure out LDAP integration everywhere. Let's just be realistic about what we need and what we can accomplish. Bartłomiej
2017-11-19 18:54 GMT+01:00 Bartłomiej Piotrowski <bpiotrowski@archlinux.org>:
On 2017-11-19 17:58, William Gathoye wrote:
I'll be using Gitlab professionally on Arch Linux. So as soon I become a TU (if it happen the Arch Linux community accepts me ;)) I think I'll help Sven-Hendrik Haase in this process. Packaging Gitlab as a single person is indeed a hard task.
There is more to reliability of service than correct and reliable packaging. By any means Gitlab isn't "fire & forget" type of project and with my infra team hat on, I'm completely unwilling to spend my evenings or lunches on making sure it's running properly.
Why don't talk with GitLab people to get a free hosted solution? As you can see in his page[1] they offer "the very best full gold plan of the death" to Open Source projects. In this way you can devote your resources to other tasks and forget the Git/GitLab administration. Greetings. [1]:https://about.gitlab.com/gitlab-com/ -- Óscar García Amor | ogarcia at moire.org | http://ogarcia.me
On 11/21/17 at 09:23am, Óscar García Amor wrote:
2017-11-19 18:54 GMT+01:00 Bartłomiej Piotrowski <bpiotrowski@archlinux.org>:
On 2017-11-19 17:58, William Gathoye wrote:
I'll be using Gitlab professionally on Arch Linux. So as soon I become a TU (if it happen the Arch Linux community accepts me ;)) I think I'll help Sven-Hendrik Haase in this process. Packaging Gitlab as a single person is indeed a hard task.
There is more to reliability of service than correct and reliable packaging. By any means Gitlab isn't "fire & forget" type of project and with my infra team hat on, I'm completely unwilling to spend my evenings or lunches on making sure it's running properly.
Why don't talk with GitLab people to get a free hosted solution? As you can see in his page[1] they offer "the very best full gold plan of the death" to Open Source projects. In this way you can devote your resources to other tasks and forget the Git/GitLab administration.
I'm quiet happy that we are still running everything on our (community sponsored) infrastructure without relying on third party's. This has a lot of benefits, we own the data, we can migrate freely to an alternative and we don't rely on externals messing things up or changing their offering. A sponsored hosted platform sounds amazing, but having a vendor lock in isn't really :-) -- Jelle van der Waa
From: Jelle van der Waa <jelle@vdwaa.nl> Sent: Tue Nov 21 10:36:08 CET 2017 To: General Discussion about Arch Linux <arch-general@archlinux.org> Subject: Re: [arch-general] [arch-dev-public] Switching the bugtracker to Bugzilla
I'm quiet happy that we are still running everything on our (community sponsored) infrastructure without relying on third party's. This has a lot of benefits, we own the data, we can migrate freely to an alternative and we don't rely on externals messing things up or changing their offering.
A sponsored hosted platform sounds amazing, but having a vendor lock in isn't really :-)
-- Jelle van der Waa ----------------------------------------
That also means you have to manage everything on your own. As Arch is quite constrained on human resources this is rather huge dis-benefit. More and more projects like gnome or debian are migrating to something like gitlab/github so the advantages are real. Thinking strategically, moving to bugzilla will be one step forward but it still leaves Arch on relative backwardness. Why don't take two steps at once instead? Migrating one service will make migrating others easier. I know that Arch devs main priority as pure volunteer project is to minimize effort however it involves danger of being left in obscurity and quite unmanageable state. In the end moving faster may require less effort. The whole web is going to vendor lock-in model and it isn't really obvious that staying out is better choice. G. K.
I'm back on this topic, because I have news. On 11/21/2017 09:18 AM, Bartłomiej Piotrowski wrote:
This is example of wishful thinking and misunderstanding what our requirements are. Reporting anything on Bugzilla isn't different from using Flyspray, and we're far from "the most friendly distribution of the year" title anyway.
Neither Gogs or Gitlab are primarily issue trackers and I hope you noticed that we're not discussing integrated code hosting solution.
To summarize, it seems we are going to the Bugzilla solution then. Gitlab/Gogs is great, but is maybe too much hassle for just a bug tracker, even if it gives further user abilities for the future and the free hosted version removes the freedom principles upon which the Arch Linux community has been founded. But before choosing which Bugzilla release to install, please consider using the Mozilla's fork which is greatly supported and brings additional welcomed features (like Markdown I was talking about, which brings parity in this regard with aurdev 4.6 released today). That version is intended to be merged back to Bugzilla master. For further comments, maybe Dylan W. Hardison could come in handy. Please read the interaction I have had with him here on Twitter: [1][2][3]
Calling Gogs extendable is overstatement. As far as I know, Gitlab supports external authentication providers only in the enterprise edition.
LDAP is supported but only basic features no subgroups which is indeed a feature reserved to the EE version.
Even if something has changed about it recently, somehow I doubt you're going to join #archlinux-devops tomorrow and say that you're eager to both maintain Gitlab and LDAP deployments, and then figure out LDAP integration everywhere.
Like I said, I'm currently bringing help in Gitlab support for Arch Linux by already rewriting the wiki page. If you need to federate help in this regard or improve Gitlab support for Arch, please let me know. I wasn't aware we had such a channel, I have just added it to my IRC bouncer, thanks!
Let's just be realistic about what we need and what we can accomplish.
Let's make the account merge and the LDAP feature for another future iteration then. [1] https://twitter.com/dylan_hardison/status/932643136958095360 [2] https://twitter.com/dylan_hardison/status/932645520333574145 [3] https://twitter.com/dylan_hardison/status/932649292954832896 William Gathoye <william@gathoye.be>
participants (9)
-
Bartłomiej Piotrowski
-
Doug Newgard
-
Eli Schwartz
-
Florian Pritz
-
Geo Kozey
-
Jelle van der Waa
-
Jerome Leclanche
-
William Gathoye
-
Óscar García Amor