On 11/1/21 16:39, Billy Morgan via arch-general wrote:

we the community WANT TO HELP. ...

[arch-general] NEW RECORD! 769 packages out of date

These two positions seem to be somewhat at odds. -Sam

On Tuesday, November 2nd, 2021 at 9:33 PM, mar77i via arch-general <arch-general@lists.archlinux.org> wrote:

Where can I post updated PKGBUILDs for review and addition to testing in order to help?

Nowhere. Arch does not allow users to submit updates via the bug tracker. If they did, and actually accepted them, I think the outdated package problem wouldn't be quite as bad as it is now. As someone mentioned elsewhere, throwing more TUs at the problem probably won't fix it: They can only commit to the community repo, and many things in core / extra are seemingly abandoned. A broader problem is that the project has lost several long-time devs in the last couple years and the burden of dropping off their hundreds of packages seems to be too much for the current team. Politics is also a factor and some people are afraid to "step on others' toes" by updating their packages for them. I've maintained a lot of local package updates for security fixes where the maintainer went missing or ignored emails about it. Unfortunately this seems to be a requirement for anyone wanting an up-to-date Arch system these days.

On 2021-11-02 20:19, Sam Mulvey via arch-general wrote:

...

I've maintained a lot of local package updates for security fixes where the maintainer went missing or ignored emails about it. Unfortunately this seems to be a requirement for anyone wanting an up-to-date Arch system these days.

This seems like something that could be informally organized in a way that would make it easier for maintainers to interface with.

We keep track of the open security issues in our security tracker [1]. If you want to help out getting some of these fixed, the most effective way would be going through this list and opening reports in our bug tracker for packages where a fix is available. In contrast to trying emailing the individual maintainers without any involvement of the security team [2], this allows us to easily see when package updates are required for security. Opening a bug report with the necessary information is very simple, just select the corresponding Arch Vulnerability Group (AVG) in the security tracker [1] and click on the "Create Ticket" button. This will open our bug tracker with a pre-filled template. The only additional information you need to provide is the "Guidance" section, i.e. a suggestion on how to fix the issues (upgrading the package to a more recent version, applying certain patches and where to find these, etc.). Please make sure to only open bug reports where a fix is actually available: we keep track of a lot of issues where no resolution is available yet, spamming the bug tracker with reports about these does not help as there is nothing we can do in this case. If you are aware of any open security issues that are not yet included in the security tracker, we would love to hear about them! The easiest way to get in touch is the #archlinux-security IRC channel on Libera Chat, but see [2] for more ways of contact. We are also always looking for more security team members to help keeping track and fixing newly disclosed vulnerabilities. If that sounds interesting to you, the IRC channel would also be the best place to start. Finally, I would like to contest the assertion that users would need "a lot of local package updates for security fixes" in order to keep a secure system: looking at the open security issues in [1], the vast majority of these are unresolved upstream, so no package update will solve them. There is indeed a non-zero number of packages that could be version-bumped or patched to fix some issues, but overall we seem to be able to keep up with security vulnerabilities relatively fine. Best, Jonas [1] https://security.archlinux.org/ [2] https://wiki.archlinux.org/title/Arch_Security_Team -- Jonas Witschel Arch Linux Developer, Trusted User and security team member PGP key: FE2E6249201CA54A4FB90D066E80CA1446879D04

On 2021-11-03 10:46, Sam Mulvey wrote:

On 11/3/21 03:42, Jonas Witschel wrote:

...

Opening a bug report with the necessary information is very simple,

With as much respect as I can textually apply, I would not describe the description that follows as "simple."

With "simple", I was referring to the fact that you do not have to find the security issue and think of an appropriate text for the bug tracker yourself, that part is all automated. The only thing the user needs to provide is a suggestion on how to actually fix the issue. Yes, this can be hard if there is no upstream release with a fix available, but that is exactly the issue with external contributions: It is one thing to know that a package is out of date. To this end, it is possible to flag packages for the maintainer's attention. This is useful since Arch does not have a central version update checker, so it is up to the individual maintainers to keep track of upstream updates. However, what people often do not realise (which is certainly not their fault, but probably stems from unfamiliarity with the packaging ecosystem) is that bumping the version number in a PKGBUILD is *not* the hard part. What takes time is actually building, testing and releasing the updated package. Unfortunately, Arch is not in a position to automate these tasks yet, so this has to be done by the individual maintainers in the (often limited) time they are able to afford to spend for Arch. This does not mean in any way that external contributions are not welcome! Updating and testing an outdated package yourself locally can be a great place to start. If you experience any issues, like having to apply patches, test suite failures that you need to overcome etc., opening a bug report for these is highly welcome! Even if you do not experience issues, mentioning the fact that you have built and tested the updated package yourself in an out-of-date flag is useful since it helps the maintainer evaluate the danger of breakage when upgrading the repository package.

If there isn't a problem, trying to organize the stated issues into actual solutions would make that clearer.

I really struggle to understand what you are trying to say here. Could you rephrase or elaborate, please?

...

Finally, I would like to contest the assertion that users would need "a lot of local package updates for security fixes" in order to keep a secure system: looking at the open security issues in [1], the vast majority of these are unresolved upstream, so no package update will solve them.

This is a very mild microcosm of my experiences with Arch Linux, and why a thread about "a plea for communication" speaks to me. I installed Arch for the first time when I did something unspeakable to a macbook and needed something until I fixed it. Not too long after that every device I could make run Arch was running Arch. Technically, it's simple and magnificent.

Yet, as soon as a person is involved simple goes out the window. Most of my interfaces with the Arch team have always been challenging, and every time I dip my toe in I end up having someone "contest" what I'm saying in varying degrees.   The only major package I maintain in AUR happened because I accidentally offended the TU who was maintaining the package.

I am genuinely sorry to hear that your experiences with staff members have been less than ideal in the past. For the record, the statement that I disagree with were not even your words, I was just replying to your email because you brought up the "how to organise security updates" aspect that I wanted to illuminate with my last email, so that felt the appropriate place to reply to.

There are a lot of unspoken rules to the Arch Linux community. More than I'm used to from a volunteer organization and I work 100% in the volunteer space.   Thus far I have been unable to navigate it.   Since Arch continues to make good technical decisions-- even when I disagreed with those decisions-- I decided to keep using it and just keep my trap shut.

I agree that we could do far better regarding documentation on how to get involved. I would not go as far as to call these "unspoken rules", Arch simply has far less hard and fast guidelines than other projects, for better or worse. I completely understand that this can intimidating and sometimes even prohibit contribution. Therefore the goal of my last email was explaining possible ways to take part in order to lower this barrier. If you have more questions in that regard, please do not hesitate to get in touch! The vast majority of team members I had the pleasure of interacting with have been welcoming and helpful to work with.

When someone else seemed like they were facing the same issues I was, I decided to speak up. [...]

Again, I want to make clear that I am not trying to invalidate your - or anybody else's - opinion that there is no really clear-cut way towards contributing to Arch. I am trying to fill this gap in documentation by providing ways to contribute, in this case to the security team.

Nonetheless, you do good work and I thank you for it.

Thank you, on behalf of the Arch team! :) Cheers, Jonas -- Jonas Witschel Arch Linux Developer, Trusted User and security team member PGP key: FE2E6249201CA54A4FB90D066E80CA1446879D04

On 11/3/21 12:27, Archange wrote:

Thanks Jonas, you wrote the mail I wanted to sent. :)

Yet you said it again. It took two people with an official email address to tell me how wrong I am.  This is such a perfect example of the problem that it kind of hurts.   There's a word I've been trying to avoid using, because it's pretty semantically weighted: gatekeeping. Let me cut out the bits here that communicate it, in order:

I’d like to emphasize that contributions are welcome You want my help.

as long as they are not trivial changes that don’t bring value. But only if it meets a vague standard set by individual maintainers.

this is also a way to get yourself known by dev/TUs But I also have to *know the right people*.  Yikes.

That said, not all gatekeeping is horrible.   Arch says right on the tin that it has technical knowledge requirements to play ball.   I get that.  I also run a volunteer organization with technical requirements to fully participate.  But what that meant to us is that we had to spend a lot of thought avoiding using that gate as a way to support other, less worthwhile gates[1]. It doesn't feel like Arch is spending time on that, so:

All that being said, we certainly do lack human resources

Sometimes I really feel like I should give back, but it looks like a damn long walk and I have other things I could do.   So I'm glad for (and grateful to) the people who find that walk a lot less onerous than I.  I sometimes worry how many other people like me are out there.   If it becomes too many, the distribution kind of stops being a distribution.   So here I am, managing three packages in AUR and blathering about gatekeeping and social domain problems of a Linux distribution.

*: Although quite an extreme example by the amount of changes versus the amount of the maintainer available free time (me), it took me roughly a year to have enough of it to look deeply into vtk9 changes, package the dependencies, solve multiple issues (including several PR in different upstream projects). While a vtk9 package was available in the AUR, it did not provide most of the features, and certainly did not take into account several of the issues we had while rebuilding dependent packages. I was asked several times by people why I did not bump yet, I explained the issue and how people could help, but then it seems people realized this was difficult because I did not get further answers.

I'm really not unsympathetic to this.  GNOME is wacky, audacity is broken, blender was stonking huge the last time I had to care about it, and packaging python stuff is space magic to me since I don't use it, but exactly: extreme example.  For every "my partner needs OBS to do horrible thing X" there were a an order of magnitude more packages that were done by incrementing a number in vim and then generating new checksums. Feeling a bit preachy[2] so I'm going to bow out here.   I'll say it again: you all do good work, and it is appreciated.   If you'd like to talk to me further, I'm pretty easy to find. -Sam [1]: I'd love to tell you what we're working on, but we're a project that is perforce local so we can do things that an international project like Arch would find difficult. [2]: Which is kind of ironic, really.

However, what people often do not realise (which is certainly not their fault, but probably stems from unfamiliarity with the packaging ecosystem) is that bumping the version number in a PKGBUILD is *not* the hard part. What takes time is actually building, testing and releasing the updated package.

Since my focus is on communicating a little more than the technical things themselves, I will say that this response feels a little automatic.   You write later in the post that you understand that I did update the PKGBUILD, that I did build it, and that I did (at least some) testing, so I'm not completely in the dark here, especially as I'm apparently doing my testing much the same way maintainers are. The automated feel of this response is augmented by the subsequent agreement from someone else with an "archlinux.org" address.

This does not mean in any way that external contributions are not welcome! Updating and testing an outdated package yourself locally can be a great place to start. If you experience any issues, like having to apply patches, test suite failures that you need to overcome etc., opening a bug report for these is highly welcome!

The details of my experience are somewhat at odds with this very nice invitation.

...

If there isn't a problem, trying to organize the stated issues into actual solutions would make that clearer. I really struggle to understand what you are trying to say here. Could you rephrase or elaborate, please?

Sorry, that was more for other folks in the thread.  If other people take a crack at fixing what they consider problems, they might discover interesting things.  When one stops writing posts and starts writing code, some of the "problems" might turn out to be mirages of one sort or another.  This is why when I encounter a problem I try to fix it myself first.  Maybe I'm the problem. As I was the problem here: I was too vague.

...

There are a lot of unspoken rules to the Arch Linux community. More than I'm used to from a volunteer organization and I work 100% in the volunteer space.   Thus far I have been unable to navigate it.   Since Arch continues to make good technical decisions-- even when I disagreed with those decisions-- I decided to keep using it and just keep my trap shut. I agree that we could do far better regarding documentation on how to get involved. I would not go as far as to call these "unspoken rules", Arch simply has far less hard and fast guidelines than other projects, for better or worse.

I wasn't planning on responding again as I felt like I was prevailing too much on your time, but this is pretty key. I'm going to ask you to consider this from the perspective of a person with literally no connection to any other Arch user, social or otherwise. The lack of written hard and fast rules does not preclude the existence of hard and fast rules.  Decisions have to be made. Decisions become guidelines, guidelines become rules.   If that rule isn't stated somewhere, or more likely, said rules are distributed via so many communication channels that a single person has difficulty even being aware of them let alone keeping up, those rules are going to feel "unspoken" or "unwritten" no matter how those who live with them may think. And those are the rules I appear to run into, time and time again. This is, unfortunately, where "simplicity" and "KISS" can absolutely fail.  Under this model simplicity for you means complexity for me, where a little bit of intention could potentially keep it simple for everyone.  I'm not going to pretend that's an easy thing to do, but that doesn't mean the problem isn't there.

I completely understand that this can intimidating and sometimes even prohibit contribution. Therefore the goal of my last email was explaining possible ways to take part in order to lower this barrier. If you have more questions in that regard, please do not hesitate to get in touch! The vast majority of team members I had the pleasure of interacting with have been welcoming and helpful to work with.

I'm going to read between the lines and suspect that I'm taking up too much of your time.  :D  Thank you so much for responding to me, it does help. -Sam

Hi Christopher,

Politics is also a factor and some people are afraid to "step on others' toes" by updating their packages for them.

How it worked in Dept. 1271 at Bell Labs, i.e. where Unix was born and raised, was the last one to touch a program ‘owned it’. So, add an option to grep(1) and folks then came to you for support. Perhaps that's too severe for Arch's maintainers, but an atmosphere of ‘We're all pulling together to keep Arch bleeding edge’ rather than strict ownership might help. Maybe that's already present and just not visible from here. I thought the comparison may help. -- Cheers, Ralph.