[arch-dev-public] A contrib repository
There is a *lot* of small tools people have written over the years that resides in bin/ directories which could be useful for more people. We also have several such tools on soyuz, where sogrep was added to devtools this week.
I have been thinking it could be great to have a simple `contrib` repository where every team member has commit access. This could work as a staging area for tools we would like to promote to `devtools` later on.
We could maybe sort this into directories for its purpose: * packaging * security * devops * testing * bugwrangler * misc
Tools that can be added here is the `ch` scripts from Bluewind, and the pkg-* tools eli has created. I also have some tools to look for pkgname in archweb, check them out from svn and check them against a nvchecker file.
This would hopefully give us a space where we can experiment with new maintainer tools in a collaborative manner. I'd love to hear some feedback or thoughs on this!
On 3/14/19 8:46 AM, Morten Linderud via arch-dev-public wrote:
There is a *lot* of small tools people have written over the years that resides in bin/ directories which could be useful for more people. We also have several such tools on soyuz, where sogrep was added to devtools this week.
I have been thinking it could be great to have a simple `contrib` repository where every team member has commit access. This could work as a staging area for tools we would like to promote to `devtools` later on.
We could maybe sort this into directories for its purpose:
- packaging
- security
- devops
- testing
- bugwrangler
- misc
Tools that can be added here is the `ch` scripts from Bluewind, and the pkg-* tools eli has created. I also have some tools to look for pkgname in archweb, check them out from svn and check them against a nvchecker file.
This would hopefully give us a space where we can experiment with new maintainer tools in a collaborative manner. I'd love to hear some feedback or thoughs on this!
I was fairly sure any user can create a git repo on our server. Look at "Developer Projects" on https://git.archlinux.org/ . Or use github, where some of these scripts are already located.
I don't see the need for another repository.
A
On Thu, Mar 14, 2019 at 09:46:32AM +1000, Allan McRae wrote:
I was fairly sure any user can create a git repo on our server. Look at "Developer Projects" on https://git.archlinux.org/ .
No. Luna isn't setup with the archuser role and all git operations like creating repositories are done by hand. If you have access, I assume you can create repositories under the `/users` directory at will.
This is why we have talked about adding gitolite to host these repositories, but that have been somewhat stalled along with the svn->git migration talks along with me not finishing up the role completely :x
Or use github, where some of these scripts are already located.
I'm a bit unsure what you are trying to say. Do you want the authors to just publicize it themselves as separate git repositories? On personal accounts or the Arch Linux organization?
Morten Linderud via arch-dev-public arch-dev-public@archlinux.org on Thu, 2019/03/14 01:02:
This is why we have talked about adding gitolite to host these repositories
I've wanted to propose this a long time ago...
In fact you can give every user full access to it own area in every repository, while important branches are still protected. I have several git servers that use something like this:
repo devtools RW+ = admin RW+ personal/USER/ general/ = @all RW master$ develop$ refs/tags/ = @devtools R = @all
-> admin can do everything -> random user 'foo' can do everything in personal/foo/ and general/ -> random user 'bar' can do everything in personal/bar/ and general/ -> members of group 'devtools' can push to master and develop and push tags, but forced push is denied -> random (authenticated) user can read everything
On Thu, Mar 14, 2019 at 09:14:22AM +0100, Christian Hesse wrote:
Morten Linderud via arch-dev-public arch-dev-public@archlinux.org on Thu, 2019/03/14 01:02:
This is why we have talked about adding gitolite to host these repositories
I've wanted to propose this a long time ago...
In fact you can give every user full access to it own area in every repository, while important branches are still protected. I have several git servers that use something like this:
repo devtools RW+ = admin RW+ personal/USER/ general/ = @all RW master$ develop$ refs/tags/ = @devtools R = @all
-> admin can do everything -> random user 'foo' can do everything in personal/foo/ and general/ -> random user 'bar' can do everything in personal/bar/ and general/ -> members of group 'devtools' can push to master and develop and push tags, but forced push is denied -> random (authenticated) user can read everything
Yep!
Here is the proposed patch set compared to an outdated master branch: https://github.com/Foxboron/infrastructure/compare/foxboron/git
It was put on hold as there is a cgit vs gitlab dicussion in relation to the svn->git migration. But lets not derail the `contrib` repo discussion :)
[2019-03-13 23:46:10 +0100] Morten Linderud via arch-dev-public:
There is a *lot* of small tools people have written over the years that resides in bin/ directories which could be useful for more people. We also have several such tools on soyuz, where sogrep was added to devtools this week.
I have been thinking it could be great to have a simple `contrib` repository where every team member has commit access. This could work as a staging area for tools we would like to promote to `devtools` later on.
We could maybe sort this into directories for its purpose:
- packaging
- security
- devops
- testing
- bugwrangler
- misc
Tools that can be added here is the `ch` scripts from Bluewind, and the pkg-* tools eli has created. I also have some tools to look for pkgname in archweb, check them out from svn and check them against a nvchecker file.
This would hopefully give us a space where we can experiment with new maintainer tools in a collaborative manner. I'd love to hear some feedback or thoughs on this!
I think it's a great idea but it needs a solid maintainer. Without a clear leader it's (probably) going to be a free for all and we'll drown under bikeshedding issues within a month. But of course that doesn't mean we'd lose anything trying anyhow.
Among other things, I'd personally like to see the repo maintainer enforce sensible and consistent naming for the tools, preferring longer, explicit names over shorter ones. For instance, I'm sure many of us have one-letter scripts and if we contribute them all there's bound to be collisions along with the problem of not knowing at first glance what each tool does. We could maintain a bash alias file containing everyone's favorite nickname for each tool.
My two cents.
On Thu, 14 Mar 2019 at 01:05, Gaetan Bisson via arch-dev-public < arch-dev-public@archlinux.org> wrote:
[2019-03-13 23:46:10 +0100] Morten Linderud via arch-dev-public:
There is a *lot* of small tools people have written over the years that
resides
in bin/ directories which could be useful for more people. We also have
several
such tools on soyuz, where sogrep was added to devtools this week.
I have been thinking it could be great to have a simple `contrib`
repository
where every team member has commit access. This could work as a staging
area for
tools we would like to promote to `devtools` later on.
We could maybe sort this into directories for its purpose:
- packaging
- security
- devops
- testing
- bugwrangler
- misc
Tools that can be added here is the `ch` scripts from Bluewind, and the
pkg-*
tools eli has created. I also have some tools to look for pkgname in
archweb,
check them out from svn and check them against a nvchecker file.
This would hopefully give us a space where we can experiment with new
maintainer
tools in a collaborative manner. I'd love to hear some feedback or
thoughs on
this!
I think it's a great idea but it needs a solid maintainer. Without a clear leader it's (probably) going to be a free for all and we'll drown under bikeshedding issues within a month. But of course that doesn't mean we'd lose anything trying anyhow.
I also really like the idea but I think we could have a bunch of maintainers and have everybody else just make PRs. However, then it would be kind of like the devtools repo I suppose. We should decide exactly how this repo would be different from devtools. If both of them necessitate high-quality submissions with a PR-only system, I see them overlapping. We can resolve this in two ways:
1. Not have an extra repo but strongly encourage PRs against devtools with new tools. 2. Open-for-all devtools-contrib repo as originally suggested by Morten.
I actually prefer 2. as this encourages openly sharing even half-baked scripts. This sets a lower barrier for entry and good tools with good documentation could still be promoted to be part of devtools.
Among other things, I'd personally like to see the repo maintainer enforce sensible and consistent naming for the tools, preferring longer, explicit names over shorter ones. For instance, I'm sure many of us have one-letter scripts and if we contribute them all there's bound to be collisions along with the problem of not knowing at first glance what each tool does. We could maintain a bash alias file containing everyone's favorite nickname for each tool.
I see your concerns and I'm not sure we should even try to address these for what is meant to be a heap of random useful tools. I think there is a lot of value in collecting the tools. I think about it a little bit like AUR where the low barrier of entry makes everybody just throw stuff out there and then the best tools are promoted.
This would hopefully give us a space where we can experiment with new maintainer tools in a collaborative manner. I'd love to hear some feedback or thoughs on this!
I think it's a great idea but it needs a solid maintainer. Without a clear leader it's (probably) going to be a free for all and we'll drown under bikeshedding issues within a month. But of course that doesn't mean we'd lose anything trying anyhow.
+1 here. Without a solid maintainer keeping things in check and helping order things around (e.g., how are these scripts going to be distributed) things will be somewhat hard to be useful.
Among other things, I'd personally like to see the repo maintainer enforce sensible and consistent naming for the tools, preferring longer, explicit names over shorter ones. For instance, I'm sure many of us have one-letter scripts and if we contribute them all there's bound to be collisions along with the problem of not knowing at first glance what each tool does. We could maintain a bash alias file containing everyone's favorite nickname for each tool.
I agree with this. I'd also like to add that it'd be nice to have (even if just philosophically) the goal of having contrib/ scripts being promoted to devtools or somewhere else.
Thanks, -Santiago.
On Wed, Mar 13, 2019 at 02:05:47PM -1000, Gaetan Bisson via arch-dev-public wrote:
I think it's a great idea but it needs a solid maintainer. Without a clear leader it's (probably) going to be a free for all and we'll drown under bikeshedding issues within a month. But of course that doesn't mean we'd lose anything trying anyhow.
Among other things, I'd personally like to see the repo maintainer enforce sensible and consistent naming for the tools, preferring longer, explicit names over shorter ones. For instance, I'm sure many of us have one-letter scripts and if we contribute them all there's bound to be collisions along with the problem of not knowing at first glance what each tool does. We could maintain a bash alias file containing everyone's favorite nickname for each tool.
If we want someone to a dedicated maintainer, I can probably do so. But I believe that we can give everyone commit access, block commits to master and just enforce a system where two reviews are needed before merge.
I really don't think more is needed, but as noted; I can probably take some responsibilities if the devs think that is warranted.
When it comes to packaging and naming conflicts, I wonder if it's just easier to drop all the supplied files into `/usr/share/archcontrib` or something. Makes it easier to package and doesn't clutter anyone's PATH with a lot of (sometimes) unneeded tools.
On Mon, Mar 18, 2019, 04:31 Morten Linderud via arch-dev-public <arch-dev-public@archlinux.org mailto:arch-dev-public@archlinux.org> wrote:
On Wed, Mar 13, 2019 at 02:05:47PM -1000, Gaetan Bisson via arch-dev-public wrote: > I think it's a great idea but it needs a solid maintainer. Without a > clear leader it's (probably) going to be a free for all and we'll drown > under bikeshedding issues within a month. But of course that doesn't > mean we'd lose anything trying anyhow. > > Among other things, I'd personally like to see the repo maintainer > enforce sensible and consistent naming for the tools, preferring longer, > explicit names over shorter ones. For instance, I'm sure many of us have > one-letter scripts and if we contribute them all there's bound to be > collisions along with the problem of not knowing at first glance what > each tool does. We could maintain a bash alias file containing > everyone's favorite nickname for each tool.
If we want someone to a dedicated maintainer, I can probably do so. But I believe that we can give everyone commit access, block commits to master and just enforce a system where two reviews are needed before merge.
I really don't think more is needed, but as noted; I can probably take some responsibilities if the devs think that is warranted.
When it comes to packaging and naming conflicts, I wonder if it's just easier to drop all the supplied files into `/usr/share/archcontrib` or something. Makes it easier to package and doesn't clutter anyone's PATH with a lot of (sometimes) unneeded tools.
-- Morten Linderud PGP: 9C02FF419FECBE16 -----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEktnGzemaICTWkKdu50JoO6CMsv8FAlyOvK0ACgkQ50JoO6CM sv8lBQ//aoNFKxUDM4Qcpi6nXXDDw51loxJp4AB+SYTv/Vml5tag7fVAkniUphLD ujJYY8/sSVqke29YYpG+OyJIIp91jP7WJ6nwtRVY2lsfOOUGkEX3HKEN0NCHxe9/ GpQHn1GqZAp3+FjSwxvJabpm+aGDL4CLVErdbiDKocpZJf61WSAXNHpGPj26PzUm L+fIkPDu8SEqENBVVkC9cKnf2+oInQ2ECHknSN/lBbNbwh2z2kAW46kglJXryJcE jO2gYt9kzU6NqcfKwHXHY0XCQd0H0pVMjfaK6PB1N+dwbgHAAbyhV/SIziFEP9kc PDK5NAFAcgQI+999Q9T0nufc+lqvPn9MuQ89OPINQhwTGCxEkUlicVrBYY/9ZFhC rrsTp7LExeBsMhAE6havnv2UHgkJH9ttv3FmSi81HsUYsgrxsUXpijqZUSacra5l 4VEGR9pPjsdERco2UZo9hcneLutQn0T2mCVLgIiCdjS2ZjsQuuHZ5RNBkyeNnWOC lBOBq7lddxhIiWGAjz60KOox5KL68OYlY4mZXbMQ2x2n3v2XF5VVL/ncp9F9CFo4 J6JGjab38LEemMnOFBgFS2XkRMmV7G+siD1++VdEKSCoy2tICPm5SAQiG9Ueo48y nUMD2+gfkzFNJmVQdym9YVxAKlbLWDGVHGd9/a43ZzjKKIReJgk= =y2os -----END PGP SIGNATURE-----
So shall we progress with this? I don't think we can lose anything by just trying it out. Let's put up a GitHub repo and get coding. Of course it should be mirrored to our infra.
On Thu, Mar 21, 2019 at 11:34:05PM +0100, Sven-Hendrik Haase via arch-dev-public wrote:
On Mon, Mar 18, 2019, 04:31 Morten Linderud via arch-dev-public <arch-dev-public@archlinux.org mailto:arch-dev-public@archlinux.org> wrote: So shall we progress with this? I don't think we can lose anything by just trying it out. Let's put up a GitHub repo and get coding. Of course it should be mirrored to our infra.
+1.
Morten, why don't you start a repository on GitHub and we start figuring things out while things move. I fully agree with your policy on no pushes to master and 2 reviews before merging.
I'm also more than happy helping with maintenance work, although I don't think I have enough time for being the only maintainer of this.
Thanks, -Santiago.
On Thu, Mar 21, 2019 at 06:58:56PM -0400, Santiago Torres-Arias via arch-dev-public wrote:
Morten, why don't you start a repository on GitHub and we start figuring things out while things move.
Moving github repos between organization and user is a hassle, so would prefer if a dev could create a repo and provide access on github at least.
I'm also more than happy helping with maintenance work, although I don't think I have enough time for being the only maintainer of this.
Yasss! +1
Yo!
A followup mail that the contrib repository has been created :)
It currently located at https://github.com/archlinux/contrib and all scripts welcome!
Currently there is a few scripts there currently! `co-maintainers` helps you find packages in repositories that has N maintainers. Super handy if you need to figure out if any packages you have installed is missing a co-maintainer or two.
There is also a quick script to help fetch PKGBUILDs from AUR users for review and Sven has uploaded a quick script to build a package on dragon.
Going to try figure out the packaging bit through the weekend.
On Wed, Mar 27, 2019 at 11:14:17PM +0100, Morten Linderud via arch-dev-public wrote:
Going to try figure out the packaging bit through the weekend.
And one long weekend later and we have "archlinux-contrib" in [community]!
There are currently a few scripts available.
- checkservices Originally writen by seblu, and commited to contrib by Giancarlo, it checks which services needs to be restarted after packages have been updated.
- co-maintainers It queries archweb and helps you figure out which of your packages has how many co-maintainers. It's useful in the case where you are going on vacation and want to produce a list of packages which currently only has you as the maintainer.
- review Helps you download PKGBUILDs from an AUR maintainer. This is useful if you want to review TU candidates.
- compose-asa Aids security team members composing the advisory email locally.
- security-tracker-check Checks stdin for CVEs and cross-checks with our security tracker if they have been added or not. Useful if you are looking over oss-security emails :)
svenstaro also wrote a `offline-build` tool that went into contrib. Eli polished it and moved it to devtools after some work. Jelle also has a pull-request working on "repo-sec-checker" checks if there are binaries in a package/repo which does not have the appropriate security hardening applied.
It should be noted that all of this is currently being packaged to `/usr/share/archlinux/contrib` as it seems like an better idea to not clutter peoples paths with misc tools.
I'll be super happy if people contribute other misc scripts, and everyone that wishes will be added to the github org and can handle pull-requests.
On 9/11/19 11:10 PM, Morten Linderud via arch-dev-public wrote:
svenstaro also wrote a `offline-build` tool that went into contrib. Eli polished it and moved it to devtools after some work.
offload-build, not offline-build, fwiw!
participants (8)
-
Allan McRae
-
Christian Hesse
-
Gaetan Bisson
-
Morten Linderud
-
Robin Broda
-
Santiago Torres-Arias
-
Sven-Hendrik Haase
-
Sven-Hendrik Haase