Call for participation: Git packaging POC 🚀🎉
Hi everyone! Levente and I have been busy preparing a test environment for the new git package workflow, which is going to replace the svn repository. To test the new git package setup install `devtools-git-poc` from the [community] repository and use the new `pkgctl` utility. Please check each time if there is a new upgrade before playing around. The goal of the testing is to figure out UX issues, bugs and larger issues that would need to be dealt with before a git migration can happen. It's therefor very important that people sit down and play around with the tools :) Support and feedback will be dealt with in the #archlinux-projects channel. Report bugs in GitLab with the following additional information: - devtools-git-poc package version - full PKGBUILD or pkg repo including the Git hash - terminal input and output The workflow should be the following: Basic setup ----------- ### setup packager Make sure your personal `makepkg.conf` contains your packager identity and gpg key. ### authenticate to gitlab pkgctl auth maintain existing packages -------------------------- ### clone an existing package pkgctl repo clone existing-package ### build a package pkgctl build ### release a package pkgctl release pkgctl db update ### combined UX The last three steps can also be expressed as: pkgctl build --release --db-update ### rebuild into staging vim PKGBUILD pkgctl build --staging pkgctl release --staging pkgctl db update This can be completely condensed by: pkgctl build --rebuild --staging --release --db-update maintain new packages --------------------- ### create a new package repo without history pkgctl repo create new-pkgbase pkgctl repo clone new-pkgbase ### add existing AUR repo cd packagerepo pkgctl repo create pkgctl repo configure ### first official build/release Specify the target repo on first run, as the tool can't know it's destination automatically yet pkgctl build --repo extra pkgctl release --repo extra --db-update --- The packages are uploaded into a test group here: https://gitlab.archlinux.org/bot-test/packages This test also includes junior developer and junior packagers. They can only push packages into staging and testing, and not release into the stable repositories. We currently have a couple of people with these permissions to try and figure out issues on that part of the setup. If anyone is interested helping out on this please reach out on the #archlinux-projects IRC channel on Libera. Background on the changes can be found here: https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.or... Cheers!
Quick addendum as it may not have been clear enough: while only interacting with `pkgctl` and nothing else, you will operate against an isolated sandbox environment. Go nuts, here be dragons! On 2/2/23 01:45, Morten Linderud wrote:
To test the new git package setup install `devtools-git-poc` from the [community] repository and use the new `pkgctl` utility. Please check each time if there is a new upgrade before playing around.
We just release a quick fix, please make sure to use 20230202-3 or later: pacman -Syu 'devtools-git-poc>=20230202-3' Cheers and happy testing, Levente
On 02/02/2023 01:45, Morten Linderud wrote:
Hi everyone!
Levente and I have been busy preparing a test environment for the new git package workflow, which is going to replace the svn repository.
To test the new git package setup install `devtools-git-poc` from the [community] repository and use the new `pkgctl` utility. Please check each time if there is a new upgrade before playing around.
The goal of the testing is to figure out UX issues, bugs and larger issues that would need to be dealt with before a git migration can happen. It's therefor very important that people sit down and play around with the tools :)
While working on the archweb side of things I noticed that we now have extra, extra-testing, extra-staging. Is this change intentional? This would require some manual intervention as the default is to use testing/staging. On the PoC there is some curious difference between testing and extra-testing https://repos.sandbox.archlinux.org/extra-testing/os/x86_64/ https://repos.sandbox.archlinux.org/testing/os/x86_64/ Many tools/integrations probably depend on testing, so this surely will break. Greetings, Jelle
Jelle van der Waa 於 2023/2/18 22:36 寫道:
On 02/02/2023 01:45, Morten Linderud wrote:
Hi everyone!
Levente and I have been busy preparing a test environment for the new git package workflow, which is going to replace the svn repository.
To test the new git package setup install `devtools-git-poc` from the [community] repository and use the new `pkgctl` utility. Please check each time if there is a new upgrade before playing around.
The goal of the testing is to figure out UX issues, bugs and larger issues that would need to be dealt with before a git migration can happen. It's therefor very important that people sit down and play around with the tools :)
While working on the archweb side of things I noticed that we now have extra, extra-testing, extra-staging. Is this change intentional?
This would require some manual intervention as the default is to use testing/staging. On the PoC there is some curious difference between testing and extra-testing
https://repos.sandbox.archlinux.org/extra-testing/os/x86_64/ https://repos.sandbox.archlinux.org/testing/os/x86_64/
Many tools/integrations probably depend on testing, so this surely will break.
Greetings,
Jelle
Hi, As a record, I proposed [extra-testing] and [extra-staging] some time ago at [1], and it is considered postponed at that time. I'm not sure if other discussions made a decision or not. Anyway, manual intervention is already needed with the introduction of [core-testing] and [core-staging] repos. Regards, Chih-Hsuan Yen (yan12125) [1] https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/14#note_75324
On 2/18/23 15:36, Jelle van der Waa wrote:
On 02/02/2023 01:45, Morten Linderud wrote:
Hi everyone!
Levente and I have been busy preparing a test environment for the new git package workflow, which is going to replace the svn repository.
To test the new git package setup install `devtools-git-poc` from the [community] repository and use the new `pkgctl` utility. Please check each time if there is a new upgrade before playing around.
The goal of the testing is to figure out UX issues, bugs and larger issues that would need to be dealt with before a git migration can happen. It's therefor very important that people sit down and play around with the tools :)
While working on the archweb side of things I noticed that we now have extra, extra-testing, extra-staging. Is this change intentional?
This would require some manual intervention as the default is to use testing/staging. On the PoC there is some curious difference between testing and extra-testing
https://repos.sandbox.archlinux.org/extra-testing/os/x86_64/ https://repos.sandbox.archlinux.org/testing/os/x86_64/
Many tools/integrations probably depend on testing, so this surely will break.
Greetings,
Jelle
Hi there, Yes, we were required to change things in dbscripts and devtools anyway. Considering that all secondary tooling needs adjustments to reflect the [core-testing] layout, we have decided to just go simple and remove special casing any stability layers. It was best to break secondary tooling once (which was the case either way), instead of multiple times. So now we ended with a simple and easy to code stability layer layout without special cases: ${repo} ${repo}-staging ${repo}-testing Sincerely, Levente
We're excited to share that we're almost there with our Git packaging implementation, and we need your help to push us over the finish line. We're asking *every* package maintainer to test our proof of concept [0] at least once. Your feedback is crucial to ensure that our system works well before we release it to the public. We would greatly appreciate it if you could focus on testing as many aspects of the implementation as possible, including the changes we've listed below. Here are the latest changes to our implementation: - We're now providing GitLab tokens via the `DEVTOOLS_GITLAB_TOKEN` environment variable, which eliminates the need to store plaintext files containing secret tokens. This makes it easier for developers to integrate with our system securely. - We allow passing nocheck to the build command for bootstrap builds, which should help streamline the build process for initial bootstrap cycles. - We've added support for handling subrelease pkgrels, which should make it easier to manage different versions of our software. - The rebuild option now only bumps the pkgrel once, even when executed multiple times without committing. This should help reduce the number of unnecessary bumps in the version number. - We've improved how we handle the chosen editor for commit messages. - We are now managing all Package Maintainers in a dedicated GitLab group [1] - Additionally, we've also fixed a number of bugs that were reported in the previous round of testing. We believe that your contribution to this testing phase will help us improve the quality of our implementation and allow for a smoother migration. Your feedback will be invaluable in making our system the best it can be. Thank you in advance for your help and support. Let's work together to make our Git packaging implementation a success! [0] https://archlinux.org/packages/community/any/devtools-git-poc/ [1] https://gitlab.archlinux.org/archlinux/teams/package-maintainer-team Cheers Levente
As a headsup, we would like to move along the progress of this migration sooner rather than later. Currently 14 packagers has tested this, and considering we are 71 packagers that isn't great. It's important to sit down and test this. Both to ensure that we have all our needs covered by the tools, and to ensure things are understandable once the migration finally happens. The work started 3-4 years ago and hundreds of hours have been spent working on this, everyone has 20 minutes to sit down and try the git migration and give some feedback. -- Morten Linderud PGP: 9C02FF419FECBE16
Great work on this! It's even better than when I tested it last time. :) On 2023-04-12 18:14, Levente Polyak wrote:
- We're now providing GitLab tokens via the `DEVTOOLS_GITLAB_TOKEN` environment variable, which eliminates the need to store plaintext files containing secret tokens. This makes it easier for developers to integrate with our system securely.
I think it would also be nice to have a "password command" option wherein a command would be run every time it needs the token. For instance: token_cmd='pass pkgctltoken' That way there isn't a token hanging around in the environment.
participants (6)
-
Brett Cornwall
-
Chih-Hsuan Yen
-
gromit
-
Jelle van der Waa
-
Levente Polyak
-
Morten Linderud