Self-managed Arch repo & build-server
Hello Arch-DevOps Community We'd like to ask you, if there's a possibilty to clone all core & extra packages from the ArchLinux GitLab Packages Group to one of our build-servers and build them for our self. I've found the pkgctl utility, but when trying to execute 'pkgctl repo clone --protocol https --universe', it creates the respective directories but is then stuck at the cloning-process. The reason for this request is: We're running 60+ Arch clients for our employees (tendency increasing), which all are managed through a central application, where we can execute OS-updates, software-installs, firmware-updates, etc. The main problem is, that if we execute a company-wide update, some people directly install them and some postpone them as far as possible, sometimes taking vacation without installing the updates first. This leads us to the problem that the clients don't have the same state, sometimes with a 2 week or more gap, which makes it difficult for us to troubleshoot any problems. We had in mind, cloning all package-sources and then making snapshots of the build-directory, for a "stable" release. When using the gitlab-api to get all the urls of the repos, after some time the request times out, because we made to many requests. We do understand, that it's not the idea of Arch Linux to be a "part-stable" distribution, since of it's rolling-release model, but for maintenance-reasons it's the best option, having a rolling-release distro with up-to-date software and the ability to control, which packages are getting installed and which not. I hope you understand our proposal. -- Best regards Pascal Bryner Junior System Administrator
On 24-08-2023 09:05, Pascal Bryner wrote:
Hello Arch-DevOps Community We'd like to ask you, if there's a possibilty to clone all core & extra packages from the ArchLinux GitLab Packages Group to one of our build-servers and build them for our self.
You can do this already using rsync. More information can be found here: https://wiki.archlinux.org/title/DeveloperWiki:NewMirrors Personally, I would pick your upstream mirror based on dislike: you'll be using their bandwidth to clone the mirror, so pick the webhosting company you like the least.
The reason for this request is: We're running 60+ Arch clients for our employees (tendency increasing), which all are managed through a central application, where we can execute OS-updates, software-installs, firmware-updates, etc. The main problem is, that if we execute a company-wide update, some people directly install them and some postpone them as far as possible, sometimes taking vacation without installing the updates first. This leads us to the problem that the clients don't have the same state, sometimes with a 2 week or more gap, which makes it difficult for us to troubleshoot any problems. We had in mind, cloning all package-sources and then making snapshots of the build-directory, for a "stable" release. When using the gitlab-api to get all the urls of the repos, after some time the request times out, because we made to many requests.
I do exactly this for exactly this reason. It works well :) Kind regards, Maarten de Vries
Thank you. But I guess there was a slight confusion with my question. Do you just clone the finished pkg.tar.zst or the "source" with the PKGBUILD, as my original question was about? Because when rsyncing a mirror, (e.g. init7 https://mirror.init7.net/archlinux/) they only provide the finished pkg.tar.zst. A "source/" directory is also not in sight. We need the PKGBUILD's to build them for us, so in case we have a stable-release and e.g. chromium gets a security update, we cherry-pick the changed PKGBUILD into the "stable"-repo and build it against the old libraries. Thanks.
Maarten de Vries <maarten@de-vri.es> hat am 24.08.2023 09:55 CEST geschrieben:
On 24-08-2023 09:05, Pascal Bryner wrote:
Hello Arch-DevOps Community
We'd like to ask you, if there's a possibilty to clone all core & extra packages from the ArchLinux GitLab Packages Group to one of our build-servers and build them for our self.
You can do this already using rsync. More information can be found here: https://wiki.archlinux.org/title/DeveloperWiki:NewMirrors
Personally, I would pick your upstream mirror based on dislike: you'll be using their bandwidth to clone the mirror, so pick the webhosting company you like the least.
The reason for this request is: We're running 60+ Arch clients for our employees (tendency increasing), which all are managed through a central application, where we can execute OS-updates, software-installs, firmware-updates, etc.
The main problem is, that if we execute a company-wide update, some people directly install them and some postpone them as far as possible, sometimes taking vacation without installing the updates first.
This leads us to the problem that the clients don't have the same state, sometimes with a 2 week or more gap, which makes it difficult for us to troubleshoot any problems.
We had in mind, cloning all package-sources and then making snapshots of the build-directory, for a "stable" release. When using the gitlab-api to get all the urls of the repos, after some time the request times out, because we made to many requests.
I do exactly this for exactly this reason. It works well :)
Kind regards,
Maarten de Vries
-- Freundliche Grüsse Pascal Bryner Junior System Administrator -- Hostpoint AG Neue Jonastrasse 60 | 8640 Rapperswil-Jona | Schweiz +41 55 220 63 51 | pascal.bryner@hostpoint.ch | www.hostpoint.ch
On 23/08/24 09:05AM, Pascal Bryner wrote:
Hello Arch-DevOps Community
Hello Pascal,
We'd like to ask you, if there's a possibilty to clone all core & extra packages from the ArchLinux GitLab Packages Group to one of our build-servers and build them for our self.
I've found the pkgctl utility, but when trying to execute 'pkgctl repo clone --protocol https --universe', it creates the respective directories but is then stuck at the cloning-process.
Indeed it is kinda expected to hit the rate limits of our Gitlab instance when you clone a lot of package sources via HTTPS. If you clone via SSH instead you should not hit these limits. Just be aware that there are currently problems[0] with SSH key unlocking & the parallelization which not yet fixed in devtools[1]. As a workaround you could setup a ControlMaster[2] or cache the keys with ssh-agent and do a first clone manually.
The reason for this request is: We're running 60+ Arch clients for our employees (tendency increasing), which all are managed through a central application, where we can execute OS-updates, software-installs, firmware-updates, etc.
The main problem is, that if we execute a company-wide update, some people directly install them and some postpone them as far as possible, sometimes taking vacation without installing the updates first.
This leads us to the problem that the clients don't have the same state, sometimes with a 2 week or more gap, which makes it difficult for us to troubleshoot any problems.
I don't really understand why it is such a big problem that the machines carry different state aside from smaller bugs that are definitely not worth the hassle you are proposing as a solution here. If you just want to have all machines on the same state why don't you fully incorporate the update process in your device management system and remove the ability for users to defer the update? Users which do not take part in this should be able to figure out the bugs on their own or with limited support :) If you want to limit how new the software is the clients use you could just use something like a mirror which delays the updates and use that for all clients ...
We had in mind, cloning all package-sources and then making snapshots of the build-directory, for a "stable" release. When using the gitlab-api to get all the urls of the repos, after some time the request times out, because we made to many requests.
We do understand, that it's not the idea of Arch Linux to be a "part-stable" distribution, since of it's rolling-release model, but for maintenance-reasons it's the best option, having a rolling-release distro with up-to-date software and the ability to control, which packages are getting installed and which not.
So you considered ... creating a new distribution aka forking ArchLinux? Why would you need to build/adjust the sources? Why is it not enough to work with the packages instead of the sources? You are of course free to do all of this, but I think you need to understand that this is a lot of work which currently seems unjustified given my limited understanding of your problem/use case. So as you might have already guessed I am a bit sceptical regarding your idea(s) but also very interested in your reasons/motivation. Cheers, gromit [0] https://gitlab.archlinux.org/archlinux/devtools/-/issues/148 [1] https://gitlab.archlinux.org/archlinux/devtools/-/merge_requests/180 [2] https://wiki.archlinux.org/title/DeveloperWiki:How_to_be_a_packager#Avoid_ha...
On 2023-08-24 18:30:14 (+0200), Christian Heusel wrote:
On 23/08/24 09:05AM, Pascal Bryner wrote:
The reason for this request is: We're running 60+ Arch clients for our employees (tendency increasing), which all are managed through a central application, where we can execute OS-updates, software-installs, firmware-updates, etc.
The main problem is, that if we execute a company-wide update, some people directly install them and some postpone them as far as possible, sometimes taking vacation without installing the updates first.
This leads us to the problem that the clients don't have the same state, sometimes with a 2 week or more gap, which makes it difficult for us to troubleshoot any problems.
To add an additional datapoint to this: You might want to have a look at an image based update scenario for your clients, using systemd-sysupdate [1] etc.
So as you might have already guessed I am a bit sceptical regarding your idea(s) but also very interested in your reasons/motivation.
I can only 2nd this. From my (admittedly limited) perspective it might be more beneficial to work with an unmodified Arch Linux and only provide certain packages in an additional repository (in case you need something outside the scope of the official repositories). An image-based update scenario would then significantly cut down on the diversity of states a client system might be in. If you do anything like that and are able to share your experience I (and most likely all of us) would be very interested! Best, David [1] https://man.archlinux.org/man/systemd-sysupdate.8 -- https://sleepmap.de
participants (4)
-
Christian Heusel
-
David Runge
-
Maarten de Vries
-
Pascal Bryner