On 24-08-2023 09:05, Pascal Bryner wrote:

Hello Arch-DevOps Community We'd like to ask you, if there's a possibilty to clone all core & extra packages from the ArchLinux GitLab Packages Group to one of our build-servers and build them for our self.

You can do this already using rsync. More information can be found here: https://wiki.archlinux.org/title/DeveloperWiki:NewMirrors Personally, I would pick your upstream mirror based on dislike: you'll be using their bandwidth to clone the mirror, so pick the webhosting company you like the least.

The reason for this request is: We're running 60+ Arch clients for our employees (tendency increasing), which all are managed through a central application, where we can execute OS-updates, software-installs, firmware-updates, etc. The main problem is, that if we execute a company-wide update, some people directly install them and some postpone them as far as possible, sometimes taking vacation without installing the updates first. This leads us to the problem that the clients don't have the same state, sometimes with a 2 week or more gap, which makes it difficult for us to troubleshoot any problems. We had in mind, cloning all package-sources and then making snapshots of the build-directory, for a "stable" release. When using the gitlab-api to get all the urls of the repos, after some time the request times out, because we made to many requests.

I do exactly this for exactly this reason. It works well :) Kind regards, Maarten de Vries

On 23/08/24 09:05AM, Pascal Bryner wrote:

Hello Arch-DevOps Community

Hello Pascal,

We'd like to ask you, if there's a possibilty to clone all core & extra packages from the ArchLinux GitLab Packages Group to one of our build-servers and build them for our self.

I've found the pkgctl utility, but when trying to execute 'pkgctl repo clone --protocol https --universe', it creates the respective directories but is then stuck at the cloning-process.

Indeed it is kinda expected to hit the rate limits of our Gitlab instance when you clone a lot of package sources via HTTPS. If you clone via SSH instead you should not hit these limits. Just be aware that there are currently problems[0] with SSH key unlocking & the parallelization which not yet fixed in devtools[1]. As a workaround you could setup a ControlMaster[2] or cache the keys with ssh-agent and do a first clone manually.

The reason for this request is: We're running 60+ Arch clients for our employees (tendency increasing), which all are managed through a central application, where we can execute OS-updates, software-installs, firmware-updates, etc.

The main problem is, that if we execute a company-wide update, some people directly install them and some postpone them as far as possible, sometimes taking vacation without installing the updates first.

This leads us to the problem that the clients don't have the same state, sometimes with a 2 week or more gap, which makes it difficult for us to troubleshoot any problems.

I don't really understand why it is such a big problem that the machines carry different state aside from smaller bugs that are definitely not worth the hassle you are proposing as a solution here. If you just want to have all machines on the same state why don't you fully incorporate the update process in your device management system and remove the ability for users to defer the update? Users which do not take part in this should be able to figure out the bugs on their own or with limited support :) If you want to limit how new the software is the clients use you could just use something like a mirror which delays the updates and use that for all clients ...

We had in mind, cloning all package-sources and then making snapshots of the build-directory, for a "stable" release. When using the gitlab-api to get all the urls of the repos, after some time the request times out, because we made to many requests.

We do understand, that it's not the idea of Arch Linux to be a "part-stable" distribution, since of it's rolling-release model, but for maintenance-reasons it's the best option, having a rolling-release distro with up-to-date software and the ability to control, which packages are getting installed and which not.

So you considered ... creating a new distribution aka forking ArchLinux? Why would you need to build/adjust the sources? Why is it not enough to work with the packages instead of the sources? You are of course free to do all of this, but I think you need to understand that this is a lot of work which currently seems unjustified given my limited understanding of your problem/use case. So as you might have already guessed I am a bit sceptical regarding your idea(s) but also very interested in your reasons/motivation. Cheers, gromit [0] https://gitlab.archlinux.org/archlinux/devtools/-/issues/148 [1] https://gitlab.archlinux.org/archlinux/devtools/-/merge_requests/180 [2] https://wiki.archlinux.org/title/DeveloperWiki:How_to_be_a_packager#Avoid_ha...