- Arch-devops - lists.archlinux.org

New host keys for gitlab.archlinux.org
by Evangelos Foutras 08 May '23

08 May '23

Dear Arch DevOps Team, Following the migration of our GitLab instance from a cloud server to a new dedicated box, you will need to remove the old host keys from your $HOME/.ssh/known_hosts file. For this please do: ssh-keygen -R gitlab.archlinux.org After the above, when you attempt to connect to gitlab.archlinux.org, you will be prompted to accept a new host key. Please find below the new host key fingerprints: SHA256:ZHy7zbNNwKHukTEyFcAT2Ns+mT/wlc2vAjJULnlOroE (ECDSA) SHA256:KD3Og7bPg9glmODrqUKN0r3Jl18pQ0OVp2MT7xv/bm8 (ED25519) SHA256:2R7QoVWeZ2DpKEsZBgVKUieM+9TQu6g9N62CIJCyVtc (RSA) Please note that this change only affects the host machine and not GitLab itself. Git clones from the latter will work without change. Stay safe and frosty!

1 0

Upgrading GitLab hardware specs
by Levente Polyak 03 May '23

03 May '23

Hi all, we already seem to have a quite high base load on our current VPS instance for our GitLab. We see some periodic soft lockups for git and UI operations when resource hungry tasks are performed, like Linux kernel mirror operations or GitLab housekeeping. Soon we will have Git packaging sources on our GitLab as well, which will increase the baseline usage against our GitLab quite noticeably. We really want to avoid to cheap out on this crucial and central part of our infra. As we already sort of maxed out on the VPS capabilities, we'd proceed and move the GitLab box to a bare metal system. The best bang for the bucks seems to be the AX52 [0], which will give us a spicy and more responsive GitLab instance. Cheers, Levente [0] https://www.hetzner.com/de/dedicated-rootserver/ax52

1 0

Moving archlinux.page domain to Hetzner
by Evangelos Foutras 12 Mar '23

12 Mar '23

Gandi is taking away my free mailboxes (unused but still) [1] so it's only fair we take away archlinux.page from them and move it to Hetzner. We also gain full control of the domain this way, because right now we need to ask SPI for e.g. transfer authorization code. Assuming nobody objects, I believe we can do the transfer next weekend, or at least poke SPI to get us the code and proceed from there. [1] https://news.ycombinator.com/item?id=35080777

2 1

[arch-devops] Request to reset password for farseerfc@archlinux.org mail account
by Jiachen YANG 10 May '22

10 May '22

Hi DevOps team, This is from farseerfc, with my personal email account (farseerfc(a)gmail.com) I forget my password for my farseerfc(a)archlinux.org mail account, therefore I want to reset the password for that account. This request will be signed with my old packager key. I am going to create a new key and open a new packager key issue on gitlab after I tested my email account. Thank you all.

2 1

[arch-devops] Junior DevOps application - Leonidas Spyropoulos
by Leonidas Spyropoulos 03 May '22

03 May '22

Hi, My name is Leonidas Spyropoulos (known as artafinde / inglor on many platforms) and I'm a Trusted User at Arch Linux [1]. I'm applying to become a Junior DevOps in the devops Arch Linux team. I've been running Arch Linux from circa 2007 daily with dual boot in the old days and VFIO now for gaming. I've been contributing in open-source projects [2][3] and in Arch Linux for about a year or so (when I asked in #archlinux-devops for a gitlab account) with patches/MRs for infrastructure, work on keycloak, aurweb and more recently a tool for exporting gitlab metrics for prometheus [4][5]. I've been using ansible, terraform, vault and jenkins at work and recently changed to gitlab so I think I could apply some of this knowledge to Arch Linux infrastructure. Furthermore I got experience in Cloud infrastructure (AWS) and Kubernetes cluster along with private Docker registry (in AWS) from both devops and software development point of view. My interest in DevOps team is the admin part and automation. I've been having personal server(s) [6] for years now and I was part of the admin team at my previous job. I think automation and infrastructure-as-code is the way to manage infrastructure now days and although we lacked the automation at my previous job I very well understand how crucial things could be for others and the responsibilities which come moderating public facing servers. Additionally I think I can assist on the mundane tasks the devops team is currently dealing with because for me it would be a new interesting challenge. Sponsors: - Evangelos Foutras (foutrelis) - Kristian Klausen (klausenbusk) [1]: https://archlinux.org/people/trusted-users/#artafinde [2]: https://github.com/inglor [3]: https://gitlab.archlinux.org/artafinde [4]: https://gitlab.archlinux.org/artafinde/gitlab-exporter [6]: https://gitlab.archlinux.org/archlinux/infrastructure/-/issues/14 [5]: Personal "servers" could be from as early as 2004 a headless Pentium 3 with debian to currently 4x VPSs and an Unraid at home Regards, -- Leonidas Spyropoulos PGP: 59E43E106B247368

4 4

[arch-devops] GitLab 2FA Login Problems
by Томас 27 Mar '22

27 Mar '22

Hello, I'm looking for who I can contact for help with GitLab 2FA login problems. I set up my GitLab account today, literally a couple hours ago (username "70m4c"), but the file that held the 2FA info became corrupted. I am able to log into the SSO just fine, but cannot complete the final step to get into GitLab due to not being able to generate the appropriate 2FA codes. I also lost the 2FA recovery codes as they were in the same file. I did however upload my public keys to GitLab and can prove my identity that way. I can also access a resource from the same IP address that was used to set up the account if need be. I have access to my email and the original emails that were sent when I signed up as well. If anyone can help me reset the GitLab 2FA, I would appreciate that very much. Thank you.

2 1

[arch-devops] gluebuddy deployed
by Jelle van der Waa 20 Jan '22

20 Jan '22

Hi all, I've deployed gluebuddy.archlinux.org and it just successfully ran gluebuddy apply. So all Gitlab user permission settings are now applied from gluebuddy automated and predictable. If there are issues / wrong permissions please reach out to Levente or me. Happy crab! Jelle

1 0

[arch-devops] Meeting tomorrow 19:30 CET
by Jelle van der Waa 08 Dec '21

08 Dec '21

Hi all, I'd like to have another DevOps meeting with as topics: * Gitlab status / ToS * Mailman 3 progress * CDN status * Gluebuddy * DBScripts debug packages deployment More information is available in the pad [1] [1] https://md.archlinux.org/2H6y_4lFTOyOmw61VrEMnA

1 1

[arch-devops] Gluebuddy progress
by Jelle van der Waa 03 Dec '21

03 Dec '21

Hi all, Yesterday anthraxx and I hacked together on getting gluebuddy ready for production. Gluebuddy is a tool to automatically put Arch Linux Staff in the correct Gitlab organization/teams and can later be expanded to enforce more repository settings. The open pull request was updated to not remove our archceo Arch Linux group owner and handles our three devops onboarding/offboarding tasks of adding users to the Staff team, Infrastructure Team and Arch Linux group. [1] There are a few open questions: We match on extern_id which is the username in Gitlab and not the keycloak id, is that correct and is that an issue? For keycloak access we now use the admin account, we should rather use an openid client which has “realm-management roles” such as “query-groups, query-users, view-users” The gitlab personal token used for changing For deploying it to a live server we need: Setup a new VPS for running gluebuddy Create a systemd/service with timer so gluebuddy runs every X minutes Find a way to distribute gluebuddy, an option is to use Gitlab release where we upload a signed locally build gluebuddy (retrieve and veriy this in the ansible role). As packaging doesn’t make much sense here. Create an ansible role for gluebuddy [1] https://gitlab.archlinux.org/archlinux/gluebuddy/-/merge_requests/2 Greetings, Jelle van der Waa

1 0

[arch-devops] Wanting to give access to kevr to aur-dev
by Sven-Hendrik Haase 20 Nov '21

20 Nov '21

Hello, as some of you might know, kevr has been hard at work rewriting the AUR in Python and FastAPI. He now wants to deploy this and has been asking for infra access to the aur-dev test server. We evaluated different ways of giving him partial access that involves just mysql and logs access but as it turns out, loki and grafana make this very annoying and burdensome right now, requiring us to host a second loki and grafana just for this. As such, I suggest the practical solution of giving him a user on aur-dev in order for us to move on with this. He wouldn't have access to the real production AUR, just the test box. What this effectively entails is me giving him a user in the docker group which should give him all the access he might need. Any objections to this? Cheers, Sven

3 3