- Arch-devops - lists.archlinux.org

[arch-devops] DevOps meeting tomorrow 2020-09-10 18:30 UTC - 20:30 CEST
by Jelle van der Waa 09 Sep '20

09 Sep '20

Hi All, Tommorow we have another DevOps meeting with the following topics: * Mail migration * Keycloak opening up * Keycloak group structure [1] * Kape Servers configuration coordination * Prometheus changes & Zabbix [1] https://gitlab.archlinux.org/archlinux/infrastructure/-/issues/99

1 0

[arch-devops] terraform 0.13 upgrade
by Sven-Hendrik Haase 27 Aug '20

27 Aug '20

Hey guys, Just a quick heads-up to please not upgrade to Terraform 0.13 if you want to use Terraform currently. I'll handle the upgrade once the Keycloak provider is available in the registry. In the mean time, please stay on Terraform 0.12. See https://gitlab.archlinux.org/archlinux/infrastructure/-/issues/100 Cheers, Sven

3 3

[arch-devops] DevOps meeting today 18:30 UTC - 20:30 CEST
by Jelle van der Waa 13 Aug '20

13 Aug '20

Hi all, Subjects are: * Mail migration/situation * Grafana/Prometheus update * Mirror.pkgbuild.com * Keycloak theme * Keycloak webauthn * AUR load issues * Next Keycloak steps! Greetings, Jelle

1 0

[arch-devops] Resignation
by Bartłomiej Piotrowski 06 Aug '20

06 Aug '20

Hi all, I think it's safe to say that I won't be able to find enough time to meaningfully contribute to DevOps team at this point. Thanks and good luck, Bart

2 1

[arch-devops] Stepping down from DevOps
by Florian Pritz 02 Aug '20

02 Aug '20

Hi, As you have probably noticed, my involvement in DevOps tasks has dwindled in recent months. I have given away a few tasks to others already, but since I hardly use my root access any more, I think it is finally time to officially open up the spot and let someone else take over. Back when we started the DevOps team nearly 5 years ago (good god, time flies), Allan asked me if I wanted to lead the "infrastructure team" (old name). I am glad that I have never needed to use the team leader hat since we could discuss pretty much every problem and reach an agreement. Given that things have worked so well in the past, I'm inclined to simply discontinue the (unused) team leader hat. Considering how little, well, actually not at all, I have used the hat, most people are probably not even aware of it, so this feel like the most logical choice. I think having a core team of active people that discuss the issues and then come to an agreement has worked well in the past and it is something worth keeping. I'm still amazed by what you guys accomplished over the last year and I hope that making space and letting others join will only allow the team to be even more awesome! All that said, please disable my root access and remove me from the various infrastructure/devops/root related email aliases and services (kanboard and gitlab come to mind). Please do keep me on the mirrors(a)archlinux.org alias for now though since I still handle mirror related requests. I also see that I am the only one on the dmarc-reports@ alias. If nobody is interested in the reports, please remove the relevant line from the DMARC DNS entry in the archlinux.org zone. Also, please remove my email address from the explicit hetzner contact list. If you think of anything else that I forgot, please disable it or, if in doubt, feel free to ask on IRC if I still need it. Thanks for the journey! Florian

2 1

[arch-devops] Meeting 30 July 18:30 UTC - 20:30 CEST
by Jelle van der Waa 28 Jul '20

28 Jul '20

Hi All, On Thursday we have another devops meetings with the following topics: * Gitlab go live, limits and monitoring * Grafana/prometheus * AUR migration notes/improvements * Next server(s) to migrate Greetings, Jelle

1 0

[arch-devops] Meeting 16 July 18:30 UTC - 20:30 CEST
by Jelle van der Waa 15 Jul '20

15 Jul '20

Hi all, Tommorow we have another devops meeting, topics: * prometheus monitoring * revoking access to inactive members * finer grained access * password manager * freswa Cya tommorow, Jelle

1 0

[arch-devops] Add Frederik Schwan to the devops team
by Sven-Hendrik Haase 15 Jul '20

15 Jul '20

Hey all, I propose adding Frederik Schwan to the devops team. He's eager to help Arch, he's knowledgeable, and does devops professionally. He's got a good amount of experience with mail stacks and cloud stuff and he'd like to help us out. In fact, he's already helped quite a bit with testing and some accessless devops tasks but the number of tasks requiring access that we currently have far outweigh the accessless tasks and I think he'd like to become more involved. Thoughts? Cheers, Sven

6 7

[arch-devops] Let's change our primary backup to a managed storage box
by Sven-Hendrik Haase 09 Jul '20

09 Jul '20

Hey all, I want to suggest we change our primary backup box to a managed box, namely a BX60 from Hetzner [0] for 40€/m with 10TB capacity. We currently have a SB35 with 2x3TB in a RAID1. The current box (vostok) is getting full and is sitting at 20% capacity remaining. The storage box supports Borg and so it's unlikely that we'll have to change our backup scheme. Why a BX60 and not a BX50 even though the capacity would currently suffice? I'd like to also start backing up the archive. The archive will become more relevant in Arch as reproducible builds gain more popularity and I think keeping the archive alive and well is an important part of that. We currently do not back up the archive for capacity reasons but we should start doing that. BX50 with its 5TB puts us too close to our projected needs (roughly 2.3TB (current effective backup size) + 2.3TB (archive)). Advantages of a BX60: - Less stuff to manage as Hetzner will handle disk failures and updates for us - More redundancy (FAQ says it can tolerate multiple disk failures while we can currently only handle a single disk failure) - Much higher capacity (about 3.4x as much) Disadvantages: - 10€ more per month - We'll have to migrate (though probably pretty easy to do as the backup stuff is pretty nicely ansibled) - Only 10 connections allowed at once (workable for us but worth mentioning) - We currently have no operating experience with storage boxes from Hetzner What do you think? Cheers, Sven [0] https://www.hetzner.de/storage/storage-box/bx60

2 5

[arch-devops] Resignation
by fukawi2 | Arch Linux 30 Jun '20

30 Jun '20

Hi all, For various reasons, I have to formally resign from the ArchLinux DevOps team. I will maintain my role as Forum Administrator, so I would appreciate keeping my @archlinux.org email address. Non-root access to the mail server would also be great to maintain if possible, for debugging issues for forum users (eg, missing password reset emails etc). If not, I'll just harass #-devops IRC when I need to get the logs for something ;D Cheers, ~Phil

2 1