November 2019 - Arch-devops - lists.archlinux.org

[arch-devops] cloud-images
by Christian Rebischke 02 May '20

02 May '20

Dear devops team, Recently I have pushed cloud-init back to [community][1], one Arch Linux user did some improvements on the cloud-init codebase as well and they are going to officially support arch linux with version 19.3[2]. Therefore I have added support for cloud-images to the arch-boxes project[3]. These images are qcow2 images and are expected to be installed in cloud-environments like Openstack. The only thing what is missing is a way to distribute the images and release them. My current plan has been that I build and sign them locally and then upload them to a web directory (This is the most secure way as possible right now). For this I need answers for a few questions: 1. Can I have a web-directory, where only I have access to? 2. Which key should I use to sign them? A new cloud-image signing key or my personal key? (I think latter should be enough). 3. I expect to build them monthly like the ISOs, how many images do you want to keep? My current assumption is that 1 year of image backup of cloud-images (qcow2) only would cost us around 30-50GB. 4. Do we want to mirror the images? Best greetings Chris [1] https://www.archlinux.org/packages/community/any/cloud-init/ [2] https://git.launchpad.net/cloud-init/commit/?id=5d5a32e039782ce3e1c0843082f… [3] https://github.com/archlinux/arch-boxes/commit/9d9f04f9618367e2af8d92bef2d5…

2 3

[arch-devops] soyuz (pkgbuild.com) server move to homedir.archlinux.org
by Sven-Hendrik Haase 01 Jan '20

01 Jan '20

Hi all, I set up a new Hetzner VPS that is going to become our new homedir/public_html server available to all TUs and Devs like soyuz was. We decided to decommission soyuz and put the public_html stuff on its own server for security reasons, to cut costs, and so that we can compartmentalize further. The server uses a Hetzner Cloud Volume which we can scale if we want but for now, it's 100GiB of zstd-compressed btrfs. If possible, we'd like to keep it at this size for the time being. You can host your own repos there if you want and that's fine. Please talk to us beforehand if you see yourself exhausting the volume with what you want to do. If you had stuff hosted in the public_html of soyuz, I'd ask you to transfer stuff over to the new box which is already reachable at the names pkgbuild.com (you'll get an SSH error because of this) and homedir.archlinux.org. Please check if you can throw away some old stuff/junk that you might not necessarily need on the new server. This new box is *not for building*. That's what dragon is for. Please only put documents/files onto the new pkgbuild.com that you actually want people to be able to access publicly. The box has no building facilities and you get no sudo'd commands. soyuz is going to be decommissioned no sooner than 2020-01-01 but do not expect it to hang around for long after that. We will not transfer any files for you from soyuz. In other words: All data not explicitly transferred by you will be destroyed after 2020-01-01. Please bring up any problems you might have with this new server on the arch-devops list or IRC (#archlinux-devops). Enjoy, Sven

2 3

[arch-devops] Meeting Tonight 20:30 UTC
by Jelle van der Waa 28 Nov '19

28 Nov '19

Hi All, Due to missing the meeting last week we will have a meeting tonight, the topics are: - progress regarding action points from the last meeting - homedir server review - enabling linger for everyone - rsync.net - zabbix problems/upgrading servers - soyuz migration - AUR ansiblification Notes from last meeting: https://wiki.archlinux.org/index.php/DeveloperWiki:DevopsMeetings/2019-11-27 -- Jelle van der Waa

1 0

[arch-devops] AUR unavailable due to Python 3.8 upgrade
by Jelle van der Waa 24 Nov '19

24 Nov '19

Hi all, I updated most of our servers last night which led to the unfortunate upgrade of Python to 3.8. I managed to python3 setup.py install the new AUR module which seemed to fix stuff but now we have issues. The aur is now in maintenance mode till this is figured out. This needs to become a priority on our agenda, getting the AUR into ansible and dealing with the big issue which is a vendored cgit which will also break time from time.... (apart from the security implications). Greetings, -- Jelle van der Waa

3 3

[arch-devops] Monitoring issues and questions
by Jelle van der Waa 23 Nov '19

23 Nov '19

Hi all, I've noticed some things which require attention and feedback. While setting up borg for homedir.archlinux.org I found out we backed up /var/tmp, I guess we should exclude this? bbs.archlinux.org is not fully resized to make use of its 40GB of storage. We need to resize btrfs to make full use of it. vostok still has backups for old machines, for example dom0.archlinux.org, since space is now not an issue (1.1T available) it's not a big issue. But I guess we should handle this more pro-active. We now have more server oriented playbooks, which however do not contain all roles such as borg. In my opinion we shouldn't rely on all-hosts-basic but add all required roles to a playbook. -- Jelle van der Waa

2 1

Re: [arch-devops] [arch-dev-public] soyuz (pkgbuild.com) server move to homedir.archlinux.org
by Sven-Hendrik Haase 18 Nov '19

18 Nov '19

On Mon, 18 Nov 2019 at 09:00, Konstantin Gizdov <arch(a)kge.pw> wrote: > Sounds good. One question - where do we move our email aliases? > Not quite sure what you mean. All email is still being handled by apollo unless I'm mistaken and soyuz had more duties than I thought. In the latter case, we should migrate that to apollo. Could anyone shed some light on this?

2 1

[arch-devops] Flyspray (bugs.archlinux.org) slow when saving/creating
by Jelle van der Waa 18 Nov '19

18 Nov '19

Hi all, I've noticed that the bugtracker is super slow when assigning bugs. It takes around ~6-8 seconds to save a task I'm wondering if this is due to one of or recent changes either the mail relay or fail2ban. Anyone have an idea how to figure out if it's due to the mail relay? Greetings, -- Jelle van der Waa

3 2

[arch-devops] DevOps meeting 2019-11-07 21:30 CET
by Jelle van der Waa 07 Nov '19

07 Nov '19

Hi all, Tommorow we have another devops meeting on mumble, join #archlinux-devops @ freenode for an invite. The topic for tommorow are: - archive mirroriing status - soyuz migration - cloud-images - PIA server status - rsync.net sponsorship - bbs migration status # Small topics - fstrim required on our vps-es? - backups, logrotate, man-db, updatedb, and sa-update all running at the same time seems suboptimal. Previous meeting notes are located here: https://wiki.archlinux.org/index.php/DeveloperWiki:DevopsMeetings/2019-10-24 -- Jelle van der Waa

1 1

[arch-devops] Test 2
by Phillip Smith 04 Nov '19

04 Nov '19

Testing mailman. Please ignore.

1 0

[arch-devops] Test
by Phillip Smith 04 Nov '19

04 Nov '19

Testing mailman. Please ignore.

1 0