- Arch-dev-public - lists.archlinux.org

[arch-dev-public] moving bridge-utils to [extra]
by Christian Hesse 02 Jul '20

02 Jul '20

Hello everybody, with the release of bridge-utils v1.7 the utility has been deprecated in favor of iproute2 which supports even more features. I would like to move it from [core] to [extra] if nobody raises concerns. https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/bridge-utils.git… -- main(a){char*c=/* Schoene Gruesse */"B?IJj;MEH" "CX:;",b;for(a/* Best regards my address: */=0;b=c[a++];) putchar(b-1/(/* Chris cc -ox -xc - && ./x */b/42*2-3)*42);}

1 0

[arch-dev-public] repos.archlinux.org has been migrated to a new server
by Jelle van der Waa 26 Jun '20

26 Jun '20

Hi all, repos.archlinux.org, svn.archlinux.org and rsync.archlinux.org are now on a new server which has plenty of diskspace for us to continue packaging for a while (16T free). Some important notes: The ssh hostkeys changed for repos.archlinux.org which means if you use archco/communityco this will fail without a useful error. So take care of your .ssh/known_hosts. The ssh host keys for gemini/repos.archlinux.org are: # gemini.archlinux.org 1024 SHA256:F1Corf6i2U72yub+CIzzGHLOMVKVnjALh1YHM8gBjxE root(a)gemini.archlinux.org (DSA) 256 SHA256:If51DkTftUpDAFz65totgDfTd/ddu/2w/RBZIHtY74U root(a)gemini.archlinux.org (ECDSA) 256 SHA256:wUrJYf9+zOpIEUQ3ndgarK0PjzPICa1frmu7mpL4e14 root(a)gemini.archlinux.org (ED25519) 3072 SHA256:Rltnuln3bjsHJwVbys/LnYCj7hO6srPoa15JP8QhmlQ root(a)gemini.archlinux.org (RSA) https://gitlab.archlinux.org/archlinux/infrastructure/-/blob/master/docs/ss… https://gitlab.archlinux.org/archlinux/infrastructure/-/blob/master/docs/ss… If anything unexpected happens please either mail arch-devops(a)lists.archlinux.org or #archlinux-devops on irc. Greetings, Jelle

3 5

[arch-dev-public] Thursday 25th June 14:00 UTC repos.archlinux.org migration
by Jelle van der Waa 23 Jun '20

23 Jun '20

Hi al, On Thursday 25th June at 14:00 UTC work will begin on migrating the following services to a new machine: * repos.archlinux.org * svn.archlinux.org * Arch mirror infrastructure This will impact all packaging and svn activities as those services are not available during the migration period. The services will be migrated to a new server as diskspace is running low on the current server, the migration is expected to take at least four hours. When the migration is successfully completed an email will be send that the services are available and packaging can resume. TLDR: No packaging can be done on the 25th of June starting at 14:00 UTC till the migration is done. Greetings, Jelle

1 0

Re: [arch-dev-public] SDR package naming
by Filipe Laíns 05 Jun '20

05 Jun '20

Hi all, I want to bring up two issues relating Kyle Keen's packages. I tried to contact him and got an initial reply but I have not heard from him since. It has been almost 4 months and I have sent him 3 different followup emails. Since it was not super urgent I let a pretty significant amount of time pass but I would like to have this resolved. Here are the two issues I seek resolution: 1) Rename libuhd to uhd UHD[1] is the upstream name. The software is pretty well known by that name. I have only ever seen it being addressed as libuhd when talking about the library it provides specifically, but even then UHD is the standard. It also provides some tools alongside the library. Following the Arch philosophy I think the package should be named "uhd". 2) Use the gr- prefix instead of gnuradio- for GNURadio[2] blocks The GNURadio ecosystem has a pretty well established prefix for GNURadio blocks, which is gr-. Similarly to the previous issue, I have never seen them being refereed as gnuradio-something. The upstream names are gr-something. The gnuradio- prefix might seem to make things more clear to new users, and even then I have my doubts, but it definitely makes things more confusing for actual users of the project. Previewing some counter-arguments to this, as I have discussed it with people previously. This is totally different than, for eg., the python ecosystem where we use the python- prefix. The gr- is given by the upstream and is the *de facto* naming scheme for the project. If you don't agree with the use of the gr- prefix, I think that variation of the name should *at least* be present in provides. These might seem like nitpicks, but I do very much care about them and I believe they should be corrected. You can find my attempt at communication with Kyle below. I am very conflicted about this, but I think before I end the email I should note that Kyle is packaging and he appears to have replied to other team member(s). I am not sure if I have done something wrong or behaved in an improper manner. I don't think I did, but if that's the case please let me know. [1] https://github.com/EttusResearch/uhd [2] https://github.com/gnuradio Cheers, Filipe Laíns On Thu, 2020-02-06 at 09:42 +0000, Filipe Laíns wrote: > On Wed, 2020-02-05 at 23:21 -0500, keenerd wrote: > > Sometimes my automatic email filing puts things in the wrong place, > > but searching across all folders finds no emails from you about this. > > How did you try to contact me before? > > > > UHD I am ambivalent on. However several other distros also call it > > "libuhd" and it is a library after all. > > > > Gnuradio vs gr I don't agree with. A project needs to be very notable > > for a two-letter naming convention. Python, for example, is > > sufficient. Internally the Gnuradio project uses "gr" because, well, > > they type it hundreds of times every day. Outside of their project, > > it is less helpful and more confusing. > > > > There are several people deeply involved with Gnuradio who are also > > Arch users. I more or less started packaging Gnuradio for them, and > > their input was instrumental in making our packages the way they are. > > They've never raised any of these issues. > > > > You did not provide any reasoning for why you think changes are necessary? > > > > -Kyle > > > > On 2/4/20, Filipe Laíns <lains(a)archlinux.org> wrote: > > > Hey Kyle, > > > > > > I think I have contacted out about this already but without much > > > success. I would like to adjust the names of some SDR packages. I > > > believe `libuhd` should be called `uhd` and packages that > > > provide GNU > > > Radio blocks should be named `gr-name` instead of `gnuradio-name` > > > (this > > > affects `gnuradio-{fcdproplus,iqbal,osmosdr}`). > > > > > > Can we adjust the naming? What do you think? > > > > > > Thank you, > > > Filipe Laíns > > > > > Sorry, my bad. > > Regarding uhd, that's what Debian does, not us. uhd is the upstream > name and we should honor it. If you want examples just look at [1], > from a quick browse I wasn't able to find any instances of packages > where we add the lib prefix. > > Regarding GNU Radio blocks, it more of a grey area so I understand and > respect your position. My reasoning is that the gr- prefix comes from > the upstream, and AFAIK everybody else on earth uses it. When I started > using Arch it was a bit confusing why the packages had different names, > especially for a distribution that tries to keep as close to the > upstream as possible. Do you think it would make sense to ask the GNU > Radio guys what they think? Either way, it's ultimately your call. > But if you don't want to rename it could we add the gr- prefixed name > to provides/conflicts? > > Just some other unrelated things, I am bringing a few more SDR packages > to Arch. I just pushed the necessary bits for Adalm-Pluto support, I am > working on srsLTE (actually, almost finished, just waiting for some > replies from the upstream) and I want to get some misc GNU Radio > blocks. If you want to co-maintain something just drop an email or > contact me on irc. Also, I currently own a LimeSDR, Adalm-Pluto, HackRF > and rtl-sdr, so if you need any testing please let me know :) > > [1] https://www.archlinux.org/packages/?q=library > > Thank you, > Filipe Laíns

3 5

[arch-dev-public] Disowning the Kea DHCP server package
by Baptiste Jonglez 05 Jun '20

05 Jun '20

Hi, I don't use Kea on Arch anymore, and I don't have the time and energy to maintain it in [community]. It's currently stuck at 1.5.0. Kea 1.6 came out 9 months ago, but it moved some files around and would require extensive testing to make sure everything still works as expected. If somebody is interested in maintaining it, please adopt the package, otherwise I will move it to the AUR this summer. FYI, it has a quite low usage according to pkgstats: https://pkgstats.archlinux.de/packages#query=kea Baptiste

2 2

[arch-dev-public] Transparency report May 2020
by Levente Polyak 03 Jun '20

03 Jun '20

# Transparency report May 2020 ## Staff ### TU addition We are happy to welcome Frederik aka freswa among the Trusted Users [0]. Some of you may already know him as one of our bug wranglers where he joined forces in February 2020. ## Packaging ### Go The Go package guidelines [1] have been overhauled and in conjunction the go-pie package has been removed [2]. The major difference is a new set of CGO/GOFLAGS that ensure all our distro flags are respected appropriately, leading to Go binaries with RELRO, PIE and fortify hardening. A to-do list to reflect these changes is pending. ### CMake The CMake package guidelines [3] have been created which describe some important bits to consider when packaging software using cmake. Most notably appropriate release type option that may otherwise have undesired effects, removal of non required RPATH usage as well as some convenience options to build in subdirectories without manually creating them. As CMake still does not respect CPPFLAGS itself, which results in fortify hardening being ignored, a temporary patch [4] has been added to circumvent this misbehavior, a to-do list to reflect these changes is pending. ## Reproducible builds You may have noticed a couple of large rebuilds that occurred recently. These fixed issues of non-reproducible file ordering with old versions of makepkg. This and other hard work by the team improving our tooling and fixing packaging issues has resulted in 96% of [core] being reproducible, and 90% of [extra]. You can see the status of which packages are reproducible here [5]. A full progress report can be found on the corresponding thread [6] on arch-dev-public. We have set up a fleet of three rebuilderd [7] runners to continuously test our distributed repository packages and populate our status page [8]. Some integration into archweb to indicate the current reproducibility status is planned. ## Infrastructure ### GitLab We're in the process of switching our Git hosting from our custom cgit instance to GitLab! We created https://gitlab.archlinux.org [9] and have started moving some projects. We'll continue moving projects onto GitLab and will then get rid of our cgit instance. Users can currently not collaborate with us on GitLab as we still need to get some monitoring in place to make sure we can keep a close eye on usages to ward off abuse. However, we're planning on doing this soon and then we'll open up GitLab to everyone. Finally you can collaborate on Arch like it's 2020! We're also going to use GitLab for other things such as bug tracker (instead of Flyspray), Kanban board (instead of Kanboard), and service desk (for GDPR requests and such). ### Single-sign-on/Keycloak Arch operates many different services - all of which with their own login systems and account databases. This is not ideal from a security and convenience standpoint. We'd like to enforce the same security requirements for all users while also allowing everyone to use the same account to log in to all of our services. It'll also finally allow you to use 2-factor authentication for all Arch services. We still have a long way to go here in terms of integrating all services via SAML/OIDC and figuring out how to let users continue using their old accounts. ### SVN to Git migration The git migration plans have been picked up again, and started working towards a proof-of-concept implementation [9]. This would allow packagers to avoid the SVN mono repository and manage each package as a separate git repository, and facilitate some modernization of our current tooling. More information about the plans and implementation will hit the [arch-dev-public] list in the upcoming week. ## Projects ### Archiso The archiso project has been moved [10] to Arch Linux' GitLab instance [11]. Furthermore over the past weeks smaller and larger fixes found their way into the repository. In the future all merge requests and releases will be handled via GitLab. CI integration and more elaborate test suites are still being developed to ensure a more robust setup for our monthly release images and any custom use cases. ### Pacman Initial support for parallel downloads landed in the pacman [12] code base, requiring large changes to the codebase. Many patches providing the finishing to this feature have been submitted. Once the code churn around this feature request has slowed, we will make a beta release for wider testing. Additionally, one obscure area of non-reproducility in packages was discovered through the Arch Linux reproducible builds effort, and subsequently fixed in makepkg. Furthermore, the move from using autotools to meson for the pacman build system was completed. Discussion is ongoing on moving the pacman codebase to the Arch Linux GitLab instance, with initial CI setup being added to the codebase. ## SPI For the upcoming SPI annual report 2019, a section about some of Arch Linux achievements has been assembled to represent our project. A link to the report will be distributed once it's available. [0] https://lists.archlinux.org/pipermail/aur-general/2020-May/035734.html [1] https://wiki.archlinux.org/index.php/Go_package_guidelines [2] https://lists.archlinux.org/pipermail/arch-dev-public/2020-March/029898.html [3] https://wiki.archlinux.org/index.php/CMake_package_guidelines [4] https://git.archlinux.org/svntogit/packages.git/commit/trunk/cmake-cppflags… [5] https://reproducible.archlinux.org/ [6] https://lists.archlinux.org/pipermail/arch-dev-public/2020-May/029981.html [7] https://github.com/kpcyrd/rebuilderd [8] https://reproducible.archlinux.org/ [9] https://github.com/Foxboron/dbscripts/commits/morten/git-support [10] https://lists.archlinux.org/pipermail/arch-releng/2020-May/003963.html [11] https://gitlab.archlinux.org/archlinux/archiso [12] https://git.archlinux.org/pacman.git/commit/?id=6a331af27fe6dc7842725d067fd…

2 1

[arch-dev-public] HEADS UP: Qt 5.15 in [testing]
by Antonio Rojas 03 Jun '20

03 Jun '20

The latest (and last) Qt 5 release is now in [testing]. This is the usual reminder that any package compiled against it must stay in [testing] until Qt itself moves.

4 6

[arch-dev-public] Reproducible builds progress report #3
by Allan McRae 30 May '20

30 May '20

Hi all, A quick updated on the progress of reproducible builds. You may have noticed a couple of large rebuilds that occurred recently. These fixed issues of non-reproducible file ordering with old versions of makepkg. This and other hard work by the team improving our tooling and fixing packaging issues has resulted in 96% of [core] being reproducible, and 90% of [extra]. You can see the status of which packages are reproducible here [1]. The remaining packages to fix in [core] are dnssec-anchors, linux, linux-lts, nss and perl. With the possible exception of perl, these are in the "hard" basket. There is plans on how to fix the kernel packages, but that will require some time to sort out. We would be happy for more people to help out so we can get [core] to 100% reproducible. We have investigated some of the packages in [extra] that fail to reproduce here [2]. Note that there are quite a few packages that currently "Failed to build from source" (FTBFS) - it would be very helpful for the reproducible builds team if their maintainers can help fix the packages. You can also use the CI of Arch packages run by Debian to get an overview what the issue is with these packages and see many other packages that are currently failing to build [3]. We also need help to investigate and fix the packages that fail to reproduce that we have not investigated as of yet. There are two easy to use tools to attempt to reproduce a package - "makerepropkg" from devtools and "repro" from the archlinux-repro package. Once these have rebuilt a package, you can use the "diffoscope" tool to look at the differences. Jump in the #archlinux-reproducible IRC channel if you want help interpreting the output, or you could just link to a copy of it in the wiki. [1] https://reproducible.archlinux.org/ [2] https://wiki.archlinux.org/index.php/Reproducible_Builds/Status [3] https://tests.reproducible-builds.org/archlinux/extra.html

3 2

[arch-dev-public] RFC: go-pie removal in favour of GOFLAGS
by Morten Linderud 18 May '20

18 May '20

# Introduction To enable PIE compilation, we have relied on a patched version of the go compiler which has been distributed as `go-pie` since around 2017. However, full RELRO support for go binaries has been a bit back and forth the past years. With some thing working, and other things don't. With the release of Go 1.11 there was support for a general `GOFLAGS` variable that lets us pass flags directly to the compiler. This email details what these flags should be going forward. # Flags Expected environment variables in PKGBUILDs: export CGO_LDFLAGS="$LDFLAGS" export GOFLAGS="-buildmode=pie -trimpath -mod=vendor -modcacherw" Explanation: * `CGO_LDFLAGS` passes the proper `LDFLAGS` to the linker. This should enable full RELRO when used in conjunction with `GOFLAGS`. * `-buildmode=pie` is the proper way to enable PIE and replaces the `go-pie` patch. * `-trimpath` this is to achieve reproducible builds and remove PWD from the binary. * `-modcacherw` modules are downloaded to `$GOPATH/pkg/mod` and by default have the permissions 444 for god knows why. If we want to run `makepkg -c` or `git clean` we won't have the correct permissions. This is probably not a big problem for repository packages, but it's a nice addition so they work as expected. Notice that `-mod=vendor` is also added to `GOFLAGS`. This will make sure we are using the vendored dependencies in the project. If they are not present, please ensure they are downloaded in the `prepare` function: prepare(){ cd $pkgname-$pkgver go mod vendor } If the project is *not* using Go 1.11 modules, missing `go.mod` and/or `go.sum` in the project root, then disable it with `export GO111MODULE=off` and continue with symlink hacks. Some upstreams override these values for strange reasons in their `Makefile` and build systems. You *need* to read over them and ensure this does not happen! # Pacman Clearly we shouldn't have to specify this in every PKGBUILD, so I have been playing with a `pacman` patch that passes all of the variables. However I have been struggling with debug support and figuring out that part of the flags, so nothing have been upstreamed yet. However this is only applicable to around 126 packages, so I guess it's fine? ¯\_(ツ)_/¯ # In conclusion... If there are no objections to the New Way Of Doing Things™, I'll be updating the package guidelines within the next week or two and drop the `go-pie` package containing the patch. For the sake of compatibility, the `go` package will contain a `replaces=('go-pie')`. I also expect people packaging go packages to follow the guidelines! -- Morten Linderud PGP: 9C02FF419FECBE16

6 22

[arch-dev-public] GitLab switchover and SSO migration
by Sven-Hendrik Haase 17 May '20

17 May '20

Hi everyone, As some of you know, we've been toying with two ideas for a while: Arch-wide centralized user management as well as using GitLab to consolidate some of our current services. The overall goal is manifold. In no particular order, the goals are to - make Arch more contributor friendly - provide more modern tools for ourselves - enable more automation - make Arch services more secure - make team management activities less error-prone and more streamlined These two topics (SSO + Gitlab) are a bit intertwined because we wanted to have SSO on GitLab before starting off with that so we'd have a properly validated user base to work with going forward. Also, GitLab seemed like a good first service for SSO due to it having good support for that. After looking at various solutions, we eventually settled on Keycloak since it seemed like a modern, well-maintained, and secure piece of software. It allows us to enable logins for services via OpenID Connect and SAML which is likely the best coverage we could hope for. It also allows us to connect other social login providers such as github.com and gitlab.com and it supports Recaptcha, 2FA, and WebAuthn out of the box. The idea is to eventually transition all our online service as well as SSH keys to Keycloak to ease on-/offboarding and make it less error-prone. As for GitLab, a few months back, we applied for a GitLab Ultimate license in their open-source program [0] and we received one [1]. It's an official program that many other open-source projects benefit from as well and we think it's safe to assume that it'll continue being a thing for the foreseeable future. We have to renew our license yearly. The current license we have has support for 1000 seats but we can likely get more seats if we need them. Our general path is: 1) Transition as many staff-only services to Keycloak as possible. We looked at our current services and put up a table on the wiki that shows support per service [2]. Some of the services that we operate are deprecated or have functionality that is also provided by GitLab. In our current understanding, this concerns Flyspray, Kanboard, and Patchwork. Of those, Kanboard and some of the Flyspray projects will be our first targets to transition to GitLab. We'll continue using Flyspray for the time being for package bugs but will discontinue its use for all non-packaging bugs. The reason for this is that how we manage our bugs for packages is somewhat intertwined with the svn2git migration which is yet to be done and might dictate a different repo structure than what we would come up with currently and we don't want to block on this. This was also discussed in a recent DevOps meeting [3]. 2) We'd like to get rid of our own cgit instance at git.archlinux.org and transition our git hosting into GitLab. AUR git access will stay as it is due to its special shell magic. 3) Eventually, after an internal testing phase of at least a few weeks, we'll want to open Keycloak and GitLab up for outside contributors. We know of the abuse potential and the potential moderation problems and have to make sure to set proper limits and set up monitoring before opening this up. 4) Connect remaining services like BBS and wiki to SSO. In 1) we only mentioned staff-only services because those are less problematic. However, in the future, we'd also like to enable our remaining services to connect to Keycloak. We tried hard to come up with a good source of users to import into Keycloak so that we could seed that database with a solid user base but sadly it appears that there is no trusted source of users that we can rely on. Potential candidates were the wiki, BBS, AUR but we ruled them all out in their current state as none of them have always had email verification and so we can't trust those emails to be the sole source of truth. In order to still allow users to keep their old contributions in cases where they can prove their identity via email, we'll build a new small web application that allows them to connect their new Keycloak identity to their other identities. For now, we seeded the Keycloak database with the only known-good source of trusted emails: Staff from Archweb. We'd like to make heavy use of GitLab CI for running automatic tests and release automation. We're aware that the implication of eventually allowing non-staff users to come in will result in untrusted code being run on our CI. This is fine by itself but security-wise would prevent us from creating trusted releases on the same CI runners. We currently have two sponsored bare-metal GitLab CI runners that we plan on using for running untrusted code. We'll get a new bare-metal box from Hetzner for trusted releases that will only run on hand-picked pipelines that only a select few of us can push to. Bare metal runners also allow us to test and build VM images and such which isn't usually possible on most VPS. On more goal we had is automatic github.com mirroring in some fashion. We looked at creating a two-way github.com <-> GitLab mirror but that setup can break easily in the case of force pushes and race conditions and also would have us looking at both places for pull requests. It seems simpler to us for the time being to have one-way mirroring from our GitLab to github.com only and then allow github.com users to easily collaborate on our GitLab via github.com social login. It's a little bit more hassle for the users than collaborating directly on github.com but it's a lot less hassle for us so it's perhaps the best compromise. These changes will affect all of you in some way but we'll try to make it as painless as possible. As we progress, we'll send concise emails with instructions on what to do. I'm sure that we missed a few details here and there but I'm still convinced that this is a change for the better across the board, even if we don't immediately get everything right. Sorry for the long mail. All of this has been a long time in the making and has been the subject of at least four hackathons with many hours spent in-between. It was a ton of work but I'm happy that we're finally at a stage where we can present something tangible with a plan of attack. Cheers, Sven and the DevOps team [0] https://gitlab.com/gitlab-com/marketing/community-relations/gitlab-oss [1] https://gitlab.com/gitlab-com/marketing/community-relations/gitlab-oss/-/co… [2] https://wiki.archlinux.org/index.php/DeveloperWiki:SSOMigration [3] https://wiki.archlinux.org/index.php/DeveloperWiki:DevopsMeetings/2020-05-06

3 5