May 2020 - Arch-dev-public - lists.archlinux.org

[arch-dev-public] Mkinitcpio replacement with Dracut
by Giancarlo Razzolini 19 Feb '21

19 Feb '21

Hi All, Recently, Dave announced his intention to step down as mkinitcpio maintainer. I have been handling a few things on it, made some small changes to our mkinitcpio-busybox [0], closed a few old bugs, while also made sure some of the other bug reports are still valid. [1] I have been testing a few patches to mkinitcpio for a while to fix those bugs, but last week Dave and I had a discussion on IRC and we agreed that we could spend better time contributing somewhere else than keeping working with mkinitcpio. Currently mkinitcpio cannot boot from NFS properly, mknitcpio-nfs-utils [2] uses very old code. Even though ipconfig works, it has several drawbacks compared to using more modern tools. Not to mention that we have two use cases for mkinitcpio, which are base and systemd enabled initramfs. Even though we use by default our base enabled initramfs, we have to support systemd related issues and we have to keep developing our systemd hooks. I have been looking into dracut for some time now, I copied some stuff from them on a few of my own scripts and they also have an actual test suite, that we currently can't use on Arch, but I plan to change that. With this in mind, I have packaged dracut for Arch and put it on Extra [3]. I have been using a dracut initramfs for a while now to boot my encrypted partition. Surprisingly, I only had to create a small patch to one of the dm-crypt module scripts [4] In this initial phase I want to ask as many of you to test this as a replacement to mkinitcpio in your setups, as many as possible, and in as many scenarios as possible. We will probably have to keep both packages on our repos for a long time, but once we are confident it's a good fit, we can replace mkinitcpio on our iso and base, so new installs get dracut by default. Please, drop in your comments as well. Regards, Giancarlo Razzolini [0] https://git.archlinux.org/svntogit/packages.git/commit/trunk/PKGBUILD?h=pac… [1] https://bugs.archlinux.org/index.php?string=mkinitcpio&project=0 [2] https://git.archlinux.org/mkinitcpio-nfs-utils.git/ [3] https://www.archlinux.org/packages/extra/x86_64/dracut/ [4] https://git.archlinux.org/svntogit/packages.git/tree/trunk/0001-90crypt-Cha…

4 7

[arch-dev-public] Abandoning all my vim-* plugin packages
by Sven-Hendrik Haase 13 Jul '20

13 Jul '20

I've started using a vim plugin manager and don't use my packages anymore. They need a new home. Please show these packages some love: * vim-a * vim-colorsamplerpack * vim-doxygentoolkit * vim-guicolorscheme * vim-minibufexpl * vim-omnicppcomplete * vim-project * vim-taglist * vim-vcscommand * vim-workspace The packages are pretty low maintenance but I'd prefer if someone who actually uses them maintained them. Anything that's not taken within 2 weeks will be dropped to AUR.

1 1

[arch-dev-public] Removing dependency on fontconfig/xorg-mkfontscale of font packages
by Frederik Schwan 02 Jul '20

02 Jul '20

We received a Feature Request today to remove fontconfig and xorg-mkfontscale dependencies from our font packages according to our own font packaging guidelines [0]. I discussed with Eli on #archlinux-bugs and we think it's a no-brainer but before creating a TODO we'd like to ask for your opinions first. Thank you [0] https://bugs.archlinux.org/task/66012

7 18

[arch-dev-public] HEADS UP: Qt 5.15 in [testing]
by Antonio Rojas 03 Jun '20

03 Jun '20

The latest (and last) Qt 5 release is now in [testing]. This is the usual reminder that any package compiled against it must stay in [testing] until Qt itself moves.

4 6

[arch-dev-public] Reproducible builds progress report #3
by Allan McRae 30 May '20

30 May '20

Hi all, A quick updated on the progress of reproducible builds. You may have noticed a couple of large rebuilds that occurred recently. These fixed issues of non-reproducible file ordering with old versions of makepkg. This and other hard work by the team improving our tooling and fixing packaging issues has resulted in 96% of [core] being reproducible, and 90% of [extra]. You can see the status of which packages are reproducible here [1]. The remaining packages to fix in [core] are dnssec-anchors, linux, linux-lts, nss and perl. With the possible exception of perl, these are in the "hard" basket. There is plans on how to fix the kernel packages, but that will require some time to sort out. We would be happy for more people to help out so we can get [core] to 100% reproducible. We have investigated some of the packages in [extra] that fail to reproduce here [2]. Note that there are quite a few packages that currently "Failed to build from source" (FTBFS) - it would be very helpful for the reproducible builds team if their maintainers can help fix the packages. You can also use the CI of Arch packages run by Debian to get an overview what the issue is with these packages and see many other packages that are currently failing to build [3]. We also need help to investigate and fix the packages that fail to reproduce that we have not investigated as of yet. There are two easy to use tools to attempt to reproduce a package - "makerepropkg" from devtools and "repro" from the archlinux-repro package. Once these have rebuilt a package, you can use the "diffoscope" tool to look at the differences. Jump in the #archlinux-reproducible IRC channel if you want help interpreting the output, or you could just link to a copy of it in the wiki. [1] https://reproducible.archlinux.org/ [2] https://wiki.archlinux.org/index.php/Reproducible_Builds/Status [3] https://tests.reproducible-builds.org/archlinux/extra.html

3 2

[arch-dev-public] RFC: go-pie removal in favour of GOFLAGS
by Morten Linderud 18 May '20

18 May '20

# Introduction To enable PIE compilation, we have relied on a patched version of the go compiler which has been distributed as `go-pie` since around 2017. However, full RELRO support for go binaries has been a bit back and forth the past years. With some thing working, and other things don't. With the release of Go 1.11 there was support for a general `GOFLAGS` variable that lets us pass flags directly to the compiler. This email details what these flags should be going forward. # Flags Expected environment variables in PKGBUILDs: export CGO_LDFLAGS="$LDFLAGS" export GOFLAGS="-buildmode=pie -trimpath -mod=vendor -modcacherw" Explanation: * `CGO_LDFLAGS` passes the proper `LDFLAGS` to the linker. This should enable full RELRO when used in conjunction with `GOFLAGS`. * `-buildmode=pie` is the proper way to enable PIE and replaces the `go-pie` patch. * `-trimpath` this is to achieve reproducible builds and remove PWD from the binary. * `-modcacherw` modules are downloaded to `$GOPATH/pkg/mod` and by default have the permissions 444 for god knows why. If we want to run `makepkg -c` or `git clean` we won't have the correct permissions. This is probably not a big problem for repository packages, but it's a nice addition so they work as expected. Notice that `-mod=vendor` is also added to `GOFLAGS`. This will make sure we are using the vendored dependencies in the project. If they are not present, please ensure they are downloaded in the `prepare` function: prepare(){ cd $pkgname-$pkgver go mod vendor } If the project is *not* using Go 1.11 modules, missing `go.mod` and/or `go.sum` in the project root, then disable it with `export GO111MODULE=off` and continue with symlink hacks. Some upstreams override these values for strange reasons in their `Makefile` and build systems. You *need* to read over them and ensure this does not happen! # Pacman Clearly we shouldn't have to specify this in every PKGBUILD, so I have been playing with a `pacman` patch that passes all of the variables. However I have been struggling with debug support and figuring out that part of the flags, so nothing have been upstreamed yet. However this is only applicable to around 126 packages, so I guess it's fine? ¯\_(ツ)_/¯ # In conclusion... If there are no objections to the New Way Of Doing Things™, I'll be updating the package guidelines within the next week or two and drop the `go-pie` package containing the patch. For the sake of compatibility, the `go` package will contain a `replaces=('go-pie')`. I also expect people packaging go packages to follow the guidelines! -- Morten Linderud PGP: 9C02FF419FECBE16

6 22

[arch-dev-public] GitLab switchover and SSO migration
by Sven-Hendrik Haase 17 May '20

17 May '20

Hi everyone, As some of you know, we've been toying with two ideas for a while: Arch-wide centralized user management as well as using GitLab to consolidate some of our current services. The overall goal is manifold. In no particular order, the goals are to - make Arch more contributor friendly - provide more modern tools for ourselves - enable more automation - make Arch services more secure - make team management activities less error-prone and more streamlined These two topics (SSO + Gitlab) are a bit intertwined because we wanted to have SSO on GitLab before starting off with that so we'd have a properly validated user base to work with going forward. Also, GitLab seemed like a good first service for SSO due to it having good support for that. After looking at various solutions, we eventually settled on Keycloak since it seemed like a modern, well-maintained, and secure piece of software. It allows us to enable logins for services via OpenID Connect and SAML which is likely the best coverage we could hope for. It also allows us to connect other social login providers such as github.com and gitlab.com and it supports Recaptcha, 2FA, and WebAuthn out of the box. The idea is to eventually transition all our online service as well as SSH keys to Keycloak to ease on-/offboarding and make it less error-prone. As for GitLab, a few months back, we applied for a GitLab Ultimate license in their open-source program [0] and we received one [1]. It's an official program that many other open-source projects benefit from as well and we think it's safe to assume that it'll continue being a thing for the foreseeable future. We have to renew our license yearly. The current license we have has support for 1000 seats but we can likely get more seats if we need them. Our general path is: 1) Transition as many staff-only services to Keycloak as possible. We looked at our current services and put up a table on the wiki that shows support per service [2]. Some of the services that we operate are deprecated or have functionality that is also provided by GitLab. In our current understanding, this concerns Flyspray, Kanboard, and Patchwork. Of those, Kanboard and some of the Flyspray projects will be our first targets to transition to GitLab. We'll continue using Flyspray for the time being for package bugs but will discontinue its use for all non-packaging bugs. The reason for this is that how we manage our bugs for packages is somewhat intertwined with the svn2git migration which is yet to be done and might dictate a different repo structure than what we would come up with currently and we don't want to block on this. This was also discussed in a recent DevOps meeting [3]. 2) We'd like to get rid of our own cgit instance at git.archlinux.org and transition our git hosting into GitLab. AUR git access will stay as it is due to its special shell magic. 3) Eventually, after an internal testing phase of at least a few weeks, we'll want to open Keycloak and GitLab up for outside contributors. We know of the abuse potential and the potential moderation problems and have to make sure to set proper limits and set up monitoring before opening this up. 4) Connect remaining services like BBS and wiki to SSO. In 1) we only mentioned staff-only services because those are less problematic. However, in the future, we'd also like to enable our remaining services to connect to Keycloak. We tried hard to come up with a good source of users to import into Keycloak so that we could seed that database with a solid user base but sadly it appears that there is no trusted source of users that we can rely on. Potential candidates were the wiki, BBS, AUR but we ruled them all out in their current state as none of them have always had email verification and so we can't trust those emails to be the sole source of truth. In order to still allow users to keep their old contributions in cases where they can prove their identity via email, we'll build a new small web application that allows them to connect their new Keycloak identity to their other identities. For now, we seeded the Keycloak database with the only known-good source of trusted emails: Staff from Archweb. We'd like to make heavy use of GitLab CI for running automatic tests and release automation. We're aware that the implication of eventually allowing non-staff users to come in will result in untrusted code being run on our CI. This is fine by itself but security-wise would prevent us from creating trusted releases on the same CI runners. We currently have two sponsored bare-metal GitLab CI runners that we plan on using for running untrusted code. We'll get a new bare-metal box from Hetzner for trusted releases that will only run on hand-picked pipelines that only a select few of us can push to. Bare metal runners also allow us to test and build VM images and such which isn't usually possible on most VPS. On more goal we had is automatic github.com mirroring in some fashion. We looked at creating a two-way github.com <-> GitLab mirror but that setup can break easily in the case of force pushes and race conditions and also would have us looking at both places for pull requests. It seems simpler to us for the time being to have one-way mirroring from our GitLab to github.com only and then allow github.com users to easily collaborate on our GitLab via github.com social login. It's a little bit more hassle for the users than collaborating directly on github.com but it's a lot less hassle for us so it's perhaps the best compromise. These changes will affect all of you in some way but we'll try to make it as painless as possible. As we progress, we'll send concise emails with instructions on what to do. I'm sure that we missed a few details here and there but I'm still convinced that this is a change for the better across the board, even if we don't immediately get everything right. Sorry for the long mail. All of this has been a long time in the making and has been the subject of at least four hackathons with many hours spent in-between. It was a ton of work but I'm happy that we're finally at a stage where we can present something tangible with a plan of attack. Cheers, Sven and the DevOps team [0] https://gitlab.com/gitlab-com/marketing/community-relations/gitlab-oss [1] https://gitlab.com/gitlab-com/marketing/community-relations/gitlab-oss/-/co… [2] https://wiki.archlinux.org/index.php/DeveloperWiki:SSOMigration [3] https://wiki.archlinux.org/index.php/DeveloperWiki:DevopsMeetings/2020-05-06

3 5

[arch-dev-public] Reproducible builds progress report #2 and package rebuilders
by Morten Linderud 05 May '20

05 May '20

Yo! Most should be familiar with the reproducible builds efforts going on in Arch Linux. The goal is to figure out how to make our packages reproducible, which can let users verify that our packages are a product of the PKGBUILD we upload and the source we claim it uses [1]. Last status update was in November [2]! Allan wrote about our attempts at manually reproducing core packages to find mistakes in them. This went fairly well and we managed to reproduce a great deal of packages [3]. The progress since then has been great. Jelle went to Marrakesh for the annual Reproducible Builds summit [4]. Improvements to the tooling have also been made. Most notably kpcyrd has written rebuilderd which was announced on the reproducible builds mailing list last week [5]. rebuilderd aims to be a general package rebuilder, supporting multiple distros with Arch being the first supported one. Rebuilderd allows anyone to easily create package rebuilders to reproduce distributed packages [6]. It currently utilizes `repro` for the reproduction itself [7]. As of writing this we have managed to reproduce 86%-90% of the `[core]` repository across 2-3 rebuilders! One of the rebuilders currently running is our own rebuilder [8]! The current setup runs with 3 worker boxes: * repro1.pkgbuild.com - Arch * repro2.pkgbuild.com - Arch * repro3.pkgbuild.com - Debian 10 One can also find a list of rebuilders currently running on the wiki [9]. A usecase for these rebuilders is to check the packages on your system is currently verified with one or more rebuilders. kpcyrd wrote ismyarchverifiedyet to check this [10]. It should be noted that everything is very much a work in progress. Just because a package is listed as bad doesn't mean it's unreproducible. It might be tooling bugs or other issues. However, if you want to take a look at it you can do so with `repro`, or `makerepropkg` in devtools[11]. Cheers from the Reproducible Builds Team! Sources: [1]: https://reproducible-builds.org/ [2]: https://lists.archlinux.org/pipermail/arch-dev-public/2019-November/029721.… [3]: https://wiki.archlinux.org/index.php/DeveloperWiki:ReproduciblePackages [4]: https://reproducible-builds.org/events/Marrakesh2019/ [5]: https://lists.reproducible-builds.org/pipermail/rb-general/2020-April/00190… [6]: https://github.com/kpcyrd/rebuilderd [7]: https://github.com/archlinux/archlinux-repro [8]: https://reproducible.archlinux.org/ [9]: https://wiki.archlinux.org/index.php/Package_rebuilders [10]: https://github.com/kpcyrd/ismyarchverifiedyet [11]: https://git.archlinux.org/devtools.git/tree/makerepropkg.in -- Morten Linderud PGP: 9C02FF419FECBE16

1 0

[arch-dev-public] Spring cleaning (moving orphaned packages from [community] to AUR)
by Alexander F Rødseth 01 May '20

01 May '20

Hello, It's time for the semi-yearly package cleanup of [community] packages, as is tradition. I have gathered a list of "unneeded orphans" in [community] (packages that currently has no maintainer, and no other package depends on them) from the excellent list on our web page. TUs and Devs, please adopt, if you're interested: adapta-kde - Adapta theme for KDE Plasma 5 agg - High Quality Rendering Engine for C++ aide - A file integrity checker and intrusion detection program. arch - A modern and remarkable revision control system. arc-kde - Arc theme for KDE Plasma 5 aspell-it - Italian dictionary for aspell aspell-ru - Russian dictionary for aspell bar - A script for showing progress bars. bomberclone - Clone of the game Atomic Bomberman cellwriter - A grid-entry natural handwriting input panel. curseofwar - Fast-paced RTS/Action game with ncurses interface dcfldd - DCFL (DoD Computer Forensics Lab) dd replacement with hashing dumb-a4 - IT, XM, S3M and MOD player library (for Allegro 4) easystroke - Use mouse gestures to initiate commands and hotkeys. eclipse-cpp - Highly extensible IDE for C++ eclipse-java - Highly extensible IDE for Java eclipse-javascript - Highly extensible IDE for JavaScript eclipse-jee - Highly extensible IDE for JEE eclipse-php - Highly extensible IDE for PHP eclipse-rust - Highly extensible IDE for Rust esmtp - An easy SMTP forwarder. gimp-dbp - David's batch processor for the GIMP gimp-plugin-fblur - Makes out of focus with luminosity and depth gimp-plugin-lqr - Plugin for The GIMP providing Liquid Rescale gimp-plugin-wavelet-denoise - Tool to reduce noise in each channel of an image separately gimp-refocus - A sharpen plugin for gimp using FIR Wiener filtering glide - Dependency management and vendoring for Go projects glitz - OpenGL image compositing library gnunet-gtk - A frontend for GNUnet gtk-theme-switch2 - Gtk2 theme switcher gvmd - greenbone-vulnerability-manager gvm-tools - greenbone-vulnerability-manager tools ifuse - A fuse filesystem to access the contents of an iPhone or iPod Touch iverilog - Icarus Verilog compiler and simulation tool javasqlite - Java wrapper including a basic JDBC driver for the SQLite 2/3 database engine matrix-appservice-irc - Node.js IRC bridge for Matrix minbif - An IRC gateway to IM networks that uses libpurple. mod_wsgi - Python WSGI adapter module for Apache monica - Monitor calibration tool mythes-pl - Polish thesaurus ode - High performance library for simulating rigid body dynamics openvas - Vulnerability scanning Daemon pinta - Drawing/editing program modeled after Paint.NET. It's goal is to provide a simplified alternative to GIMP for casual users processing - Programming environment for creating images, animations and interactions projectm-pulseaudio - Music visualizer which uses 3D accelerated iterative image based rendering (pulseaudio) projectm-sdl - Music visualizer which uses 3D accelerated iterative image based rendering (sdl) python2-pyxmpp - Python XMPP and Jabber implementation based on libxml2 python-pyinsane - Python library to access and use image scanners qtspim - New user interface for spim, a MIPS simulator. rdiff-backup - A utility for local/remote mirroring and incremental backups. remacs - Emacs with parts of it written in Rust steghide - Embeds a message in a file by replacing some of the least significant bits surf - A simple web browser based on WebKit/GTK+. tabbed - Simple generic tabbed fronted to xembed aware applications. transset-df - A patched version of X.Org's transset with added functionality. tuxpaint-config - Tux Paint configuration tool uncrustify - A source code beautifier vim-jellybeans - Colorful, dark color scheme, inspired by ir_black and twilight vim-ultisnips - TextMate-style snippets for Vim. wren - Small, fast, class-based concurrent scripting language xautomation - Controls X from the command line and does "visual scraping". xbindkeys - Launch shell commands with your keyboard or your mouse under X zapcc - C++ compiler based on Clang, but significantly faster zeal - An offline API documentation browser I'll wait two weeks before adding them to AUR, running both `db-remove` and `svn rm` on the unmaintained packages. PS. If unmaintained orphans in [core] could be moved to [extra], and unmaintained orphans in [extra] could be moved to [community], that could be a nice "trickle down" effect and would be an appreciated cleanup as well, but that is just my opinion. To quote our newly elected leader, in connection with the previous spring cleaning: > Cleanup is always good > > - Levente Polyak -- Best regards, Alexander F. Rødseth / xyproto

6 6