March 2024 - Arch-dev-public - lists.archlinux.org

zlib-ng
by Jelle van der Waa 11 Nov '24

11 Nov '24

Hey All, I noticed Fedora is discussing switching from zlib to zlib-ng. zlib-ng is a fork of zlib, which seems to offer ~ 40% better performance and a new API. We have about ~ 1000 packages depending on zlib, zlib-ng does offer a compat option to keep zlib API compatibility. [1] # Proposal From a discussion in #archlinux-staff. It was determined that a possible plan would be to introduce zlib-ng as package in [extra] with the modern API enabled, we can then migrate packages over that are compatible to the new API. A zlib-ng-compat package can be provided, as proposed by Fedora, that activates the zlib API compatibility mode and directly provides libz.so. This package can provide/conflict the original zlib package, and potentially be target to replace zlib all together. In short: zlib-ng would provide libz-ng.so zlib-ng-compat would provide libz.so # Packages which might be able to use zlib-ng Alpine offers a package which has 6 required packages.[2] Debian's codesearch suggests a few packages seem to mention a possible zlib-ng compatibility. [3] These notes where written down last Thursday, in the meantime, Anthraxx uploaded zlib-ng to [extra], so we can start switching packages which use the modern API over. Replacing zlib with zlib-ng-compact is something we need to discuss, I've overheard so far that some test suites fail because they check the compressed test artifact's hash which with zlib-ng can be different. [1] https://lists.fedoraproject.org/archives/list/devel%40lists.fedoraproject.o… [2] https://pkgs.alpinelinux.org/package/edge/community/x86/zlib-ng [3] https://codesearch.debian.net/search?q=zlib-ng&literal=1&perpkg=1 Greetings, Jelle van der Waa & Anthraxx

3 3

Python 3.12
by Jelle van der Waa 21 Apr '24

21 Apr '24

Hi all, We should get started on Python 3.12, 3.13 alpha is already getting released! To be 100% honest, I am not super looking forward to it as it's a ton of work due to our lack of automated tooling. In basics a plan would be to first tackle low hanging fruit to-do's: * https://archlinux.org/todo/pytest-runner-removal/ * https://archlinux.org/todo/imp-module-removal-in-python-312/ * https://archlinux.org/todo/use-pyprojecttoml-for-building-if-supported/ * drop unused python modules in our repo https://archlinux.org/devel/reports/unneeded-orphans/ We've finished python-bootstrap, and can bootstrap the required build packages (python-setuptools, build, etc.). And figuring out the rebuild order. (Big thanks to david, for the help here!) [1] [2] So soon we can start the rebuild which will probably take a while to complete as we have no good automated tooling yet for mass rebuilds. This rebuild will likely block [staging] for a good 2 or more weeks, so when would be a good time to schedule this? I expect two things to break our package builds in 3.12: * The removal of distutils see the snippet below, adding python-setuptools to makedepends should resolve this. For example: GISCAN ZBar-1.0.gir Traceback (most recent call last): File "/usr/bin/g-ir-scanner", line 99, in <module> from giscanner.utils import dll_dirs File "/usr/lib/gobject-introspection/giscanner/utils.py", line 385, in <module> import distutils.cygwinccompiler ModuleNotFoundError: No module named 'distutils' * The removal of the imp module, the todo list above has been open for a while, but not all packages have been fixed. * The usual failing tests, missing sources you know it :) [1] https://gitlab.archlinux.org/archlinux/python-bootstrap/-/merge_requests/3 [2] https://notes.vdwaa.nl/arch/python.html Greetings, Jelle

2 6

Proposal to increase the default vm.max_map_count value
by Robin Candau 01 Apr '24

01 Apr '24

Hi everyone, I'm writing this mail as proposal to increase the default `vm.max_map_count` [1] value in Arch Linux. The default `vm.max_map_count` 65530 value is making some Windows games crash (or even prevent them to start at all) while being played through Wine or Steam Play (Proton). Examples are (but not limited to) Star Citizen, THE FINALS, Hogwarts Legacy, DayZ or Counter-Strike 2 (see [2] for a list of bug reported to Valve for these games on that matter). In that regard, SteamOS is shipping an increased `vm.max_map_count` default value to address the above issue and numerous distributions have since then decided to implement that change on their side as well, such as Fedora [3], NixOS [4] and, more recently, Ubuntu [5]. While I'm aware that Arch is a user centric/DIY distro and everyone can set their own `vm.max_map_count` value fairly easily, shipping an increased value on our side would be a beneficial change to make in my opinion as it would result in a smoother gaming experience out of the box for our users without representing any downside/side effect (as far as I'm aware). In terms of implementation, the change basically consists of shipping the following sysctl file: ``` # Increase the number of virtual memory areas that one process may request vm.max_map_count=1048576 ``` For reference, Fedora added it to their systemd package [6], while Ubuntu handled it at the procps level [7]. I personally don't have a strong opinion on the implementation way/place. To sum up, I think this would be a reasonable and positive change to make to our distro. I'd be happy to read your thoughts (or eventual objection)! 😊 [1] https://docs.kernel.org/admin-guide/sysctl/vm.html#max-map-count [2] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2057792/comments/5 [3] https://fedoraproject.org/wiki/Changes/IncreaseVmMaxMapCount [4] https://github.com/NixOS/nixpkgs/pull/238459 [5] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2057792 [6] https://src.fedoraproject.org/rpms/systemd/blob/f39/f/10-map-count.conf [7] https://git.launchpad.net/ubuntu/+source/procps/commit/?h=applied/2%254.0.4… -- Regards, Robin Candau / Antiz

3 4

Mirror hardening - complementing RFC 29
by pitastrudl 28 Mar '24

28 Mar '24

Hi, A new RFC (request for comment) has been opened here: https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/35 Please visit the above link for discussion. Summary: Before this proposal, there were a lack of security definitions and measures on mirrors within the Arch Linux community. This RFC therefore outlines the requirements, definitions and guidelines surrounding not only hardening but also maintaining proper security of a mirror. Best regards, Arun

1 0

RFC Final Comment Period: Allow all staff to create RFCs
by Kristian Klausen 23 Mar '24

23 Mar '24

An RFC has now entered Final Comment Period. In 14 days, discussion will end and the proposal will either be accepted, rejected or withdrawn: https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/31 Please visit the above link for discussion. Summary: Grant all Arch Linux staff members, not limited to those in packaging roles, the privilege to initiate RFCs directly, aligning with the broad range of topics these documents encompass.

1 0

Arch Linux minimal container userland 100% reproducible - now what?
by kpcyrd 21 Mar '24

21 Mar '24

hello, in last week's email to the reproducible-builds email list[1] about reproducible Arch Linux I mentioned there's only one unreproducible package left in docker.io/library/archlinux. [1]: https://lists.reproducible-builds.org/pipermail/rb-general/2024-March/00329… Due to amazing work by dvzrv and Foxboron this package is now also reproducible! INFO arch_repro_status > All packages are reproducible! INFO arch_repro_status > Your system is 100.00% reproducible. To try for yourself use: podman run --rm -t archlinux sh -c 'pacman -Suy arch-repro-status --noconfirm && arch-repro-status' However: Where do we go from here? It would be cool if the OCI container image itself could also be reproduced (bit-for-bit), but I'm not sure if there's any prior work (specifically for images listed as 'official' on Docker Hub)? Specifically what I mean - given a line like this: FROM archlinux@sha256:2dbd72d1e5510e047db7f441bf9069e9c53391b87e04e5bee3f379cd03cec060 I want to reproduce the artifact(s) that are pulled in by this, with the packages our Arch Linux rebuilders have reproduced from source code. From what I understand this hash points to a json manifest that is not contained in the container image itself and was generated by the registry (should we archive them?), and this manifest then points to the sha256 of the tar containing the filesystem (I'm possibly missing an indirection here). Hopefully one of the many SBOM formats can help with this. :P I know the container image is built from these two repositories but I don't have any in-depth knowledge: - https://github.com/docker-library/official-images/blob/master/library/archl… - https://gitlab.archlinux.org/archlinux/archlinux-docker The only work towards reproducible container images I'm aware of is by Akihiro Suda: https://github.com/reproducible-containers/repro-get#are-container-images-b… I'm suspecting the current scripts used by Arch Linux would still be prone to mirror changes[2] though, meaning new package uploads would end up in our reproduced artifacts (causing mismatches) and the container image could only be reproduced for a short amount of time. [2]: https://gitlab.archlinux.org/archlinux/archlinux-docker/-/blob/98cd79111dd5… I'm also not sure if there's a missing puzzle piece with reproducible containers in regards to this manifest json that is generated by the registry. The image digest being unpredictable has also been mentioned in a cosign github issue[3]. [3]: https://github.com/sigstore/cosign/issues/2516 Input much appreciated! ## Caveats Probably worth mentioning, at the time of writing there's no consensus across multiple orgs yet, the https://reproducible.archlinux.org instance reports this status, two other rebuilders don't report the full 100% yet. $ arch-repro-status -r https://reproducible.crypto-lab.ch [...] INFO arch_repro_status > 3/118 packages are not reproducible. INFO arch_repro_status > Your system is 97.46% reproducible. $ arch-repro-status -r https://wolfpit.net/rebuild [...] INFO arch_repro_status > 3/118 packages are not reproducible. INFO arch_repro_status > Your system is 97.46% reproducible. The packages in question are part of this rebuild todo (specifically gcc-libs, glibc, ncurses): https://archlinux.org/todo/rebuild-core-with-reproducible-pacman/ Meaning there's currently some luck involved for these 3 packages, e.g. using btrfs currently increases your chances to get an exact match (after a few tries). We're obviously trying to get rid of this caveat though. --- If you appreciate this flavor of supply-chain security you may be interested in repro-env[4] that I'm currently trying to land[5] in ubuntu 24.04 LTS, but is blocked by Debian's libnettle[6]. [4]: https://github.com/kpcyrd/repro-env [5]: https://tracker.debian.org/pkg/rust-repro-env [6]: https://tracker.debian.org/pkg/nettle cheers, kpcyrd

3 2

Revising Go packaging guidelines (2024)
by kpcyrd 21 Mar '24

21 Mar '24

hello, the Arch Linux Go packaging guidelines[0] currently suggest this for building software written in Go: > export CGO_CPPFLAGS="${CPPFLAGS}" > export CGO_CFLAGS="${CFLAGS}" > export CGO_CXXFLAGS="${CXXFLAGS}" > export CGO_LDFLAGS="${LDFLAGS}" > export GOFLAGS="-buildmode=pie -trimpath -ldflags=-linkmode=external -mod=readonly -modcacherw" > > # or alternatively you can define some of these flags from the CLI options > go build \ > -trimpath \ > -buildmode=pie \ > -mod=readonly \ > -modcacherw \ > -ldflags "-linkmode external -extldflags \"${LDFLAGS}\"" \ > . [0]: https://wiki.archlinux.org/title/Go_package_guidelines There's also section "2.2.2 Supporting debug packages" which contradicts the previous section. ## Suggestion I'd like to propose changing this to something like: > prepare() { > cd ${pkgname}-${pkgver} > go mod download > } > > build() { > cd ${pkgname}-${pkgver} > go build . > } > > package() { > cd ${pkgname}-${pkgver} > install -Dm0755 -t "${pkgdir}/usr/bin/" "${pkgname}" > } Plus any extra flags (for example for debug packages to work, if any are needed at all - there's a note about this towards the end of this essay). Instead of suggesting two (three) different approaches, the packaging guidelines should endorse argv over environment variables (how to set them with environment variables can still be documented in a later section of the guideline). The reason I'm suggesting this: ## Passing flags over argv Setting these options through environment variables is for cases when `go` isn't executed directly by our PKGBUILD, but instead by some indirection like a Makefile. Makefiles are more popular in Go projects than they are in e.g. Rust projects, but presumably because for many years (10+) there was no analogous to `env!("CARGO_PKG_VERSION")` to embed a version string into the binary for --version output. Since almost exactly two years ago, in Go 1.18 released March 2022, there's now `runtime/debug.ReadBuildInfo` which gives you a reference to an embedded struct that contains a version string[1]. [1]: https://pkg.go.dev/runtime/debug#ReadBuildInfo Hopefully more Go projects are going to favor this in the future over custom build scripts and passing `-X main.Version=...` with output from `git describe --tags`. If this catches on, people might think of `go build` more like they currently think of `cargo build` and I think the Go package guideline should encourage this. Not just for Go but for software in general, using `git describe` should be considered an anti-pattern. In the way the Arch Linux build system currently works, only the checked out files are authenticated and pinned by sha256sums= and friends, but `git describe` explicitly works on unauthenticated git objects in `.git/`. I've published a writeup on how to take advantage of this on the reproducible builds list in September 2023[2]. [2]: https://lists.reproducible-builds.org/pipermail/rb-general/2023-September/0… ## Using the Go native Linker Binaries built by the Go compiler have been reproducible for a long time, however Go binaries built by Arch Linux usually aren't. This has led to some confusion, with bug reports to upstreams of Go software because it wasn't obvious that the package not being reproducible in this case isn't a problem in upstream's source code, but the way Arch Linux compiles it. Arch Linux generally prefers binaries with hardening flags enabled[3] and therefore explicitly opts into a non-default linker (Cgo), but it seems to be the unpopular choice. Cgo seems to have less support than normal Go on Linux (evidently, since Cgo has been known to have reproducible build issues for years now). Other Linux distributions seem to stick to normal Go too, Nix has Cgo off by default[4] and searching through nixpkgs for "buildGoModule" yields about 2k results, searching for "CGO_ENABLED" has 105 results, with more than half being "CGO_ENABLED=0" though. Alpine Linux only sets GOFLAGS globally[5] (which Arch Linux currently doesn't), searching for `rg -l --multiline '\tgo.*build'` in aports matches 356 files, `rg -l CGO_ENABLED=0` matches 23 files and `rg -l CGO_ENABLED=1` matches 18 files. [3]: https://github.com/jvoisin/compiler-flags-distro [4]: https://github.com/NixOS/nixpkgs/blob/b081342f1c16e4cbe4f40f139bbdda1475ea3… [5]: https://git.alpinelinux.org/abuild/tree/default.conf?h=3.12.0#n5 The global GOFLAGS as specified in Alpine Linux are: > export GOFLAGS="-buildmode=pie -modcacherw -trimpath -buildvcs=false" Which we could consider setting globally too, like we do with RUSTFLAGS. Figuring out what Debian does was somewhat challenging, their integration with the go build system is called dh-golang[6], they seem to set CGO_ENABLED=1 only when cross-compiling, with go >= 1.13 (released 2019) they seem to build with `go install -trimpath ...` and no further GOFLAGS environment variable. [6]: https://salsa.debian.org/go-team/packages/dh-golang/-/blob/debian/sid/lib/D… I understand the concern of Go possibly being more prone to ROP-style exploits when 1) built with the native linker and 2) used with e.g. a vulnerable version of libgit2, however, as of 2024, there have been barely any memory-corruption based exploits for Go software. ## Motivation - Most of our Go software is currently not reproducible due to Cgo, including core/libcap, which is the last unreproducible package in docker.io/library/archlinux - The barrier for packaging Go in Arch Linux is currently somewhat high (compared to e.g. packaging Rust), the guideline requires too much interpretation and could be improved - Quirks that are only needed for old Go projects (like 2.1.1) should be listed towards the end instead of being the first code block in the guideline --- cheers, kpcyrd

2 1

Spring cleanup '24
by gromit 20 Mar '24

20 Mar '24

Hello everyone, the amount of birds that are chirping in front of my flat make it clear that spring is coming (at least to Heidelberg). :p Therefore it is time to do our annual spring cleanup and free the repos from any leftovers. Please head to https://archlinux.org/devel/reports/unneeded-orphans/ and adopt packages that you'd like to keep in the repos. The devs among us should maybe also have a look at the (few) [orphans in [core]][0]. This is maybe also a good time to re-evaluate the current package set you maintain and disown anything that you've lost interest in so others can properly maintain it either in the repos or the AUR. Feel free to drop a message here if its anything major. I will start dropping packages on 2024-03-17, so please try to have a look at the lists in the next few days. Cheers, gromit [0]: https://archlinux.org/packages/?sort=&repo=Core&q=&maintainer=orphan

5 7

RFC Final Comment Period: Create new AUR moderator role
by Torsten Keßler 16 Mar '24

16 Mar '24

An RFC has now entered its Final Comment Period. In 14 days, discussion will end and the proposal will either be accepted, rejected or withdrawn https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/19 Please visit the above link for the discussion. Summary: Create a dedicated role for the moderation of the AUR that supports and eventually replaces the Package Maintainers as the current moderators.

1 1

pacman in testing can cause some issues for packagers
by Morten Linderud 15 Mar '24

15 Mar '24

Yo, With `pacman` in testing `[core-testing]` there might be some issues for packagers that use `makepkg --geninteg` or `updpkgsums` and submitting the source packages to build.archlinux.org. As the *sums=() array will contain an integrity check for git sources in pacman 6.1 and beyond the older pacman version on build.a.o is not going to understand this and fail the build. Just set the array to 'SKIP' as usual and it should build fine. I'm not aware of any other packager issues as of writing. -- Morten Linderud PGP: 9C02FF419FECBE16

2 2