- Arch-dev-public - lists.archlinux.org

Python 3.12
by Jelle van der Waa 21 Apr '24

21 Apr '24

Hi all, We should get started on Python 3.12, 3.13 alpha is already getting released! To be 100% honest, I am not super looking forward to it as it's a ton of work due to our lack of automated tooling. In basics a plan would be to first tackle low hanging fruit to-do's: * https://archlinux.org/todo/pytest-runner-removal/ * https://archlinux.org/todo/imp-module-removal-in-python-312/ * https://archlinux.org/todo/use-pyprojecttoml-for-building-if-supported/ * drop unused python modules in our repo https://archlinux.org/devel/reports/unneeded-orphans/ We've finished python-bootstrap, and can bootstrap the required build packages (python-setuptools, build, etc.). And figuring out the rebuild order. (Big thanks to david, for the help here!) [1] [2] So soon we can start the rebuild which will probably take a while to complete as we have no good automated tooling yet for mass rebuilds. This rebuild will likely block [staging] for a good 2 or more weeks, so when would be a good time to schedule this? I expect two things to break our package builds in 3.12: * The removal of distutils see the snippet below, adding python-setuptools to makedepends should resolve this. For example: GISCAN ZBar-1.0.gir Traceback (most recent call last): File "/usr/bin/g-ir-scanner", line 99, in <module> from giscanner.utils import dll_dirs File "/usr/lib/gobject-introspection/giscanner/utils.py", line 385, in <module> import distutils.cygwinccompiler ModuleNotFoundError: No module named 'distutils' * The removal of the imp module, the todo list above has been open for a while, but not all packages have been fixed. * The usual failing tests, missing sources you know it :) [1] https://gitlab.archlinux.org/archlinux/python-bootstrap/-/merge_requests/3 [2] https://notes.vdwaa.nl/arch/python.html Greetings, Jelle

2 6

News draft: Arch Linux Leader Election Results
by gromit 12 Apr '24

12 Apr '24

Hello everyone, a few people have asked for the election results to be published as a News announcement. If you have any feedback regarding the draft below feel free to send it as a response or ping me on IRC. I will post the announcement at the beginning of next week. ^_^ Cheers, gromit === Recently we held our leader election, and the previous Project Leader Levente "anthraxx" Polyák ran again while no other people were nominated for the role. As per [our election rules](https://wiki.archlinux.org/title/DeveloperWiki:Project_Leader#Electi… he is re-elected for a new term. The role of of the project lead within Arch Linux is connected to [a few responsibilities](https://wiki.archlinux.org/title/DeveloperWiki:Project_Le… regarding decision making (when no consensus can be reached), handling financial matters with SPI and overall project management tasks. **Congratulations to Levente and all the best wishes for another successful term!** ===

1 0

Re: Arch Linux Rust packaging licensing problem
by Jan Alexander Steffens (heftig) 07 Apr '24

07 Apr '24

On Sat, Apr 6, 2024 at 10:42 PM Arvid Norlander <arvid(a)vorpal.se> wrote: > Hi, > > After talking to people on Arch Linux IRC channels (mpan in particular) > about this, they recommended I contact you directly about this, since it > affects many packages (so filing a bug on any specific package wasn't > really > appropriate). > > Arch only packages the final binary crates for Rust (as opposed to separate > packages for every single Rust dependency, which would rightfully drive > people crazy). As a result, you only get one single package that contains > many libraries. > > Lets take an example: ripgrep, which links to ~53 dependencies (maybe less, > not all dependencies are used on all platforms, etc). Many of these > dependencies use MIT license or similar that requires: > > > The above copyright notice and this permission notice shall be included > in > > all copies or substantial portions of the Software. > > Looking in /usr/share/licenses/ripgrep I only see the copyright notice > for ripgrep itself. This is a bit of a problem. > > Similarly, one of it's dependencies is unicode-ident which uses > "Unicode-DFS-2016" as it's license. That is missing. encoding_rs is > "(Apache-2.0 OR MIT) AND BSD-3-Clause" (which may collapse down to MIT, not > sure, I am not a lawyer). > > So I can see two problems here: > > 1. Copyright notices that need to be included for dependencies, aren't. > 2. The SPDX expression may be incorrect (missing some things from > dependencies that should be included). > > Looking at some other packages (not just ripgrep) I see similar issues > across all of those packages. > > Now, for something like rust it would be impossible to require the > maintainers to handle this by hand. So lets talk solutions. > > * There is cargo-about (already packaged in Arch, also has suspect > license/copyright info). It doesn't do quite what you want: Given a > config and a handlebar template it will generate an HTML page with all > the licenses you use. It will collapse "OR" in the license info based on > a priority list of which licenses you prefer. > > You could perhaps wrangle the handlebar template to generate a text > file instead of a HTML file, not sure. It can also output a JSON file > instead though (I have used that at my dayjob for license compliance). > > While it is a good solution, it is not a drop-in solution without > additional scripting on top (and you need to specify accepted licenses > for your project). > > * I believe Fedora has some automated tooling based on this comment by a > Fedora packager: > https://users.rust-lang.org/t/psa-check-if-your-cargo-crates-are-clean-and-… > > That is for the SPDX expression. I have not looked into the details of > their tooling. > > * Debian probably have thought about this too, they tend to be rather > careful (some may say up-tight even) about this sort of issues. It might > be worth checking out what they did. > > I'd like some automated tooling that works for AUR too, I maintain some > Rust packages there (some of which I'm also the upstream for). For that > reason, I'd be happy to stay "in the loop" on this issue as well as > possibly help (time and energy permitting, due to recently recovering from > burnout, energy tends to vary on a day to day basis), rather than treating > it as a one-off issue report. > > Is it a high priority issue? No. > Should it be solved eventually? Yes, for legal reasons. > > Best regards, > Arvid Norlander > (Arch Linux user and Rust / C++ software developer) > Hi Arvid, Thanks for bringing this issue to my attention and your detailed email about it. I'm CCïng our public development mailing list in this response so our other maintainers get informed, too. I agree that Arch needs a solution for this eventually. Unlike Fedora we do not package Rust libraries so I think we need some help from Cargo for this. Preferably from upstream, but a third-party tool would work as well. Ideally, I think there we would create a SPDX license expression from the entire crate tree and then simplify it, e.g. to turn `(MIT) AND (MPL-2.0 OR MIT) AND (MIT AND BSD-2-Clause) AND (MPL-2.0 OR BSD-3-Clause)` into `MIT AND BSD-2-Clause AND (MPL-2.0 OR BSD-3-Clause)`. Or perhaps even simpler if the tool had knowledge about which licenses are covered by others. We could call such a tool in the `package()` function to set the `license` for the package. I'm not sure how feasible this would be. Are crates required to use SPDX expressions? Greetings, Jan

1 0

News draft: Increasing the default vm.max_map_count value
by Robin Candau 05 Apr '24

05 Apr '24

MD pad available at https://md.archlinux.org/xl4oBGN9SDWRXnfvHuLhgg?both ---- # Increasing the default vm.max_map_count value As per [this proposal](https://lists.archlinux.org/archives/list/arch-dev-public@lists.a…, we decided to increase the [vm.max_map_count](https://docs.kernel.org/admin-guide/sysctl/vm.html#max-ma… value from the default `65530` to `1048576`, based on the same change that multiple distributions already did on their respective side. This change should help addressing performance, crash or start-up issues for some memory intensive applications, particularly for (but not limited to) [some Windows games](https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2057792/comment… played through Wine/Steam Proton. Overall, it goes forward a smoother user experience out of the box with no expressed concerns about potential downsides in the related proposal. This `vm.max_map_count` increase is introduced in the `2024.04.07` release of the [filesystem package](https://archlinux.org/packages/core/any/filesystem/) and will be effective right after the upgrade. Before upgrading, in case you're already providing your own value for that parameter in a `sysctl` config file, either remove it (to switch to the new default value) or make sure your config file will be read [with a higher priority](https://man7.org/linux/man-pages/man8/sysctl.8.html#SYSTEM_FILE_P… than the `/usr/lib/sysctl.d/10-arch.conf` file (to supersede the new default value). -- Regards, Robin Candau / Antiz

1 0

New supply-chain security tool: backseat-signed
by kpcyrd 03 Apr '24

03 Apr '24

Hello, I'm going to keep this short, I've been writing a lot of text recently (which is quite exhausting, on top of my dayjob and all the code I wrote today afterwards. Apologies if you're still waiting for a reply in one of the other threads). I figured out a somewhat straight-forward way to check if a given `git archive` output is cryptographically claimed to be the source input of a given binary package in either Arch Linux or Debian (or both). I believe this to be the "reproducible source tarball" thing some people have been asking about. As explained in the README, I believe reproducing autotools-generated tarballs isn't worth everybody's time and instead a distribution that claims to build from source should operate on VCS snapshots instead of tarballs with 25k lines of pre-generated shell-script. Building from VCS snapshots is already the case for a large number of Arch Linux packages (through auto-generated Github tarballs). Some packages have been actively converted to VCS snapshots by Arch Linux staff in response to the xz incident. This tool highlights the concept of "canonical sources", which is supposed to give guidance on what to code review. This is also why I think code signing by upstream is somewhat low priority, since the big distros can form consensus around "what's the source code" regardless. https://github.com/kpcyrd/backseat-signed The README shows how to verify Arch Linux and Debian build cmatrix from the same source code - they may both still apply patches (which would be considered part of the build instructions), but the specified source input is the same. This tarball can also be bit-for-bit reproduced from VCS by taking a `git archive` snapshot of the v2.0 tag in the cmatrix repository. (If somebody ever tells you programming in Rust is slower, I wrote the entirety of this codebase within a few hours of a single day) Let me know what you think. 🖤 Happy feet, kpcyrd

1 0

Re: Proposal to increase the default vm.max_map_count value
by Robin Candau 02 Apr '24

02 Apr '24

> On April 1, 2024 12:09:47 PM UTC, Robin Candau <antiz(a)archlinux.org> wrote: > > On 3/25/24 9:10 AM, Robin Candau wrote: > > Hi everyone, > > I'm writing this mail as proposal to increase the default > `vm.max_map_count` value in Arch Linux. > > > It's been a week since this proposal thread [1] was made. For now it > only received a few but all positives feedback (both from staff > members and users in a related thread on arch-general [2]). > > Do we have any more thoughts anyone wants to share? > > Also, if we are to implement this change, anyone has an opinion > about which package should provide it? So far, the following > packages have been mentioned: > - systemd (as Fedora did) > - procps-ng (as Ubuntu did) > - filesystem > > Given that the proposal goes through, I'll create the MR to the > chosen package accordingly. Also, despite this change being > impact-free (as far as we can tell), I think a related news item to > inform users about this change would be a great idea (I'll write > that as well). > > Please, share your opinion about the change itself or the best > package to provide it if you have any :) > > [1] > https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.o… <https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.o…> > [2] > https://lists.archlinux.org/archives/list/arch-general@lists.archlinux.org/… <https://lists.archlinux.org/archives/list/arch-general@lists.archlinux.org/…> > On 4/1/24 11:11 PM, Ehalls wrote: > I would keep it out of systemd. Do you think it is the approproate place? > > (Copying the above response I received from someone off-list, as it is staff only) Since the change consists of shipping a sysctl file and that sysctl is provided by procps-ng, I personally think procps-ng is the appropriate place. -- Regards, Robin Candau / Antiz

1 2

Proposal to increase the default vm.max_map_count value
by Robin Candau 01 Apr '24

01 Apr '24

Hi everyone, I'm writing this mail as proposal to increase the default `vm.max_map_count` [1] value in Arch Linux. The default `vm.max_map_count` 65530 value is making some Windows games crash (or even prevent them to start at all) while being played through Wine or Steam Play (Proton). Examples are (but not limited to) Star Citizen, THE FINALS, Hogwarts Legacy, DayZ or Counter-Strike 2 (see [2] for a list of bug reported to Valve for these games on that matter). In that regard, SteamOS is shipping an increased `vm.max_map_count` default value to address the above issue and numerous distributions have since then decided to implement that change on their side as well, such as Fedora [3], NixOS [4] and, more recently, Ubuntu [5]. While I'm aware that Arch is a user centric/DIY distro and everyone can set their own `vm.max_map_count` value fairly easily, shipping an increased value on our side would be a beneficial change to make in my opinion as it would result in a smoother gaming experience out of the box for our users without representing any downside/side effect (as far as I'm aware). In terms of implementation, the change basically consists of shipping the following sysctl file: ``` # Increase the number of virtual memory areas that one process may request vm.max_map_count=1048576 ``` For reference, Fedora added it to their systemd package [6], while Ubuntu handled it at the procps level [7]. I personally don't have a strong opinion on the implementation way/place. To sum up, I think this would be a reasonable and positive change to make to our distro. I'd be happy to read your thoughts (or eventual objection)! 😊 [1] https://docs.kernel.org/admin-guide/sysctl/vm.html#max-map-count [2] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2057792/comments/5 [3] https://fedoraproject.org/wiki/Changes/IncreaseVmMaxMapCount [4] https://github.com/NixOS/nixpkgs/pull/238459 [5] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2057792 [6] https://src.fedoraproject.org/rpms/systemd/blob/f39/f/10-map-count.conf [7] https://git.launchpad.net/ubuntu/+source/procps/commit/?h=applied/2%254.0.4… -- Regards, Robin Candau / Antiz

3 4

Mirror hardening - complementing RFC 29
by pitastrudl 28 Mar '24

28 Mar '24

Hi, A new RFC (request for comment) has been opened here: https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/35 Please visit the above link for discussion. Summary: Before this proposal, there were a lack of security definitions and measures on mirrors within the Arch Linux community. This RFC therefore outlines the requirements, definitions and guidelines surrounding not only hardening but also maintaining proper security of a mirror. Best regards, Arun

1 0

RFC Final Comment Period: Allow all staff to create RFCs
by Kristian Klausen 23 Mar '24

23 Mar '24

An RFC has now entered Final Comment Period. In 14 days, discussion will end and the proposal will either be accepted, rejected or withdrawn: https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/31 Please visit the above link for discussion. Summary: Grant all Arch Linux staff members, not limited to those in packaging roles, the privilege to initiate RFCs directly, aligning with the broad range of topics these documents encompass.

1 0

Arch Linux minimal container userland 100% reproducible - now what?
by kpcyrd 21 Mar '24

21 Mar '24

hello, in last week's email to the reproducible-builds email list[1] about reproducible Arch Linux I mentioned there's only one unreproducible package left in docker.io/library/archlinux. [1]: https://lists.reproducible-builds.org/pipermail/rb-general/2024-March/00329… Due to amazing work by dvzrv and Foxboron this package is now also reproducible! INFO arch_repro_status > All packages are reproducible! INFO arch_repro_status > Your system is 100.00% reproducible. To try for yourself use: podman run --rm -t archlinux sh -c 'pacman -Suy arch-repro-status --noconfirm && arch-repro-status' However: Where do we go from here? It would be cool if the OCI container image itself could also be reproduced (bit-for-bit), but I'm not sure if there's any prior work (specifically for images listed as 'official' on Docker Hub)? Specifically what I mean - given a line like this: FROM archlinux@sha256:2dbd72d1e5510e047db7f441bf9069e9c53391b87e04e5bee3f379cd03cec060 I want to reproduce the artifact(s) that are pulled in by this, with the packages our Arch Linux rebuilders have reproduced from source code. From what I understand this hash points to a json manifest that is not contained in the container image itself and was generated by the registry (should we archive them?), and this manifest then points to the sha256 of the tar containing the filesystem (I'm possibly missing an indirection here). Hopefully one of the many SBOM formats can help with this. :P I know the container image is built from these two repositories but I don't have any in-depth knowledge: - https://github.com/docker-library/official-images/blob/master/library/archl… - https://gitlab.archlinux.org/archlinux/archlinux-docker The only work towards reproducible container images I'm aware of is by Akihiro Suda: https://github.com/reproducible-containers/repro-get#are-container-images-b… I'm suspecting the current scripts used by Arch Linux would still be prone to mirror changes[2] though, meaning new package uploads would end up in our reproduced artifacts (causing mismatches) and the container image could only be reproduced for a short amount of time. [2]: https://gitlab.archlinux.org/archlinux/archlinux-docker/-/blob/98cd79111dd5… I'm also not sure if there's a missing puzzle piece with reproducible containers in regards to this manifest json that is generated by the registry. The image digest being unpredictable has also been mentioned in a cosign github issue[3]. [3]: https://github.com/sigstore/cosign/issues/2516 Input much appreciated! ## Caveats Probably worth mentioning, at the time of writing there's no consensus across multiple orgs yet, the https://reproducible.archlinux.org instance reports this status, two other rebuilders don't report the full 100% yet. $ arch-repro-status -r https://reproducible.crypto-lab.ch [...] INFO arch_repro_status > 3/118 packages are not reproducible. INFO arch_repro_status > Your system is 97.46% reproducible. $ arch-repro-status -r https://wolfpit.net/rebuild [...] INFO arch_repro_status > 3/118 packages are not reproducible. INFO arch_repro_status > Your system is 97.46% reproducible. The packages in question are part of this rebuild todo (specifically gcc-libs, glibc, ncurses): https://archlinux.org/todo/rebuild-core-with-reproducible-pacman/ Meaning there's currently some luck involved for these 3 packages, e.g. using btrfs currently increases your chances to get an exact match (after a few tries). We're obviously trying to get rid of this caveat though. --- If you appreciate this flavor of supply-chain security you may be interested in repro-env[4] that I'm currently trying to land[5] in ubuntu 24.04 LTS, but is blocked by Debian's libnettle[6]. [4]: https://github.com/kpcyrd/repro-env [5]: https://tracker.debian.org/pkg/rust-repro-env [6]: https://tracker.debian.org/pkg/nettle cheers, kpcyrd

3 2