- Arch-dev-public - lists.archlinux.org

[arch-dev-public] Starting x86_64_v3 port
by Allan McRae 08 Feb '22

08 Feb '22

Hi all, You may remember long in the past we discussed adding an x86_64_v3 port. From memory, pkgstats shows this will benefit ~2/3 (or 3/4?) of our users! So lets get this underway! Apart from tooling (devtools/dbscripts), we need to make some decisions. My plan is to seed the x86_64_v3 repos with x86_64 packages and then they get replaced as updates/rebuilds happen (potentially supplemented with a rebuild party). Pacman 6.0 can handle this. This is not a perfectly clean new port, but is substantially less of a burden than releasing a pure x86_64_v3 port. Our .BUILDINFO files do record the package architecture, so this mixture should not affect (e.g.) our reproducible builds etc. The decision to be made is who will package for this repo? I think these are the options: A) packagers upload both x86_64 and x86_64_v3 to the repos. Our build server will help those without x86_64_v3 machines. B) we recruit some packagers to build the x86_64_v3 packages. C) Some combination of A+B. My understanding is our x86_64 port started with B, then C, then A. I think with our build infrastructure now, we can start with A, but that is more of a burden for packagers. I doubt it will be much of a burden as x86_64_v3 specific build issues are unlikely. Is there any particular objection to requiring packagers upload both architectures? Allan

14 27

[arch-dev-public] [RFC] archweb nvchecker integration
by Jelle van der Waa 07 Feb '22

07 Feb '22

We’ve wanted automatic flagging packages out of date for a while and currently every packager has to come up with it’s own solution. Let’s solve this centralized by integrating nvchecker into archweb. nvchecker is a program which can monitor versions upstreams, github, pypi, haskell, crates.io or custom provided for updates. Various packagers already use nvchecker in some form. [1] The idea is that every package in svn has a .NVCHECKER file (linux/trunk/.NVCHECKER) which contains the package specific settings i.e.: [go] source = “github” github = “golang/go” prefix = “go” use_max_tag = true exclude_regex = “.(release|weekly|rc|alpha|beta).” Archweb will have a systemd service which regularly goes through the pkgbases, check if the file exists, runs nvchecker and if required flags the package out of date automatically. For Github sources a token is required so we can do 5000 requests per hour, Arch Linux already has a Github account so this is no issue. Felix already has a script which converts a subset of packages to nvchecker. This could be extended to apply to more packages. [2] The question is if anyone object to checking these small .NVCHECKER files into our svn repository. If there are no real objections, I'll start implementing this functionality into archweb in two weeks. [1] https://nvchecker.readthedocs.io/en/latest/usage.html#check-crates-io [2] https://github.com/felixonmars/archlinux-futils/blob/master/arch-to-nvcheck… Greetings, Jelle van der Waa

12 17

[arch-dev-public] build automation discussions
by Xyne 07 Feb '22

07 Feb '22

Hi, In the recent discussions about possibly adding support for x86_64-v3 the general consensus seemed to be that we need to first work on our packaging infrastructure (build pipelines, reproducible builds, etc.). Who's currently working on that and where I can get involved? Are there already any tools in development? Aside from some helper tools such as arch-rebuild-order I didn't find anything in the list of official projects on our Gitlab. Regards, Xyne

2 1

[arch-dev-public] Arch Linux in January 2022
by Levente Polyak 06 Feb '22

06 Feb '22

## Staff We have on-boarded wiki maintainers as part of our official staff [0] and wish a warm welcoming. We would also like to welcome BrainDamage among the Arch IRC op team. ## Debug packages The required devtools release is live, which handles debug packages and makes sure to release them when available. Currently we have a debuginfod service which is capable of delivering source listings and debug symbols to users with gdb, delve and other debuggers. This can be enabled by installing the `debuginfod` package and setting the environment variable `DEBUGINFOD_URLS="https://debuginfod.archlinux.org"`. The `debuginfod` package is also going to be providing this variable. ## devtools The devtools project has migrated to GitLab as well including all currently open merge requests and issues. Unfortunately the migrated merge requests are just a branch, in case the original authors want to update them, please resubmit the patches. ## Keyring Our new keyring tooling has completely replaced the old usage and data storage. We have merged the new curated keyring data and released the first few keyrings using the new curated tooling and data. Furthermore we have added more packager keys with @archlinux.org uids so we can make sure all important keys will be reachable through our own WKD. ## Gluebuddy A new helper tool named gluebuddy has been released [1] and successfully rolled out. Gluebuddy helps devops to maintain certain aspects of our on and off-boarding with automation by syncing GitLab groups and project settings. This reduces the required manual work and ensures we have quicker on-boarding as well as do never forget to remove permissions for former staff. ## linux-firmware The linux-firmware package has been split up to reduce download and installation size for most setups. ## OpenSearch We have replaced the elasticsearch package stack with OpenSearch. On top we have started [working towards adding Arch Linux to the official OpenSearch download page[2]. Furthermore some progress has been made to allow reproducible builds[3] [0] https://gitlab.archlinux.org/archlinux/infrastructure/-/issues/375 [1] https://gitlab.archlinux.org/archlinux/gluebuddy [2] https://github.com/opensearch-project/project- website/pull/642 [3] https://github.com/opensearch-project/OpenSearch /pull/1995

1 0

[arch-dev-public] Debug packages for Arch Linux
by Morten Linderud 05 Feb '22

05 Feb '22

I'm very happy to announce that debug packages in Arch Linux has been deployed :) This work began after FOSDEM 2020 and was announced in November 2020. However because of time constraints it took quite a while before we managed to deploy it. Currently we have a debuginfod service which is capable of delivering source listings and debug symbols to users with gdb, delve and other debuggers. This can be enabled by installing the `debuginfod` package and setting the environment variable `DEBUGINFOD_URLS="https://debuginfod.archlinux.org"`. The `debuginfod` package is also going to be providing this variable :) The debug package repositories themself are not publicly accessible nor distributed to our mirrors. This is because there is a general concern around the repository size increase. The goal is to take a look at distributing them and/or making them accessible in the future. Packages are being rebuilt for debug symbols where deemed needed by the packager and I suspect we should have a discussion around enabling the debug option by default or not. However see the section below around creating debug packages as there are some caveats of the current approach. I have added some links to the work that has been done to accomplish all of this at the bottom of this email. For general information about the instance one can look at the service itself: https://debuginfod.archlinux.org/ Previous emails on this topic: * https://lists.archlinux.org/pipermail/arch-dev-public/2020-November/030222.… * https://lists.archlinux.org/pipermail/arch-dev-public/2021-December/030583.… # archlinux.org news announcement Suggestion for the news announcement on archweb. I'll post it in a day or two unless there are any objections. Subject: Debug packages and debuginfod We are very happy to announce that debug packages are now available in Arch Linux. Debug symbols and source listing are provided through our debuginfod instance which can be utilized by debuggers such as gdb and delve. In the future we are planning to make these repositories public on our mirrors. For more information on this please visit the [archwiki](https://wiki.archlinux.org/title/Debuginfod) article, Or the debuginfod service. https://debuginfod.archlinux.org/ # Creating debug packages Currently with pacman 6.0.1-3 debug package only works on C/C++ projects. This is mostly because pacman utilizes a fairly ugly awk hack to extract sources and if it encounters a binary from Go or Rust (as an example) it is simply unable to deal with them and produces invalid packages. This is fixed with a patch i wrote which replaces the AWK hack with debugedit. https://gitlab.archlinux.org/pacman/pacman/-/commit/ae2f506ddfd11d9becda721… Until this patch is backported, or a pacman release is done, I would advise us to not build debug packages for anything that isn't using gcc/clangd. Saying that, I'm unsure how we should be documenting debug packages across our distro. They probably belong in package guidelines sections of each package, or we make a common wiki page for this. Generally Rust needs a environment variable and Golang needs a bit of work to disable decompressed DWARF headers. We can probably work something out as we go along, currently I think we should try enable debug packages on our [core] packages. When we have pacman with the above patch I can try compile a list of available methods to get debug packages from Rust, Go and other languages. Obviously I'd need some help with that :) Cheers! -- Morten Linderud PGP: 9C02FF419FECBE16 # infrastructure https://gitlab.archlinux.org/archlinux/infrastructure/-/merge_requests/168 # dbscripts https://gitlab.archlinux.org/archlinux/dbscripts/-/merge_requests/21 https://gitlab.archlinux.org/archlinux/dbscripts/-/merge_requests/33 # devtools https://gitlab.archlinux.org/archlinux/devtools/-/merge_requests/78

6 9

[arch-dev-public] Scope of arch-projects mailing list
by David Runge 04 Feb '22

04 Feb '22

Hi all, while trying to use the arch-projects mailing list for an announcement I realized, that it is (probably?) not really supposed to be used for that at the moment. It explicitly states to be used for development discussion and providing patches for dbscripts, devtools, mkinitcpio, namcap, netctl, archweb and pyalpm. It seems that apart from netctl none of the above really make much use of this anymore though when it comes to development, as the projects have pivoted towards merge/pull requests on their respective VCS forges. Unless there are any objections, I propose to * add arch-repo-management and arch-release-promotion to it * change arch-projects to *also* become a general discussion list around the above mentioned Arch Linux projects * extend mailing list "about" section with information on how each project prefers to receive patches (e.g. I do not want to use the mailing list for that, but others may) Rationale: We lack a mailing list that revolves around our internal projects and which can also be used by non "elevated users" (such as Trusted Users and Developers) for more general discussion. Given the very low traffic on the arch-projects mailing list I see no issue with it becoming a per-project discussion list as well (as in our case mailing lists seem to be mainly used for discussion and not so much for development anymore these days). Best, David -- https://sleepmap.de

3 3

[arch-dev-public] arch-repo-management walkthrough 2022-02-02 19:00 CET (UTC+01:00)
by David Runge 03 Feb '22

03 Feb '22

Hi all, given recent topics for build automation and work on internal projects I would like to announce a code walkthrough for arch-repo-management [1]. I would like to give an overview of the scope of the project, its current features and which features I would like to see implemented (some of which are still in a discussion phase). I will try to add a few more tickets beforehand to track open features and to outline them better. This way I hope that it becomes easier for interested people to follow up on them. The meeting will take place on Jitsi https://meet.jit.si/20220202-arch-repo-management on 2022-02-02 starting around 19:00 CET (UTC+01:00). Any changes to the date and/or location will be announced in this mail thread. Best, David [1] https://gitlab.archlinux.org/archlinux/arch-repo-management/ -- https://sleepmap.de

4 7

[arch-dev-public] Arch Linux Secure Boot support, get it done ;)
by Tobias Powalowski 02 Feb '22

02 Feb '22

Hi guys, next topic to step up: Secure Boot support I wanted to bring this up to the mailing list to collect all info about the status of SB support. There are different approaches on how to implement SB. My question first: Do we want a signed shim for Arch Linux? If yes, how to achieve this? We should make a process in our infrastructure and document what needs to be done and who does it. Morten would you please add your thoughts about this topic on the ML, i think you are most into SB on linux. Thanks. greetings tpowa -- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) https://www.archlinux.org <http://www.archlinux.org> tpowa(a)archlinux.org St. Martin-Apotheke Herzog-Georg-Str. 25 89415 Lauingen https://www.st-martin-apo.de info(a)st-martin-apo.de

3 2

[arch-dev-public] mkinitcpio some improvements and question on how kernel modules should be treated
by Tobias Powalowski 01 Feb '22

01 Feb '22

Hi guys, did some initcpio optimizing lately and implemented correct symlink handling. This saves already some space. https://bugs.archlinux.org/task/73439 Now one major problem left in recent initcpio, uncompressed modules: https://github.com/archlinux/mkinitcpio/pull/25 https://bugs.archlinux.org/task/72882 At the moment modules are uncompressed during initcpio creation. This leads to bigger RAM usage during boot, makes it a bit smaller on storage, due to not recompress compressed modules. On small initrds for normal boot this is not a big matter. My measurement with all patches added: Ramdisk size uncompressed modules: 13261677 Ramdisk size compressed modules: 13112083 (Note the compressed one is smaller due to the symlink patches added) Unpacked initramfs uncompressed modules: 35 MB Unpacked initramfs compressed modules: 26 MB Using this on the biggest "initramfs testsuite" archboot shows the most advantages. If you like to test it on your own initramfs setup you can use: https://gitlab.archlinux.org/tpowa/archboot/-/blob/master/usr/share/archboo… for /usr/lib/initcpio/functions and recreate your initramdisk. Now my question to the list, what should be the default how arch handles modules in initramfs? - Shall we leave the modules at compression level used during kernel build? --> This means the uncompression should be optional. - Shall we still uncompress the modules always? (Status now) --> Then using modules as is needs to be optional. Thanks for you input. greetings tpowa -- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) https://www.archlinux.org <http://www.archlinux.org> tpowa(a)archlinux.org St. Martin-Apotheke Herzog-Georg-Str. 25 89415 Lauingen https://www.st-martin-apo.de info(a)st-martin-apo.de

3 3

Re: [arch-dev-public] Debug packages for Arch Linux
by Morten Linderud 30 Jan '22

30 Jan '22

On Sun, Jan 30, 2022 at 08:59:34PM +0100, sL1pKn07 SpinFlo wrote: > El dom, 30 ene 2022 a las 20:54, Morten Linderud (<foxboron(a)archlinux.org>) > escribió: > > > On Sun, Jan 30, 2022 at 08:49:29PM +0100, sL1pKn07 SpinFlo wrote: > > > hi > > > > > > thanks for the work! > > > > Thanks! > > > > > one question. This thing works, for example, if I build one of the > > packages > > > with debug package in the arch debuginfod site with my custom flag for > > > disable an option/feature. the debug package stored in the debuginfod > > site > > > still works? or i need to create a debug package locally? > > > > The buildid would be different so our debuginfo won't work with your > > packages. > > > > You can check with: readelf -n /bin/ls | grep -A4 build.id > > > > > If I need to build a local debug package and I want to use it > > > with debuginfod, is it possible add the link of the custom debuginfod > > site > > > in the DEBUGINFOD_URLS variable? how? is , separate? or ;? > > > > Yes, space is the seperator. See > > https://man.archlinux.org/man/debuginfod-find.1#ENVIRONMENT_VARIABLES > > > > You can also add a file under /etc/debuginfod/*.urls like we did with our > > `debuginfod` package. > > > > -- > > Morten Linderud > > PGP: 9C02FF419FECBE16 > > > > Hi > > thanks for the reply > > is here documented the file/directory structure of debuginfod "repo"? > > greetings I recommend reading the documentation frankly. debuginfod just recursively scans a set of directories which is polling our package repositories. The implementation is linked in the email. https://gitlab.archlinux.org/archlinux/infrastructure/-/merge_requests/168/… -- Morten Linderud PGP: 9C02FF419FECBE16

1 1