- Arch-dev-public - lists.archlinux.org

Bootstrapping Python
by Jelle van der Waa 01 Nov '22

01 Nov '22

Hey All, Yesterday we had a successful meeting [1] on bootstrapping Python. For those who might wonder what this is about: To build python-setuptools for Python 3.11 one needs python-setuptools. The same bootstrapping issue exists for other Python packages, which provide (PEP517) build backends and are required for building the remaining ecosystem from scratch. Filipe has created a project to tackle this issue, it uses python-installer to temporarily create wheels for setuptools, pep517, installer and other packages. [2] These can then be packaged using a PKGBUILD [3] and replace the existing python-setuptools package temporarily to build the "real" python-setuptools. Bootstrapping these tools will be a multiple stage bootstrap (and assumes an already available interpreter package such as python 3.11): 1. Build python-bootstrap 2. Build python-setuptools and co. (without tests) 3. Build all makedepends and depends of the bootstrapped packages (without tests) 4. Build "real" packages (those that have been bootstrapped) and replace the initial bootstrapped packages with them (without tests). 5. Build the ecosystem using the available packages (without tests) 6. Rebuild the ecosystem with tests In the future we want to rely on python-bootstrap to allow us to transparently bootstrap the Python ecosystem on a new release and document the workflow there. Not all initially required packages are defined yet, but a first attempt at bootstrapping can be found in a custom repository already [4]. This has been a quick PoC, we still need to put in a lot of work to get this to work out of the box - Get a full bootstrap working using python-bootstrap - Figure out how we deal with sphinx being a build requirement of python-installer/builder blocking bootstrapping - Figure out how to bootstrap setuptools - Create a script to determine the bootstrap order (using our repository database) - Better tooling for creating custom repositories / devtools chroots for these rebuilds - Automatically update python-bootstrap's git submodules to the repository version - Add documentation about the entire process If you have time to help testing and developing, please come join us in #archlinux-python on libera.chat. [1] https://md.archlinux.org/s/rBZYhg_fE# [2] https://gitlab.archlinux.org/archlinux/python-bootstrap [3] https://gitlab.archlinux.org/archlinux/python-bootstrap/-/blob/PKGBUILD/PKG… [4] https://pkgbuild.com/~jelle/python3.11/

1 0

Git submodules in PKGBUILDs are broken as of git 2.38.1 update
by Orhun Parmaksiz 27 Oct '22

27 Oct '22

Hey! As of git 2.38.1 [0], the handling of submodules in PKGBUILDS is broken due to CVE-2022-39253 [1]. This situation affects the packages that use the following command for updating the submodules: > git submodule update This will result in "fatal: transport 'file' not allowed" error since the value of `protocol.file.allow` is changed to be "user" by default. It means that `file://` clones are considered unsafe by default. Currently, there are two possible fixes available: 1. git -c protocol.file.allow=always submodule update 2. git submodule--helper update The latter seems to be an internal command which does not have any public facing documentation whereas the former option is more explicit. There is a related bug report [2] and a TODO list draft [3] for updating the affected packages. I will be updating the VCS guidelines [4] to use the first proposed solution if it all looks good. [0] https://lore.kernel.org/lkml/xmqq4jw1uku5.fsf@gitster.g/ [1] https://www.cve.org/CVERecord?id=CVE-2022-39253 [2] https://bugs.archlinux.org/task/76255 [3] https://md.archlinux.org/YVwV_wIKQfG5obcNLNlCjg [4] https://wiki.archlinux.org/title/VCS_package_guidelines#Git_submodules Let's make some gains, Orhun

2 1

mkinitcpio and arch-install-scripts hackathon 22nd of October
by Morten Linderud 21 Oct '22

21 Oct '22

Yo, On the 22nd of October (tomorrow) there is going to be a small hackathon on minitcpio and arch-install-scripts! mkinitcpio needs some love before a new release can be made, the goal would also be to work on the open pull-requests before moving the project too our Gitlab instance. https://github.com/archlinux/mkinitcpio/ The current ideas is to do scope out a bugfix release and then a larger release, and we would love some help to do that. Beyond that we want to look at improving the general status of the systemd hooks in mkinitcpio, there are several projects that implements parts of this better than mkinitcpio and it would be neat to figure out how we should work on that. For ideas see issue 121 in the repo; https://github.com/archlinux/mkinitcpio/issues/121 I'm also maintaining arch-install-scripts but that is mostly out of necessity :) I'd love to find a new maintainer for the project, so any ideas or issues can also be worked on tomorrow! https://github.com/archlinux/arch-install-scripts This is a great opportunity for new people to get involved with some Arch project development :) The only requirement is to not be afraid to dive into bash/ash scripts. Join `#archlinux-projects` on the libera.chat network or through matrix if you are interested :)! -- Morten Linderud PGP: 9C02FF419FECBE16

1 0

git 2.38.0: Change in `git archive` output
by kpcyrd 16 Oct '22

16 Oct '22

hello, multiple people in Arch Linux noticed the output of our `git archive` command doesn't match the tarball served by github anymore. First I suspected an update in our gzip package until I found this line in the git 2.38.0 release notes: > * Teach "git archive" to (optionally and then by default) avoid > spawning an external "gzip" process when creating ".tar.gz" (and > ".tgz") archives. I've then found this commit that could be considered a breaking change in `git archive`: https://github.com/git/git/commit/4f4be00d302bc52d0d9d5a3d4738bb525066c710 I don't know if there's some kind of gzip standard that could be used to align the git internal gzip implementation with gnu gzip. I'm not saying this is necessarily a bug or regression but it makes it harder to reproduce github tar balls from a git repository. Just sharing what I've debugged. :) cheers, kpcyrd

1 0

Public release: updlockfiles 0.1.0
by kpcyrd 16 Oct '22

16 Oct '22

ohai! I released a new tool called updlockfiles that attempts to help manage dependency lockfiles like Cargo.lock, composer.lock, go.mod and friends. Hopefully this helps with reproducible builds in Arch Linux, but also if you want to manage your own downstream lockfile to override vulnerable dependencies. Announcement blog post: https://vulns.xyz/2022/10/updlockfiles/ Repository: https://github.com/kpcyrd/updlockfiles cheers, kpcyrd

1 0

Arch Linux software projects: Request for participation
by David Runge 16 Oct '22

16 Oct '22

Hi all, we have recently cleaned up parts of the developer wiki a bit and moved the list of internal projects to a more prominent location: https://wiki.archlinux.org/title/Getting_involved#Software_projects As you can see, we have quite a few projects and some of them would be happy about more participation from members of the Arch Linux staff as well as users. Help in fixing and extending the software is much appreciated, but testing and opening tickets that describe flaws or usability problems are equally important! If you are familiar with writing documentation, Ash, Bash, C, JavaScript, Python, or Rust and always wanted to get involved with one of the Arch Linux projects: Look no further! :) As mentioned in the wiki, we have a central mailing list [1] and IRC channel [2] for discussing most of the projects (unless they provide other means of communication). We hope to spark people's enthusiasm, as healthy projects ensure a flexible distribution and allow us to tackle larger goals (e.g. move to a git based workflow for packaging, effortlessly providing several architectures, automating our infrastructure and releases, etc.). Best, David [1] https://lists.archlinux.org/mailman3/lists/arch-projects.lists.archlinux.or… [2] ircs://irc.libera.chat/archlinux-projects -- https://sleepmap.de

1 0

[core] cleanup
by David Runge 13 Oct '22

13 Oct '22

Hi all, we need to do a bit of [core] repository cleanup. In this MD [1] (publicly visible readonly link [2]) we have a list of packages that should leave [core], some maybes that need discussion before moving them and some that should be moved to [core] from the other repositories. If you have more packages, that should not be in [core] anymore or should be moved to it, please add them to the list(s). The general rules for inclusion in [core] are, that we want to - preserve base-devel as a group there - preserve base as a meta-package there - preserve makedepends and depends of a [core] package there The checkdepends and also optdepends of a package may reside in other repositories. As we have a few removals of packages in the list that concern packages only in [core] for historical (or other) reasons, please keep the discussion about them focused on technical reasons as to why they should potentially remain there. Additionally, we have a few orphan packages in [core] that need adoption. Please adopt packages (if you can)! Best, David [1] https://md.archlinux.org/nrXL0ay5Tp6wP-f5M7N8bA# [2] https://md.archlinux.org/s/MliseALTG# -- https://sleepmap.de

6 11

GeoIP mirror of archboot?
by Tobias Powalowski 05 Oct '22

05 Oct '22

Hi guys, would it be possible to link the archboot directory to GeoIP too: archboot.mirror.pkgbuild.com/ ? Thanks. greetings tpowa -- Tobias Powalowski Arch Linux Developer & Package Maintainer (tpowa) https://www.archlinux.org tpowa(a)archlinux.org Archboot Developer https://bit.ly/archboot St. Martin-Apotheke Herzog-Georg-Str. 25 89415 Lauingen https://www.st-martin-apo.de info(a)st-martin-apo.de

2 2

Arch Linux in September 2022
by Levente Polyak 03 Oct '22

03 Oct '22

## RFC We entered the final discussion period for the RFC [0] to merge the pacman repository `[community]` into `[extra]` as well as the packaging VCS location `community` into `packages`. The `[core]` repository remains limited to Developers. One the RFC is accepted, we will implement the proposal during the migration to Git packaging. ## devtools Changes were prepared to account for the repo merger RFC and its resulting workflow. We are slowly approaching a state of Git packaging that could be used productively. Once the final bits haven been finalized, we will present the proposal and discuss the migration and switch over. ## mailman3 migration We have finished the full migration to mailman3 [1]. Rewriting of the "From" header and subject (to prepend the list name) have been disabled to keep the DKIM signature intact. This means "reply to mailing list" must be used when replying to the list and you may need to update your filters and rules matching the "From" header. All existing subscriptions are migrated and you do not need to re- subscribe. ## python2 removal Python 2 went end of life January 2020. Since then we have been actively cutting down the number of projects depending on python2 in our repositories, and we have finally been able to drop it from our distribution [2]. ## RISC-V mirror We started hosting [3] our latest efforts of the yet unofficial RISC-V port. In the past year, we have a boost in progress and currently ~87.5% of Arch x86_64 packages are available in the riscv64 repository [4]. [0] https://gitlab.archlinux.org/archlinux/rfcs/-/merge_requests/14 [1] https://archlinux.org/news/arch-linux-mailing-list-changes/ [2] https://archlinux.org/news/removing-python2-from-the-repositories/ [3] https://riscv.mirror.pkgbuild.com/ [4] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.o…

1 0

Discussion around archlinux-keyring and pacman-key
by David Runge 30 Sep '22

30 Sep '22

Hi all, I've started a discussion [1] around a potential improvement for dealing with archlinux-keyring and pacman-key. Lately we have been facing quite a few issues (e.g. initialization and ordering issues in archiso and arch-boxes) again, that emerge from how gnupg is used in the context of pacman-key. The discussion linked to above is a bit more midterm in regards to what the proposed idea represents, but it tries to draw out a possible scenario of using key data in the future. When looking at pacman-key I noticed, that it does not do locking of the gnupg keyring data [2]. This means, that in environments in which we bootstrap the system (e.g. archiso/arch-boxes) we may run into inconsistent state if we use pacman too early, or generally if we modify the keyring concurrently. Unfortunately, (to my knowledge) there is no way of telling when is "too early" or whether we are already using the keyring, due to the missing locking mechanism. In the past days there have also been a multitude of reports of people with keyring issues on reddit. While some of theme have been due to initialization issues in archiso with pacman < 6.0.1-8, there is apparently also a set of issues on existing systems where it is unclear why pacman-key would have issues validating keys that are already present on the user system (trying to get more info from the users). Overall this is a bit worrying and I think we should work towards addressing the problem that we have with signing and verifying packages. The discussion in archlinux-keyring is only one possible solution, others are implementing a locking mechanism (;_;) or working towards a signing enclave for our packages and databases to minimize the keys in use. IMHO, removing the use of gnupg as proposed in the linked discussion could reduce complexity quite a bit, but it of course also requires implementation in archlinux-keyring and pacman. I'd be happy to get more input and thoughts on this entire topic. Best, David [1] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/issues/199 [2] https://gitlab.archlinux.org/pacman/pacman/-/blob/546433b4fd8a3ede5d04d56b3… -- https://sleepmap.de

1 0