Yesterday we had a successful meeting  on bootstrapping Python. For those who might wonder what this is about: To build python-setuptools for Python 3.11 one needs python-setuptools. The same bootstrapping issue exists for other Python packages, which provide (PEP517) build backends and are required for building the remaining ecosystem from scratch.
Filipe has created a project to tackle this issue, it uses python-installer to temporarily create wheels for setuptools, pep517, installer and other packages.  These can then be packaged using a PKGBUILD  and replace the existing python-setuptools package temporarily to build the "real" python-setuptools.
Bootstrapping these tools will be a multiple stage bootstrap (and assumes an already available interpreter package such as python 3.11): 1. Build python-bootstrap 2. Build python-setuptools and co. (without tests) 3. Build all makedepends and depends of the bootstrapped packages (without tests) 4. Build "real" packages (those that have been bootstrapped) and replace the initial bootstrapped packages with them (without tests). 5. Build the ecosystem using the available packages (without tests) 6. Rebuild the ecosystem with tests
In the future we want to rely on python-bootstrap to allow us to transparently bootstrap the Python ecosystem on a new release and document the workflow there. Not all initially required packages are defined yet, but a first attempt at bootstrapping can be found in a custom repository already .
This has been a quick PoC, we still need to put in a lot of work to get this to work out of the box
- Get a full bootstrap working using python-bootstrap - Figure out how we deal with sphinx being a build requirement of python-installer/builder blocking bootstrapping - Figure out how to bootstrap setuptools - Create a script to determine the bootstrap order (using our repository database) - Better tooling for creating custom repositories / devtools chroots for these rebuilds - Automatically update python-bootstrap's git submodules to the repository version - Add documentation about the entire process
If you have time to help testing and developing, please come join us in #archlinux-python on libera.chat.
 https://md.archlinux.org/s/rBZYhg_fE#  https://gitlab.archlinux.org/archlinux/python-bootstrap  https://gitlab.archlinux.org/archlinux/python-bootstrap/-/blob/PKGBUILD/PKGB...  https://pkgbuild.com/~jelle/python3.11/