October 2022 - Arch-projects - lists.archlinux.org

[repod] 2022-10-26 meeting minutes
by David Runge 26 Oct '22

26 Oct '22

Hi all, here are the meeting minutes from today's meeting (in rendered form): https://md.archlinux.org/s/nqqUhLHQ6 Below the plain markdown version: ``` # 2022-10-26 repod meeting Date: 2022-10-26T17:00:00Z - 18:00:00Z Location: Jitsi Scribe: dvzrv ## Attendees * dvzrv * prawn * polyzen * wCPO ## Agenda ### Community outreach We sent [a mail](https://lists.archlinux.org/archives/list/arch-dev-public@lists.archl… about the internal Arch Linux projects and got one contributor in return! \o/ ### Fixing flaky tests that are flaky Took only three(!) merge requests to finally fix a flaky test... ### Write empty sync databases for empty repositories We are now able to also write empty sync databases for empty repositories [!125](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/125) and [!127](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/127). ### Refactor CLI module The CLI now does not use Tasks and Checks directly anymore, but instead relies upon functions from the `repod.action.workflow` module [!126](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/126). ### Import in bulk in integration tests Circumvent an issue with "newer/ already existing" packages in integration tests, by importing in bulk instead [!128](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/128). ### Use schema_version throughout the codebase We now use `schema_version` for the versioning of objects everywhere [!129](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/129) (thanks Isaac!). ### Track source URL per pkgbase in management repository We are now able to track source URLs (the location where package sources live) per pkgbase. This allows for configuration of a specific set of allowed URLs against which a new package's URL must match [!130](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/130). Existing URLs in the management repository are reused if none are provided. In the future we want to also allow updating/ changing the URL separately from a package addition. ### Use debug repositories per stability layer We can now define debug repositories per stability layer (stable, staging, testing) for each repository [!133](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/133). This allows to have a correct handling of debug packages per repository and the moving of packages between repositories. ### Check filename against contained metadata Work has started to check whether the filename of a given consumed file matches the metadata retrieved from it (e.g. `foo-1:1.0.0-1-any.pkg.tar.zst`) [!132](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/132). ### Outstanding milestone 0.3.0 work * The communication with e.g. forges (with or without APIs), providing package sources ([#120](https://gitlab.archlinux.org/archlinux/repod/-/issues/120) and [#118](https://gitlab.archlinux.org/archlinux/repod/-/issues/118)) needs investigation. Help is much appreciated! * Workflows for further actions need to be defined (e.g. remove [140](https://gitlab.archlinux.org/archlinux/repod/-/issues/140) and move [#141](https://gitlab.archlinux.org/archlinux/repod/-/issues/141)) and implemented. Investigating good approaches and the required checks when moving/removing is much appreciated! * Ensure that a repository is self-contained [#142](https://gitlab.archlinux.org/archlinux/repod/-/issues/142) and make this configurable (can already be hooked into existing "add packages" workflow and is a fairly easy topic). * The workflows need to be extended/ adapted to support bulk actions across repositories (e.g. "move 10 packages from testing to core and 20 packages from community-testing to community in one action") [#145](https://gitlab.archlinux.org/archlinux/repod/-/issues/145), but we currently only support bulk actions per-repository (e.g. "add these 20 packages to that repository"). To deal with this in a sane manner, we need someone to investigate how to do this via the CLI (maybe even define a separate plaintext file input format, defining sources and targets - e.g. what was discussed in https://md.archlinux.org/s/mH-Nb_oOD). * Documentation documentation documentation [#144](https://gitlab.archlinux.org/archlinux/repod/-/issues/144) ``` Best, David -- https://sleepmap.de

1 0

Re: Notification: Arch Linux Repod Meeting @ Wed Oct 26, 2022 (droidbittin@gmail.com)
by Luna Jernberg 25 Oct '22

25 Oct '22

Alright, sorry On Tue, Oct 25, 2022 at 2:19 PM David Runge <dave(a)sleepmap.de> wrote: > > On 2022-10-25 12:24:05 (+0200), Luna Jernberg wrote: > > Hey! > > > > Will miss the repod meeting tomorrow, as it collides with foss-sthlm > > October meetup here in Stockholm: https://foss-sthlm.se/ > > > > On Wed, Oct 19, 2022 at 9:00 AM Google Calendar < > > calendar-notification(a)google.com> wrote: > > > > > Arch Linux Repod Meeting > > > You have been invited by droidbittin(a)gmail.com to attend an event named > > > Arch Linux Repod Meeting on Wednesday Oct 26, 2022. > > > > > > You have an upcoming event > > > > > > Arch Linux Repod Meeting > > > Wednesday Oct 26, 2022 > > > Organizer > > > droidbittin(a)gmail.com > > > Guests > > > View all event details > > > <https://calendar.google.com/calendar/event?action=VIEW&eid=Nm5hYThpOWRidjV1…> > > > > > > Invitation from Google Calendar <https://calendar.google.com/calendar/> > > > > > > You are receiving this email because you are subscribed to calendar > > > notifications. To stop receiving these emails, go to Calendar settings > > > <https://calendar.google.com/calendar/r/settings>, select this calendar, > > > and change "Other notifications". > > > > > > Forwarding this invitation could allow any recipient to send a response to > > > the organizer, be added to the guest list, invite others regardless of > > > their own invitation status, or modify your RSVP. Learn more > > > <https://support.google.com/calendar/answer/37135#forwarding> > > > > > Hey Luna, > > as already mentioned by Brett [1], please abstain from sending these > types of notifications. They create unnecessary traffic on the mailing > lists and in this case even created a new topic. > > If you'd like to attend to the meeting, please just join. There is no > need to let anyone know about your absence or attendance. > > Best, > David > > [1] https://lists.archlinux.org/archives/list/arch-projects@lists.archlinux.org… > > -- > https://sleepmap.de

1 0

[repod] Next meeting: 2022-10-26 19:00 CEST
by David Runge 25 Oct '22

25 Oct '22

Hi all, the next meeting for repod will take place on jitsi on 2022-10-26 19:00CEST. https://meet.jit.si/20221026-repod Best, David -- https://sleepmap.de

1 0

Re: Notification: Arch Linux Repod Meeting @ Wed Oct 26, 2022 (droidbittin@gmail.com)
by Luna Jernberg 25 Oct '22

25 Oct '22

Hey! Will miss the repod meeting tomorrow, as it collides with foss-sthlm October meetup here in Stockholm: https://foss-sthlm.se/ On Wed, Oct 19, 2022 at 9:00 AM Google Calendar < calendar-notification(a)google.com> wrote: > Arch Linux Repod Meeting > You have been invited by droidbittin(a)gmail.com to attend an event named > Arch Linux Repod Meeting on Wednesday Oct 26, 2022. > > You have an upcoming event > > Arch Linux Repod Meeting > Wednesday Oct 26, 2022 > Organizer > droidbittin(a)gmail.com > Guests > View all event details > <https://calendar.google.com/calendar/event?action=VIEW&eid=Nm5hYThpOWRidjV1…> > > Invitation from Google Calendar <https://calendar.google.com/calendar/> > > You are receiving this email because you are subscribed to calendar > notifications. To stop receiving these emails, go to Calendar settings > <https://calendar.google.com/calendar/r/settings>, select this calendar, > and change "Other notifications". > > Forwarding this invitation could allow any recipient to send a response to > the organizer, be added to the guest list, invite others regardless of > their own invitation status, or modify your RSVP. Learn more > <https://support.google.com/calendar/answer/37135#forwarding> >

1 0

Hardcoded x86_64 in makerepropkg
by Hugo Osvaldo Barrera 17 Oct '22

17 Oct '22

`makerepropkg` has a hardcoded "x86_64", so fails to run on other [unofficial] architectures, because it tries to use an x86_64-specific makepkg config, which overrides CARCH=x86_64. I tried addressing it with this simple patch: diff --git a/src/makerepropkg.in b/src/makerepropkg.in index b271f25..846af06 100644 --- a/src/makerepropkg.in +++ b/src/makerepropkg.in @@ -92,7 +92,7 @@ get_makepkg_conf() { return 1 fi msg2 "using makepkg.conf from ${fname}" - bsdtar xOqf "${buildtool_file/file:\/\//}" usr/share/devtools/makepkg-x86_64.conf > "${makepkg_conf}" + bsdtar xOqf "${buildtool_file/file:\/\//}" "usr/share/devtools/makepkg-$CARCH.conf" > "${makepkg_conf}" return 0 } This half-works, but the arch derivates still needs to ship a custom `devtools` package with their own `makepkg-$CARCH.conf`. Usually, the only thing that really needs to be changed in the per-architecture custom makepkg.conf is CARCH and CHOST. This is still not ideal, and sounds like it could be refined further into something simpler. The .BUILDINFO file specifies the devtools version originally used, so that makerepropkg can download that specific version and use the right makepkg.conf file for the rebuild. It would be much simpler dump all variables from `makepkg.conf` into .BUILDINFO at build-time, and use these directly when rebuilding to test reproducibility. Instead of the package indicating the version of devtools contains its build configuration, the package just includes the build configuration directly. This removes one level of indirection. This approach would also allow checking the reproducibility of packages that were built with slight different settings (including those built by derivates/unofficial platforms). Any thoughts on this? Side note: the above patch might still have value if merged, at least in the short term. -- Hugo Osvaldo Barrera

1 0

[repod] 2022-10-12 meeting minutes
by David Runge 13 Oct '22

13 Oct '22

Hi all, you can find the meeting minutes in rendered form here: https://md.archlinux.org/s/c4y-uNz5M The plain markdown dump below: ``` # 2022-10-12 repod meeting Date: 2022-10-12T17:00:00Z Location: Jitsi Scribe: dvzrv ## Attendees * dvzrv * lobachevsky * prawn * polyzen * Torxed * wCPO ## Agenda ### CPU architecture test The architecture of consumed packages is checked against the architecture of the target repository. Packages of type "any" may still be added to architecture specific repositories [!117](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/117) ### Link between package names and their pkgbases A symlink in the management repository is introduced to link the name of packages to their respective pkgbases. This allows for easily finding the pkgbases (or changes to them) of updated packages. [!118](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/118) ### Bugfix: Parsing of file modes from mtree files The .MTREE files may have file modes such as `66` in them, so we zero-fill the retrieved file modes (to a four char string with leading zeros) to consolidate the data properly. [!119](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/119) ### Cleanup: Type hinting Type hinting got modernized. [!120](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/120) ### Cleanup: Regular expressions for paths Replace regular expressions used for checking relative/absolute paths with `pathlib.Path.is_absolute()` to increase validation speed and mitigate issues with non-word UTF-8 symbols. [!121](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/121) ### Check packages against management repository A set of checks test, that consumed pkgbases are newer than the ones found in the management repository and that packages can safely replace packages of the same name (but of differing pkgbase). [!122](https://gitlab.archlinux.org/archlinux/repod/-/merge_requests/122) ### Funding Requests for funding have been sent in to [PrototypeFund](https://prototypefund.de/) and [NLnet](https://nlnet.nl/) for their October round of funding. Also contacted Pierre-Loup of Valve about repod, potential use of it in their own infrastructure and funding. ### Milestone 0.3.0 The following issues (and more) still need solving/ implementing: * Lock files need to be written [#117](https://gitlab.archlinux.org/archlinux/repod/-/issues/117). * Cleanup CLI module [#127](https://gitlab.archlinux.org/archlinux/repod/-/issues/127). * Tracking of source URL [#39](https://gitlab.archlinux.org/archlinux/repod/-/issues/39). * Writing of empty sync databases when there are no packages in the repository [#123](https://gitlab.archlinux.org/archlinux/repod/-/issues/123). * Cleanup of symlinks on update and removal of packages [#107](https://gitlab.archlinux.org/archlinux/repod/-/issues/107). ### Add visualization for repod components Add a visualization of the project and its workflows (current and future) [#133](https://gitlab.archlinux.org/archlinux/repod/-/issues/133). ### Increase exposure for repod and other Arch software projects Add a list of all Arch Linux projects (and a short explanatory sentence) in the [Getting involved wiki article](https://wiki.archlinux.org/title/Getting_involved#Software_project… to increase their exposure and publish the change [#134](https://gitlab.archlinux.org/archlinux/repod/-/issues/134). ``` Best, David -- https://sleepmap.de

3 2

[repod] Next meeting: 2022-10-12 19:00 CEST
by David Runge 12 Oct '22

12 Oct '22

Hi all, the next meeting for repod will take place on jitsi on 2022-10-12 19:00CEST. https://meet.jit.si/20221012-repod Best, David -- https://sleepmap.de

2 2

[repod] 2022-09-28 meeting minutes
by Brett Cornwall 05 Oct '22

05 Oct '22

https://md.archlinux.org/s/dyjtzuJMo Raw markdown: # 2022-09-28 repod meeting Date: 2022-09-28T17:00:00Z Location: Jitsi Scribe: brett ## Attendees * alad * artafinde * brett * dvzrv * polyzen * prawn * torxed * wCPO ## Agenda ### New features * Writing to databases via transactions functionality added. * Lots of work for error checking, such as: * architecture/target checking to help guard against pushing the wrong architecture into a repository * Pushing non-debug packages into debug repo * Pushing a version that's older than the current * "Self-healing" when mistakes like that are made, i.e. corrects state when errors are encountered. Testing against upstream repositories are a little tough right now since there's no repod deployment, i.e. there's no repository yet. ### Source packages Currently, source packages are created by some script somewhere and may or may not be working as expected. repod will likely have functionality that will create/push source packages as part of the build process. It will need to do so based on license parsing. ([Issue #125](https://gitlab.archlinux.org/archlinux/repod/-/issues/125)) (Additionally, archweb doesn't appear to be exposing the source package download.) ### Reaching out to downstreams Reaching out to downstreams led to several visitors in the meetings. Kindly visited by Prawn from Kupfer Linux this time. We've unfortunately yet to be visited again by some of those that have previously expressed interest. It'd be good to somehow retain visitors so we get more contributions and we can all get off the manual scripting hacks we all individually write. Downstreams to contact (again): Asahi Linux, Arch Linux ARM, Valve

2 2