Systemd service and timer for refreshing archlinux-keyring keys via WKD
Hi all, I currently have a MR open against archlinux-keyring [1], that adds a systemd service and timer, which would automatically refresh valid and existing keys on user systems. For people without access to the repository's ticket and merge request features, it is possible to browse the commits in question on the respective branch [2]. To quote from the accompanying ticket [3]: ``` For situations in which we do have new signatures which have been released, we may want a service on user systems that update existing keys via WKD to have an updated set of keys before updating archlinux-keyring. Rationale: Keys are updated in WKD after a release of archlinux-keyring (currently this is still done in the https://gitlab.archlinux.org/archlinux/wkd repo). User systems may upgrade any time after that and pacman will pull new keys from WKD automatically, however it will not pull updates for existing keys. Packages that are signed with a key that still had marginal trust in release A (and therefore already existed on the user system since release A) and gained full trust in release B will not be updated before the user does a system upgrade. This leads to the requirement of installing archlinux-keyring before doing a system upgrade, as otherwise the key will still have marginal trust on the user system and the signatures of other updated packages using the key in question will fail to validate. To remedy this situation I think it would be helpful to have a service update existing keys from WKD twice daily on a timer. I am not 100% sure whether this has side-effects in regards to revocations, but as we only ever update WKD after issuing a release and not when updating the default branch, this should be fine(?). ``` In short: The service in question will update any valid keys in the pacman keyring on user systems, that use Arch Linux's relevant domains as UID to retrieve updated keys (e.g. extended expiration, additional signatures). Invalid keys and those of other domains are ignored. This feature is implemented to circumvent (some, but not all) cases of "update archlinux-keyring before doing a system upgrade". Cases not covered are (in no particular order): * *long* outdated system is updated * new main signing key is required for a packager key to gain full trust for the packages about to be updated * local system's archlinux-keyring does not yet offer this feature * local system's pacman does not yet offer the feature of downloading package signatures for new keys from WKD * packager key on local system does not yet have any main key signatures and would gain full trust with an update of archlinux-keyring Currently the timer which triggers this service is supposed to be vendor enabled (i.e. symlink in /usr/lib/systemd/system/timers.target.wants/) and run daily with a deviation of up to 12h. Members of the DevOps team have raised concerns about the interval as we do not really have a clear picture about how many Arch Linux installations there are world-wide to get numbers on expected median load for this. If you have ideas for improvement or general questions, please direct them at the merge request. Best, David [1] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/merge_requests/13... [2] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/tree/wkd_sync_ser... [3] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/issues/184 -- https://sleepmap.de
On 23.07.2022 18.38, David Runge wrote:
Hi all,
I currently have a MR open against archlinux-keyring [1], that adds a systemd service and timer, which would automatically refresh valid and existing keys on user systems. For people without access to the repository's ticket and merge request features, it is possible to browse the commits in question on the respective branch [2].
To quote from the accompanying ticket [3]: ``` For situations in which we do have new signatures which have been released, we may want a service on user systems that update existing keys via WKD to have an updated set of keys before updating archlinux-keyring.
Rationale: Keys are updated in WKD after a release of archlinux-keyring (currently this is still done in the https://gitlab.archlinux.org/archlinux/wkd repo). User systems may upgrade any time after that and pacman will pull new keys from WKD automatically, however it will not pull updates for existing keys. Packages that are signed with a key that still had marginal trust in release A (and therefore already existed on the user system since release A) and gained full trust in release B will not be updated before the user does a system upgrade. This leads to the requirement of installing archlinux-keyring before doing a system upgrade, as otherwise the key will still have marginal trust on the user system and the signatures of other updated packages using the key in question will fail to validate.
To remedy this situation I think it would be helpful to have a service update existing keys from WKD twice daily on a timer.
I am not 100% sure whether this has side-effects in regards to revocations, but as we only ever update WKD after issuing a release and not when updating the default branch, this should be fine(?). ```
In short: The service in question will update any valid keys in the pacman keyring on user systems, that use Arch Linux's relevant domains as UID to retrieve updated keys (e.g. extended expiration, additional signatures). Invalid keys and those of other domains are ignored.
This feature is implemented to circumvent (some, but not all) cases of "update archlinux-keyring before doing a system upgrade". Cases not covered are (in no particular order): * *long* outdated system is updated * new main signing key is required for a packager key to gain full trust for the packages about to be updated * local system's archlinux-keyring does not yet offer this feature * local system's pacman does not yet offer the feature of downloading package signatures for new keys from WKD * packager key on local system does not yet have any main key signatures and would gain full trust with an update of archlinux-keyring
Currently the timer which triggers this service is supposed to be vendor enabled (i.e. symlink in /usr/lib/systemd/system/timers.target.wants/) and run daily with a deviation of up to 12h. Members of the DevOps team have raised concerns about the interval as we do not really have a clear picture about how many Arch Linux installations there are world-wide to get numbers on expected median load for this.
The load aspect should be solvable, worst-case DevOps gets annoyed ;) My main concern is putting ourselves in a position, where we know every arch installation, yes it is just the IP addresses, but still. At the same time it becomes easier to detect that a computer is running arch, by just look at the network traffic (yes you can already do that today, by checking if the computer is connecting to a arch mirror). Maybe I'm blowing it out of proportion, but with our user base being more privacy aware than most people, I think it is worth mentioning. Cheers, Kristian
If you have ideas for improvement or general questions, please direct them at the merge request.
Best, David
[1] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/merge_requests/13... [2] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/tree/wkd_sync_ser... [3] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/issues/184
On Sat, 23 Jul 2022 at 19:39, David Runge <dave@sleepmap.de> wrote:
Packages that are signed with a key that still had marginal trust in release A (and therefore already existed on the user system since release A) and gained full trust in release B will not be updated before the user does a system upgrade. This leads to the requirement of installing archlinux-keyring before doing a system upgrade, as otherwise the key will still have marginal trust on the user system and the signatures of other updated packages using the key in question will fail to validate.
This is solvable by not cutting a release with marginally trusted keys. Having all Arch Linux installations make 100-ish requests daily to cover such an edge case is a misutilization of resources (on both sides). I'm certain there are also privacy concerns about enabling this service by default. Furthermore, I doubt our users need to the have their systems babysat like this. In the rare situation where archlinux-keyring must be updated first, the user should be able to handle it by themselves. As you said, pacman will fetch new keys using WKD so, as long as marginally trusted keys are excluded from keyring releases, there's no issue with onboarding new people. tl;dr: I'm vibing way more with continuing to rely on the archlinux-keyring package exclusively; auto-updates are sus.
On 2022-07-23 21:24:38 (+0200), Kristian Klausen wrote:
The load aspect should be solvable, worst-case DevOps gets annoyed ;)
My main concern is putting ourselves in a position, where we know every arch installation, yes it is just the IP addresses, but still. At the same time it becomes easier to detect that a computer is running arch, by just look at the network traffic (yes you can already do that today, by checking if the computer is connecting to a arch mirror).
Maybe I'm blowing it out of proportion, but with our user base being more privacy aware than most people, I think it is worth mentioning.
Fair point, however, a similar concern was networkmanager doing a connectivity check [1]. It is up to us to configure our webserver accordingly and be as privacy conserving as possible. E.g. If we wanted to track users (which we don't), we could do so on a rudimentary basis now already by tracking downloads of packager keys. The Web Key Directory is a way more centralistic approach than e.g. SKS was, but we also have better control over how and what we provide there. Best, David [1] https://github.com/archlinux/svntogit-packages/blob/8bdf9488f2845cad52794589... -- https://sleepmap.de
On 2022-07-23 23:49:09 (+0300), Evangelos Foutras wrote:
This is solvable by not cutting a release with marginally trusted keys. Having all Arch Linux installations make 100-ish requests daily to cover such an edge case is a misutilization of resources (on both sides).
Refreshing existing and valid keys not only has the upside of pulling in new signatures on marginally trusted keys, but also to update expiration dates. I do not see this as misutilization of the systems, as it solves an existing and continued issue for users and user experience by making the upgrade procedure less error-prone (and confusing). When it comes to utilization of our systems (because we would be on the giving end of this transaction), I do understand your concerns. The WKD is a very central and important resource to the distribution and we need to ensure (in any case), that we can serve many users at once and that this does not mess with our gitlab setup. If we could communicate more detailed numbers there and figure out a way to make the hosting of our WKD very stable and reliable, that would be fabulous!
I'm certain there are also privacy concerns about enabling this service by default.
Each upgrade of a user system may trigger a request towards our WKD (unrelated to the proposed service). As mentioned in my response to Kristian, I believe that it is up to us to configure our webserver infrastructure as privacy conserving as possible (for all of our services).
Furthermore, I doubt our users need to the have their systems babysat like this. In the rare situation where archlinux-keyring must be updated first, the user should be able to handle it by themselves. As you said, pacman will fetch new keys using WKD so, as long as marginally trusted keys are excluded from keyring releases, there's no issue with onboarding new people.
Marginally trusted keys may not only end up in the keyring by adding new packagers, but also by existing packagers changing their keys. As mentioned above, they may be (temporarily) expired as well. We are not really able to make the guarantees that you stated (e.g. "we will from now on only release fully trusted keys"). Scenarios such as emergency key revocations, or just plain oversight may still lead to marginally trusted keys ending up in the keyring. Establishing a workflow around the maintenance and release of archlinux-keyring is challenging in itself and changing it would not solve the problems with temporarily expiring keys either (unless we forbid expiring keys). Spending time on various fora to explain to users of all experience ranges to "first do a partial upgrade to upgrade the keyring" is not only tiring and creating friction, but also wasting a more valuable resource: Contributor time. The update procedure on Arch Linux should be as easy as possible and as resilient to failure as possible for as long as possible. Currently we have reports about this issue (for various reasons) every few weeks and those are only the users reporting these issues somewhere and are not those that want to upgrade their system after e.g. one year of downtime.
tl;dr: I'm vibing way more with continuing to rely on the archlinux-keyring package exclusively; auto-updates are sus.
I understand that. However, maintaining a keyring reliably over a longer period of time is non-trivial. If the implemented service is not wanted by users, it is always possible to disable the unit by masking it. Best, David -- https://sleepmap.de
On 24/7/22 06:49, Evangelos Foutras wrote:
On Sat, 23 Jul 2022 at 19:39, David Runge <dave@sleepmap.de> wrote:
Packages that are signed with a key that still had marginal trust in release A (and therefore already existed on the user system since release A) and gained full trust in release B will not be updated before the user does a system upgrade. This leads to the requirement of installing archlinux-keyring before doing a system upgrade, as otherwise the key will still have marginal trust on the user system and the signatures of other updated packages using the key in question will fail to validate.
This is solvable by not cutting a release with marginally trusted keys. Having all Arch Linux installations make 100-ish requests daily to cover such an edge case is a misutilization of resources (on both sides). I'm certain there are also privacy concerns about enabling this service by default.
Not shipping keys that are marginally trusted is ideal in principle... However we have seen it many times recently as main/master keys were cycled. Hopefully this is less of an issue moving forward. This is why the keyring was set up to have five main keys, to ensure packager keys were signed by at least 3 at all times. However, not everyone has all five main key signatures. Or even 4. In fact, according to the web page [1], we are screwed if anything happens to Pierre's of Florian's keys, and to a lesser extent with other keys. I am not sure the extent that page reflects current reality, but note Pierre's key is schedule for retirement... (Is it correct that Giancarlo has signed zero keys? I thought they were bought on to replace me almost a year ago!) [1] https://archlinux.org/master-keys/ We have also had marginal trust happen when a new packager (or packager with a new key) starts packaging having gotten all relevant signatures, but not having their key in WKD yet. A version of the key is attempted to be retrieved from a key server that may not have any signatures. I think this is less of an issue going forward given WKD is updated alongside new keyring changes now. Finally, this deals with keys that have expiry dates. We have hit a few cases where expiry and updating keyring happens in a period where a user has not updated their system in a while. We could monitor this and have packagers be reminded to extend their expiry date at an early enough point in time. I suggest an update needs to be at least 6 months in advance of expiry. My conclusion is the reality of the keyring is not perfect. If all packager keys were signed by all main key holders, and WKD was updated before packages were released using a key, and if expiry dates were extended at least 6 months in advance, and .... , then this would not be an issue. David has put a lot of effort into the keyring to allow us to expose these issues more easily, and has been pushing for main key holders to increase their signing coverage, but this has not progressed at any great rate. The service suggestion really works around limitations in key signing activity. It would not be needed if main key holders signed everything and packagers with keys that expire updated their keys with extensive notice. Onto the service. If the server load and privacy issues were not issues, this would "solve" the problems above. Daily is probably too often, even weekly/monthly would be fine. But I guess the concerns raised mean we should focus on fixing the keyring instead (as futile as that may seem to David currently...). Allan
On 2022-07-23 23:30, David Runge wrote:
On 2022-07-23 21:24:38 (+0200), Kristian Klausen wrote:
The load aspect should be solvable, worst-case DevOps gets annoyed ;)
My main concern is putting ourselves in a position, where we know every arch installation, yes it is just the IP addresses, but still. At the same time it becomes easier to detect that a computer is running arch, by just look at the network traffic (yes you can already do that today, by checking if the computer is connecting to a arch mirror).
Maybe I'm blowing it out of proportion, but with our user base being more privacy aware than most people, I think it is worth mentioning.
Fair point, however, a similar concern was networkmanager doing a connectivity check [1]. It is up to us to configure our webserver accordingly and be as privacy conserving as possible.
E.g. If we wanted to track users (which we don't), we could do so on a rudimentary basis now already by tracking downloads of packager keys. The Web Key Directory is a way more centralistic approach than e.g. SKS was, but we also have better control over how and what we provide there.
Just wanna point out that just because we already do something it doesn't validate doing a similar thing. :) I'm fine with centralized WKD but only because of the absence of a functional key trust network. I do think that auto-key-updates are probably the wrong way forward as it sets a precedent of automatic network connectivity in the base system. NetworkManager's connectivity check can be argued is a core part of the software package in the same way that Firefox's is. I agree with Debian/Ubuntu setting automatic updates since it's a general-user-facing OS, but IMO Arch should remain more hands-on and empower the user to do the right thing but expect them to do it themselves. I do agree that the current situation of upgrading a system with an outdated keyring is annoying at best and very confusing at worst. Last year I kept receiving emails months after my old key expiration date. Updating the repo list and installing archlinux-keyring before updating the system breaks our own partial update rules as well. What about a pacman flag that hooks into keyringctl so that the upgrade command extends to e.g. pacman -Syuk? Much like how we don't have automatic package list updates, we can expect the user to update the key trust. If not integrated with pacman, why not just expect the user to update keys via keyringctl as part of the normal upgrade process? More useful error messages than "package may be invalid or corrupt" could point the user to know where the issue is and how best to solve it. Keep it simple. :)
Excerpts from David Runge's message of July 23, 2022 18:38:
Currently the timer which triggers this service is supposed to be vendor enabled (i.e. symlink in /usr/lib/systemd/system/timers.target.wants/) and run daily with a deviation of up to 12h. Members of the DevOps team have raised concerns about the interval as we do not really have a clear picture about how many Arch Linux installations there are world-wide to get numbers on expected median load for this.
I'm fine with it existing, and I would be fine with it being enabled in a vendor preset, but I'm against it being statically enabled in /usr. This is not something that's critical for the regular functioning of the system, and so is not something I think users should have to mask to get rid of rather than be able to just disable it. But daily seems a bit frequent to me. Anything more frequent than weekly feels too often to me, and even that feels a bit frequent. -- Sincerely, Johannes Löthberg :: SA0DEM :: he/him
On 24/07/2022 20:56, Johannes Löthberg wrote:
Excerpts from David Runge's message of July 23, 2022 18:38:
Currently the timer which triggers this service is supposed to be vendor enabled (i.e. symlink in /usr/lib/systemd/system/timers.target.wants/) and run daily with a deviation of up to 12h. Members of the DevOps team have raised concerns about the interval as we do not really have a clear picture about how many Arch Linux installations there are world-wide to get numbers on expected median load for this.
I'm fine with it existing, and I would be fine with it being enabled in a vendor preset, but I'm against it being statically enabled in /usr. This is not something that's critical for the regular functioning of the system, and so is not something I think users should have to mask to get rid of rather than be able to just disable it.
But daily seems a bit frequent to me. Anything more frequent than weekly feels too often to me, and even that feels a bit frequent.
I'm aligning probably against such systemd unit. Mainly because or privacy issues and given the options users will probably set something like refresh every 1 hour and then we need to go into rate-limiting just to keep our gitlab up. -- Leonidas Spyropoulos PGP: 59E43E106B247368
On 2022-07-24 10:53:24 (-0700), Brett Cornwall wrote:
Just wanna point out that just because we already do something it doesn't validate doing a similar thing. :)
No. That was not the point I was trying to make. :) It serves as a good example as to why such a thing does not have to be a privacy disaster automatically. After all, we are already dealing with this scenario (hopefully in a successful way).
I'm fine with centralized WKD but only because of the absence of a functional key trust network. I do think that auto-key-updates are probably the wrong way forward as it sets a precedent of automatic network connectivity in the base system. NetworkManager's connectivity check can be argued is a core part of the software package in the same way that Firefox's is. I agree with Debian/Ubuntu setting automatic updates since it's a general-user-facing OS, but IMO Arch should remain more hands-on and empower the user to do the right thing but expect them to do it themselves.
If there is no network connectivity available, there is no keyring update and there is also no package upgrade. To run Arch a user will likely (at some point) want to upgrade the system. FWIW, the use-case of automatically updating valid and existing keys to allow to upgrade the system normally is a different from automatically upgrading packages. Quite objectively it will be impossible to prevent less proficient or new users in the current setup to not run into an issue with our keyring and ask the same questions for the millionth time (or give up). This would all be fine as long as noone else had to care about this, but affected packagers and keyring maintainers *will be* contacted by those users.
I do agree that the current situation of upgrading a system with an outdated keyring is annoying at best and very confusing at worst. Last year I kept receiving emails months after my old key expiration date.
This is indeed unfortunately quite common and I am sure I will be contacted many more times over the next few months myself. Sure, I could just start ignoring those messages, but that is neither nice nor does it prevent me from spending time on reading a message half-way to figure out that it has to do with this issue. ;-) IMHO, trying to solve this issue with the approach of upgrading the keyring first, is solving a technological problem with "tribal knowledge" (you just "have to know" that this is an issue with the most central and common of all actions on this operating system to not be affected). This does not scale and it creates friction for users and contributors.
What about a pacman flag that hooks into keyringctl so that the upgrade command extends to e.g. pacman -Syuk? Much like how we don't have automatic package list updates, we can expect the user to update the key trust. If not integrated with pacman, why not just expect the user to update keys via keyringctl as part of the normal upgrade process? More useful error messages than "package may be invalid or corrupt" could point the user to know where the issue is and how best to solve it.
Keep it simple. :)
This may sound simple at first, but would in fact not be as trivial as one might think. The keyring provided by archlinux-keyring provides old keys (e.g. revoked by issuer or with revoked signatures) and keys (and UIDs!) using non archlinux.org domains. All of those need to be ignored as they are not relevant for (Arch Linux) packages and would lead to many more queries to our and the WKDs of others. To integrate a "sync all locally available keys before upgrade" functionality in pacman, it would need to implement a configurable(!) subsystem which implements the above filtering mechanism domain agnostic (as the package manager is used by others as well). Not saying that this can not be done (it is what the proposed service does standalone), but I know that this will likely not be trivial in the context of pacman and that I do not believe I would be able to write it myself. FTR: Keyringctl is currently very tightly integrated with archlinux-keyring (upstream). It is not really a user-facing application yet (there is no separate upstream for it) and serves the purpose of allowing us to manage the keyring in such a way that it can be maintained in a git repository as source of truth. Best, David -- https://sleepmap.de
On 2022-07-24 21:56:28 (+0200), Johannes Löthberg wrote:
I'm fine with it existing, and I would be fine with it being enabled in a vendor preset, but I'm against it being statically enabled in /usr. This is not something that's critical for the regular functioning of the system, and so is not something I think users should have to mask to get rid of rather than be able to just disable it.
At the time of writing only fwupd [1] and systemd [2] make use of systemd preset [3]. Could you elaborate on what you see as the fundamental difference (from a user's perspective) between `systemctl disable --now <unit>` and `systemctl mask --now <unit>`? My reasons for the proposed vendor enabling would be: * we are the vendor * no complicated actions required in an .install file of archlinux-keyring to add a file to the administrative layer (/etc/systemd/) of a user system (if not using a systemd preset file) * the disabling of a default unit is obvious to the user (by the existence of the /dev/null symlink) * the unit's activation state survives damaging/removing the administration layer
But daily seems a bit frequent to me. Anything more frequent than weekly feels too often to me, and even that feels a bit frequent.
I have been looking at this issue from the perspective of "how can a user system gain (mostly) functional keys before upgrading?" by looking at these two scenarios: 1. "A key gains full trust and locally has marginal trust" 2. "A key's expiry date has been extended and is locally expired" In 1. the duration between release of a new keyring to [core] and release of a package that is signed by the now fully trusted key in any of our repositories can be minutes and therefore may even affect systems that are upgraded every few hours or once daily. With a daily timer we could eradicate the issue in this scenario with a rule of "only add packages signed with a key that gains full trust in a new version of archlinux-keyring after one day of its move to [core]" (that is still a big "if", as it relies on us doing the right thing). In 2. let's assume that the release of a new archlinux-keyring is made at least two days before expiry of the key (FWIW, we've had worse!). The *persistent* daily timer (e.g. also directly triggered after having the machine powered off for a longer while) would synchronize the keys very likely before a system upgrade can be attempted. Again, this assumes that we catch the expiry early enough (for which now there is a proposed fix available, thanks to Brett [4] - thanks again for working on that!) for systems being upgraded in short intervals (e.g. daily) and do not have to emergy rebuild someone's packages (which is usually also already too late for some users). Best, David [1] https://archlinux.org/packages/community/x86_64/fwupd/ [2] https://archlinux.org/packages/core/x86_64/systemd/ [3] https://man.archlinux.org/man/systemd.preset.5 [4] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/merge_requests/12... -- https://sleepmap.de
On 2022-07-25 09:23:53 (+0100), Leonidas Spyropoulos wrote:
I'm aligning probably against such systemd unit. Mainly because or privacy issues and given the options users will probably set something like refresh every 1 hour and then we need to go into rate-limiting just to keep our gitlab up.
As mentioned in another mail [1], I believe that the privacy issues and the approach towards them do not differ from the existing setup with networkmanager and pacman querying the WKD for new keys before upgrade. On IRC a high load on the gitlab pages runner serving the WKD has been discussed. If this indeed affects the performance of gitlab (now already?), then gitlab pages is not the right tool to serve the WKD. Given that we still lack numbers on the current load and the to be expected future load, I'd suggest starting out with a longer interval than daily (e.g. weekly), although this will only solve the issues as discussed in my reply to Johannes [2] if we ensure, that packagers wait one week from gaining full trust to their first package release. Best, David [1] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.or... [2] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.or... -- https://sleepmap.de
On Sat, 2022-07-23 at 18:38 +0200, David Runge wrote:
Hi all,
I currently have a MR open against archlinux-keyring [1], that adds a systemd service and timer, which would automatically refresh valid and existing keys on user systems. For people without access to the repository's ticket and merge request features, it is possible to browse the commits in question on the respective branch [2].
To quote from the accompanying ticket [3]: ``` For situations in which we do have new signatures which have been released, we may want a service on user systems that update existing keys via WKD to have an updated set of keys before updating archlinux-keyring.
Rationale: Keys are updated in WKD after a release of archlinux-keyring (currently this is still done in the https://gitlab.archlinux.org/archlinux/wkd repo). User systems may upgrade any time after that and pacman will pull new keys from WKD automatically, however it will not pull updates for existing keys. Packages that are signed with a key that still had marginal trust in release A (and therefore already existed on the user system since release A) and gained full trust in release B will not be updated before the user does a system upgrade. This leads to the requirement of installing archlinux-keyring before doing a system upgrade, as otherwise the key will still have marginal trust on the user system and the signatures of other updated packages using the key in question will fail to validate.
To remedy this situation I think it would be helpful to have a service update existing keys from WKD twice daily on a timer.
I am not 100% sure whether this has side-effects in regards to revocations, but as we only ever update WKD after issuing a release and not when updating the default branch, this should be fine(?). ```
In short: The service in question will update any valid keys in the pacman keyring on user systems, that use Arch Linux's relevant domains as UID to retrieve updated keys (e.g. extended expiration, additional signatures). Invalid keys and those of other domains are ignored.
This feature is implemented to circumvent (some, but not all) cases of "update archlinux-keyring before doing a system upgrade". Cases not covered are (in no particular order): * *long* outdated system is updated * new main signing key is required for a packager key to gain full trust for the packages about to be updated * local system's archlinux-keyring does not yet offer this feature * local system's pacman does not yet offer the feature of downloading package signatures for new keys from WKD * packager key on local system does not yet have any main key signatures and would gain full trust with an update of archlinux-keyring
Currently the timer which triggers this service is supposed to be vendor enabled (i.e. symlink in /usr/lib/systemd/system/timers.target.wants/) and run daily with a deviation of up to 12h. Members of the DevOps team have raised concerns about the interval as we do not really have a clear picture about how many Arch Linux installations there are world-wide to get numbers on expected median load for this.
If you have ideas for improvement or general questions, please direct them at the merge request.
Best, David
[1] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/merge_requests/13... [2] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/tree/wkd_sync_ser... [3] https://gitlab.archlinux.org/archlinux/archlinux-keyring/-/issues/184
I am +1 on this, but I don't think this is the cleanest solution, and similar to others, I think the frequency should be lowered. Alan, is there any reason pacman couldn't update the keys on sync operations, avoiding this issue altogether? Cheers, Filipe Laíns
On 2022-07-24 12:23:29 (+1000), Allan McRae wrote:
Not shipping keys that are marginally trusted is ideal in principle... However we have seen it many times recently as main/master keys were cycled. Hopefully this is less of an issue moving forward. This is why the keyring was set up to have five main keys, to ensure packager keys were signed by at least 3 at all times. However, not everyone has all five main key signatures. Or even 4. In fact, according to the web page [1], we are screwed if anything happens to Pierre's of Florian's keys, and to a lesser extent with other keys. I am not sure the extent that page reflects current reality, but note Pierre's key is schedule for retirement...
The likelyhood for a packager to have three signatures or more (fast) has not been very high over the past year. However, the website is also only showing the signatures correctly, if any given packager setup their current (e.g. new) key in their archweb profile. As notified about earlier [a][b] this is currently not the case and renders the overview a bit useless. Tbh, it would make more sense to automate the population of that subpage and make it completely unreliant on user input, as packagers may have more than one key at a given time. For figuring out current trust (in the yet unreleased keyring), it is possible to list keys with given trust level using `./keyringctl` (see e.g. `./keyringctl list -h`) in the archlinux-keyring repository.
(Is it correct that Giancarlo has signed zero keys? I thought they were bought on to replace me almost a year ago!)
Unfortunately yes. I have asked about this probably a dozen times and given up due to lack of action by now.
My conclusion is the reality of the keyring is not perfect. If all packager keys were signed by all main key holders, and WKD was updated before packages were released using a key, and if expiry dates were extended at least 6 months in advance, and .... , then this would not be an issue.
That is a lot of "ifs" :)
David has put a lot of effort into the keyring to allow us to expose these issues more easily, and has been pushing for main key holders to increase their signing coverage, but this has not progressed at any great rate. The service suggestion really works around limitations in key signing activity. It would not be needed if main key holders signed everything and packagers with keys that expire updated their keys with extensive notice.
Onto the service. If the server load and privacy issues were not issues, this would "solve" the problems above. Daily is probably too often, even weekly/monthly would be fine. But I guess the concerns raised mean we should focus on fixing the keyring instead (as futile as that may seem to David currently...).
After reading some of the responses (e.g. Brett's [c] and Filipe's [d]) it might make indeed more sense to envision a system in which we grab updated keys (if we need them) using pacman upon system upgrade (I guess this action can not be paired with that of pulling new keys though as that happens upon package validation AFAIK). This would lower the amount of requests significantly if no update is needed (i.e. only one HTTP request is needed to retrieve a timestamp file) and only update keys if there is a new version of the keyring data deployed to WKD (e.g. timestamp is newer, we are synchronizing the existing keys). Even if we went with the service, a timestamp file would make sense there as well. As mentioned in my reply to Brett [e] I am not sure how to achieve that in the context of pacman and I do see a configuration overhead there that needs addressing, but if you could point me in the right direction, I might be able to figure it out! :) Best, David [a] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.or... [b] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.or... [c] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.or... [d] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.or... [e] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.or... -- https://sleepmap.de
participants (8)
-
Allan McRae
-
Brett Cornwall
-
David Runge
-
Evangelos Foutras
-
Filipe Laíns
-
Johannes Löthberg
-
Kristian Klausen
-
Leonidas Spyropoulos