On 8/11/20 9:24 AM, Allan McRae wrote:

On 11/8/20 7:44 am, Eli Schwartz wrote:

...

On 8/10/20 5:34 PM, Anatol Pomozov wrote:

...

Switching from embedded to detached signatures is a big change. This feature needs to be thoroughly tested before embedded signatures will be completely removed from the database.

To help with detached signatures testing we enable it by default. But in case if an user needs to go back to embedded signatures we add a config option to reenable it - "UseEmbeddedSignatures". What is the purpose of this? Either signature source should be equivalent, and you should be able to trivially test this by creating a repo with unsigned packages, then bulk-signing the packages after they were repo-added. I don't believe that pacman should include such an end-user option purely to double-check whether a current feature actually works.

Agreed - the user should not care where the signatures come from, so this option should not exist.

Also, I see this was proposed on arch-dev-public first. I am not subscribed there, and decisions on what is included in pacman are not dictated by Arch Linux. Proposals should be posted here.

More specifically -- decisions on what is included in pacman are not dictated by consensus of the Arch Linux team, but by the pacman team (which is in turn guided, but not dictated, by what is useful for archlinux). Making a bad or confusing package manager simply because archlinux wants it, would be a bad move due to making a bad or confusing package manager.

Now, thinking out loud here... Would an alternative be to add an "--embed-signatures" option to repo-add? So two versions of a repo could be created and those that want to test without embedded signatures can.

This is the right approach, yeah. I was thinking we'd wait until pacman 6.1 before stopping the signature embedding, to provide a transition period for people depending on SigLevel = Required (which should be everyone, and certainly includes Arch!) to upgrade to 6.x before repo-add starts generating databases useless to pacman 5.x But I'd also be fine with --no-embed-signatures for opting in early, and switching to --embed-signatures for opting out once we default to --no-* -- Eli Schwartz Bug Wrangler and Trusted User

On 27/8/20 10:26 am, Anatol Pomozov wrote:

Hi

On Mon, Aug 10, 2020 at 2:45 PM Eli Schwartz wrote:

...

On 8/10/20 5:34 PM, Anatol Pomozov wrote:

...

Switching from embedded to detached signatures is a big change. This feature needs to be thoroughly tested before embedded signatures will be completely removed from the database.

To help with detached signatures testing we enable it by default. But in case if an user needs to go back to embedded signatures we add a config option to reenable it - "UseEmbeddedSignatures". What is the purpose of this? Either signature source should be equivalent,

Indeed the signatures are equivalent. The only difference whether they are stored inside the database file or as *.sig next to the packages itself.

...

and you should be able to trivially test this by creating a repo with unsigned packages, then bulk-signing the packages after they were repo-added. I don't believe that pacman should include such an end-user option purely to double-check whether a current feature actually works.

The purpose of the change is to start using the detached signatures codepath. The detached signatures are shipped with repos for a long time and pacman can handle it. Now it is time to actually enable it by default.

"UseEmbeddedSignatures" option has been added as a fallback plan in case we find that the detached signatures codepath is broken. Do you think this is too much hassle and we should just start using detached signatures by default without any fallback config option?

I think we should test using detached signatures, and release without pacman having a fallback option in and of itself. The fallback option should be in repo-add, whether it adds the signatures to the database or not.

...

This is the right approach, yeah. I was thinking we'd wait until pacman 6.1 before stopping the signature embedding, to provide a transition period for people depending on SigLevel = Required (which should be everyone, and certainly includes Arch!) to upgrade to 6.x before repo-add starts generating databases useless to pacman 5.x

There are 2 sets of changes that need to be done: 1) make pacman to use detached signatures instead of embedded ones 2) change "repo-add" to avoid adding embedded signatures

We should release changes for #1 first, test it, make sure that detached signatures fully work (while dbs still have pacman 5.x-compatible embedded sigs). And only then release #2 to get smaller databases compatible with pacman version >= 6.0.

I was thinking #1 can be released with 6.0 and #2 with 6.1.

I was thinking #2 would be an option to repo-add. I'm looking at making signature embedding only occur with the "--add-signatures" option (or whatever I decide to call it). Arch would need to patch devtools to use this option. They would then make a News announcement about the need to have pacman-6.0 installed after 3-6 months and stop repo-add including signatures. However, I think pacman should always use the signatures in the database if they are present. Particularly if they are not embedded by default. So to actually test the detached signature path, I am thinking it best to tag 6.0.0beta1, make a package from that tag with a patch to enable using detached signatures as a priority. While that is not an ideal approach to testing, I think the current code path is well tested, and this should be a reasonably trivial patch. Allan

On 8/26/20 8:26 PM, Anatol Pomozov wrote:

The purpose of the change is to start using the detached signatures codepath. The detached signatures are shipped with repos for a long time and pacman can handle it. Now it is time to actually enable it by default.

"UseEmbeddedSignatures" option has been added as a fallback plan in case we find that the detached signatures codepath is broken. Do you think this is too much hassle and we should just start using detached signatures by default without any fallback config option?

I already stated my opinion. Firstly, that we should NOT add a configuration option for this, since it is a burden on the manpage and is completely useless except for development testing. Secondly, I believe we should continue to check both, because I see no compelling reason to reject perfectly working functionality for no reason, plus you have no way of knowing if there are thirdparty repositories which locally generate databases with sigs, but then only upload the packages and databases, but not the sigs (on the grounds that they won't be used so why bother). We don't just care about Arch Linux. Also, we don't just care about the official repos, even for Arch Linux use. Before instituting a breaking change, we need a better reason than "this is a convenient way for a pacman developer to test whether or not pacman is broken".

...

This is the right approach, yeah. I was thinking we'd wait until pacman 6.1 before stopping the signature embedding, to provide a transition period for people depending on SigLevel = Required (which should be everyone, and certainly includes Arch!) to upgrade to 6.x before repo-add starts generating databases useless to pacman 5.x

There are 2 sets of changes that need to be done: 1) make pacman to use detached signatures instead of embedded ones 2) change "repo-add" to avoid adding embedded signatures

We should release changes for #1 first, test it, make sure that detached signatures fully work (while dbs still have pacman 5.x-compatible embedded sigs). And only then release #2 to get smaller databases compatible with pacman version >= 6.0.

I was thinking #1 can be released with 6.0 and #2 with 6.1.

My vote is to not do #1 at all. I do not see why you keep insisting it "needs" to be done. #2 is all we need to generate test repositories. Here is a script to generate test repositories: ssh pkgbuild.com cd public_html/repo mkdir x86_64-detachedsigs cd mkdir x86_64-detachedsigs bsdtar -xOf ../x86_64/eschwartz.db.tar.gz | awk '/^%FILENAME%/{getline;print}' | while read -r line; do cp -v ../x86_64/"$line" ./ done repo-add eschwartz.db.tar.gz *.pkg.tar* bsdtar -xOf ../x86_64/eschwartz.db.tar.gz | awk '/^%FILENAME%/{getline;print}' | while read -r line; do cp -v ../x86_64/"$line".sig ./ done Here is a test repo you can verify against: [eschwartz] Server = https://pkgbuild.com/~eschwartz/repo/x86_64-detachedsigs/ -- Eli Schwartz Bug Wrangler and Trusted User