[pacman-dev] Package checking options (was: Changed all references to signature verification level in libalpm symbols to 'verifysig'.)
On Sun, Jun 12, 2011 at 3:15 AM, Kerrick Staley <mail@kerrickstaley.com> wrote:
I inadvertently broke my email account when I configured git-send-mail, and I only just realized it. Everything I've sent to the list since then didn't go through, so I'm resending this message, originally sent June 5 (I edited/updated it).
Whatever we call the variable doesn't really matter; I like SigLevel.
I'm working on a patch that will implement the below behavior. It's just taking a while because it's somewhat complicated.
========
Config:
# Action to take (warn or abort) when packages and databases are unsigned but were previously signed SigPreviouslySigned = warn
# Action to take (accept, warn, or abort) when neither the package nor the database have a signature SigNoSignature = warn
# Action to take (accept, warn, or abort) when a package is signed but the database is unsigned # SigDatabaseUnsigned = accept
# Action to take (accept, warn, or abort) when a package is unsigned but has a SHA256 hash in a signed database # SigHashed = accept
This is absurd as far as configuration goes. Why would we have 126 separate directives for what boils down to one thing, which as Allan and I said, could be called "CheckLevel"? ~~~~ CheckLevel = Required SignedPackage SignedDatabase NoAllowUnknown [another-repo] CheckLevel = Optional [someones-repo] CheckLevel = NoSignedDatabase AllowUnknown [insecureRepo] CheckLevel = Never ~~~~ SigPreviouslySigned- an interesting thought, but definitely not something we need on first pass- there are also a multitude of issues here considering everyone starts from a different point. If you want to be protected against this kind of attack, just use Required/Always/whatever we call it. SigHashed- covered by my previous suggestion; here it would be "CheckLevel = Required SignedDatabase NoSignedPackage"
========
A) When handling a given situation, there are three actions that can be taken: 1. The situation is accepted silently. 2. The situation is accepted with a warning. 3. Installation/update is aborted, unless an override flag is passed, in which case a warning will be given.
B) For a database, consider whether it should have a signature: 1. If SigDatabaseUnsigned = abort, but the .sig file cannot be downloaded, then keep checking servers until a .sig can be gotten, and give a warning each time a .sig download returns a 404. 2. If not (1) but, based on the various config settings, having an unsigned database would result in more warnings or aborts than a signed one, then check servers for a .sig until you get a .sig or you get 3 404's; if you get 3 404's, rescan through the list from the beginning, not checking for .sig's this time. Again, give a warning whenever a 404 happens. 3. If not (1)/(2), don't check for .sig's.
C) For a package, consider whether it should have a signature: 1. If the package was previously signed but now isn't, act based on SigPreviouslySigned. 2. For local packages, if the package indicates it should have a signature (mechanism to be implemented) but it doesn't, abort.
D) Outcomes for package/database signature validity ("signed" means GPGME_SIGSUM_VALID w/ GPGME_VALIDITY_{FULL,ULTIMATE}; "unsigned" means GPGME_SIGSUM_KEY_MISSING or no signature present, "bad" means anything else): 1. The package or database or both have bad signatures. Abort (could happen during database unpacking; the rest only happen on package check). 2. The package and database are signed. Accept. 3. The package is unsigned but the database is signed and the database uses SHA256. Act based on SigHashed. 4. The package is signed but the database is unsigned. Act based on SigDatabaseUnsigned. 5. The package is unsigned and the database is unsigned. Act based on SigNoSignature.
========
Notes:
I invented the config names more or less randomly; if you have strong objections, we'll change them, otherwise let's go with these. I have objections on your way of going about business here. I present what i thought was a reasonable approach, got some feedback by Allan, and then you attempt to throw something completely different at us without really responding to what is wrong with my proposal.
I'm trying my hardest to not get seen as a stick in the mud and impediment to progress here, but it is damn hard to keep my cool when my years of experience with this codebase gets shoved aside without so much as a "I don't think you considered this" bit of feedback. My ideas tend to be backed by those years of experience, and adding config options is not something we like to do willy-nilly, nor do we want a complicated config file.
I've made pmsiglevel_t into a struct with 4 enum members; I'm currently passing it around by value (not with pointers); please give me a heads-up if you don't think I should do it this way. The usual preference in C is to use bitflags for things like this.
No marginal signatures should come up, because I don't think we should use such signatures during the interim when not all developers have fully validated keys; a developer's key can just be excluded from pacman-keyring until it is fully validated. That is a pretty bold assumption. You do realize there are more repos out there that are not provided by a distro that people might want to use and have signatures involved, right?
-Dan
On Mon, Jun 13, 2011 at 6:13 PM, Dan McGee <dpmcgee@gmail.com> wrote:
I have objections on your way of going about business here. I present what i thought was a reasonable approach, got some feedback by Allan, and then you attempt to throw something completely different at us without really responding to what is wrong with my proposal.
I'm trying my hardest to not get seen as a stick in the mud and impediment to progress here, but it is damn hard to keep my cool when my years of experience with this codebase gets shoved aside without so much as a "I don't think you considered this" bit of feedback. My ideas tend to be backed by those years of experience, and adding config options is not something we like to do willy-nilly, nor do we want a complicated config file.
Sorry. I often prefer to write in a very direct way (e.g. less instances of "I think...", "I feel...", etc.), even when I really mean to make a suggestion, because it is terser. I thought it was appropriate in this situation because it's a feature I really want to see implemented, and while "We should..." tends to generate a response along the lines of either "yes" or "no, do this", "Should we..." tends to generate a 20+ message thread. While this isn't a bad thing, I thought it'd be better to just propose a fully-thought-out, workable scheme from the start, so that it could be built off of. I spent several hours devising this scheme and ensuring that it covers all functionality that I thought would be useful, without requiring too much configuration. Anyway, I think my writing style is somewhat in poor taste; I'll try to tone it down. I feel like the config file syntax is mostly a trivial issue anyway; it should fall into place once we have decided (a) which situations we should handle specially, and (b) which actions can be taken in those situations. I feel that having separate directives for each case would be cleaner, but the "CheckLevel" syntax works as well, and since it's a question of conventions, it's squarely your call (as is, ultimately, everything else I suggest).
No marginal signatures should come up, because I don't think we should use such signatures during the interim when not all developers have fully validated keys; a developer's key can just be excluded from pacman-keyring until it is fully validated. That is a pretty bold assumption. You do realize there are more repos out there that are not provided by a distro that people might want to use and have signatures involved, right?
OK, you're right.
SigPreviouslySigned- an interesting thought, but definitely not something we need on first pass- there are also a multitude of issues here considering everyone starts from a different point.
Yeah, I wasn't even planning on implementing it at first; I just thought it'd be something we wanted in the long term, so I put it in among the rest. On Wed, Jun 1, 2011 at 3:46 PM, Dan McGee <dpmcgee@gmail.com> wrote:
I also think we may need to be a bit more granular than our current Always/Optional/Never trifecta. We have a multitude of possibilities when checking a signature:
* Valid signature, fully trusted (or ultimate, (GPGME_VALIDITY_{ULTIMATE, FULL}).
* Valid signature, unknown trust/unknown key (GPGME_VALIDITY_UNKNOWN, GPGME_SIGSUM_KEY_MISSING). * Valid signature, trust somewhere in between (marginal, GPGME_VALIDITY_MARGINAL).
* Valid signature, user is never valid (GPGME_VALIDITY_NEVER) * Valid signature, signature is however expired (GPGME_SIGSUM_SIG_EXPIRED) * Valid signature, key is however expired (GPGME_SIGSUM_KEY_EXPIRED) * Bad signature, trust level is irrelevant (GPGME_VALIDITY_RED)
It seems that the first 3 in the last group are corner cases (I could not, for the life of me, find a real example of when GPGME_VALIDITY_NEVER would come up). For the middle two, having a signature with an unknown or nonexistent key doesn't really say anything about the validity of the package, so I think that outcome should just lumped in among the last 4. So, then, this boils down to "the signature is good", "the signature is bad", or "the signature is marginally trusted", but I think there are other cases (hashed, database unsigned) that should be considered. -Kerrick Staley
On Tue 14 June 2011 at 00:47 -0500, Kerrick Staley wrote:
On Wed, Jun 1, 2011 at 3:46 PM, Dan McGee <dpmcgee@gmail.com> wrote:
I also think we may need to be a bit more granular than our current Always/Optional/Never trifecta. We have a multitude of possibilities when checking a signature:
* Valid signature, fully trusted (or ultimate, (GPGME_VALIDITY_{ULTIMATE, FULL}).
* Valid signature, unknown trust/unknown key (GPGME_VALIDITY_UNKNOWN, GPGME_SIGSUM_KEY_MISSING). * Valid signature, trust somewhere in between (marginal, GPGME_VALIDITY_MARGINAL).
* Valid signature, user is never valid (GPGME_VALIDITY_NEVER) * Valid signature, signature is however expired (GPGME_SIGSUM_SIG_EXPIRED) * Valid signature, key is however expired (GPGME_SIGSUM_KEY_EXPIRED) * Bad signature, trust level is irrelevant (GPGME_VALIDITY_RED)
It seems that the first 3 in the last group are corner cases (I could not, for the life of me, find a real example of when GPGME_VALIDITY_NEVER would come up). For the middle two, having a signature with an unknown or nonexistent key doesn't really say anything about the validity of the package, so I think that outcome should just lumped in among the last 4. So, then, this boils down to "the signature is good", "the signature is bad", or "the signature is marginally trusted", but I think there are other cases (hashed, database unsigned) that should be considered.
I'm not sure about what you are saying: we have multiple axes: - the signature is valid/invalid/expired/there is none - the key is expired/has trust level X (where X can be any value, including "I absolutely hate that guy and swear I won't ever install any package signed by him") So of course, formally we want to reduce this to a mere boolean value, but people may want to configure how that happens. Rémy.
On Mon, Jun 13, 2011 at 6:13 PM, Dan McGee <dpmcgee@gmail.com> wrote:
I have objections on your way of going about business here. I present what i thought was a reasonable approach, got some feedback by Allan, and then you attempt to throw something completely different at us without really responding to what is wrong with my proposal.
I'm trying my hardest to not get seen as a stick in the mud and impediment to progress here, but it is damn hard to keep my cool when my years of experience with this codebase gets shoved aside without so much as a "I don't think you considered this" bit of feedback. My ideas tend to be backed by those years of experience, and adding config options is not something we like to do willy-nilly, nor do we want a complicated config file.
Sorry. I often prefer to write in a very direct way (e.g. less instances of "I think...", "I feel...", etc.), even when I really mean to make a suggestion, because it is terser. I thought it was appropriate in this situation because it's a feature I really want to see implemented, and while "We should..." tends to generate a response along the lines of either "yes" or "no, do this", "Should we..." tends to generate a 20+ message thread. While this isn't a bad thing, I thought it'd be better to just propose a fully-thought-out, workable scheme from the start, so that it could be built off of. I spent several hours devising this scheme and ensuring that it covers all functionality that I thought would be useful, without requiring too much configuration.
Anyway, I think my writing style is somewhat in poor taste; I'll try to tone it down. I'll eat some of my aggressive words as well, I'm sorry. I just come back from taking a break over the weekend and see my own time spent
On Tue, Jun 14, 2011 at 12:47 AM, Kerrick Staley <mail@kerrickstaley.com> wrote: thinking, analyzing the situation, and coding seemingly get tossed aside so I responded with some attitude.
I feel like the config file syntax is mostly a trivial issue anyway; it should fall into place once we have decided (a) which situations we should handle specially, and (b) which actions can be taken in those situations. I feel that having separate directives for each case would be cleaner, but the "CheckLevel" syntax works as well, and since it's a question of conventions, it's squarely your call (as is, ultimately, everything else I suggest).
No marginal signatures should come up, because I don't think we should use such signatures during the interim when not all developers have fully validated keys; a developer's key can just be excluded from pacman-keyring until it is fully validated. That is a pretty bold assumption. You do realize there are more repos out there that are not provided by a distro that people might want to use and have signatures involved, right?
OK, you're right.
SigPreviouslySigned- an interesting thought, but definitely not something we need on first pass- there are also a multitude of issues here considering everyone starts from a different point.
Yeah, I wasn't even planning on implementing it at first; I just thought it'd be something we wanted in the long term, so I put it in among the rest.
On Wed, Jun 1, 2011 at 3:46 PM, Dan McGee <dpmcgee@gmail.com> wrote:
I also think we may need to be a bit more granular than our current Always/Optional/Never trifecta. We have a multitude of possibilities when checking a signature:
* Valid signature, fully trusted (or ultimate, (GPGME_VALIDITY_{ULTIMATE, FULL}).
* Valid signature, unknown trust/unknown key (GPGME_VALIDITY_UNKNOWN, GPGME_SIGSUM_KEY_MISSING). * Valid signature, trust somewhere in between (marginal, GPGME_VALIDITY_MARGINAL).
* Valid signature, user is never valid (GPGME_VALIDITY_NEVER) * Valid signature, signature is however expired (GPGME_SIGSUM_SIG_EXPIRED) * Valid signature, key is however expired (GPGME_SIGSUM_KEY_EXPIRED) * Bad signature, trust level is irrelevant (GPGME_VALIDITY_RED)
It seems that the first 3 in the last group are corner cases (I could not, for the life of me, find a real example of when GPGME_VALIDITY_NEVER would come up). TRUST_NEVER seems to only come up if you do a gpg --edit-key, "trust", and then choose "never" as your level of trust for a given key, from what I can see (option 2).
Please decide how far you trust this user to correctly verify other users' keys (by looking at passports, checking fingerprints from different sources, etc.) 1 = I don't know or won't say 2 = I do NOT trust 3 = I trust marginally 4 = I trust fully 5 = I trust ultimately m = back to the main menu
For the middle two, having a signature with an unknown or nonexistent key doesn't really say anything about the validity of the package, so I think that outcome should just lumped in among the last 4. So, then, this boils down to "the signature is good", "the signature is bad", or "the signature is marginally trusted", but I think there are other cases (hashed, database unsigned) that should be considered.
Looking at this last night, I came up with something like the following. Basically the first enum is what we have now, but moving toward a more standard name. The real change comes with the second enum, which unlike the first, will be bitflags (I just didn't assign values there yet). checklevel works more or less as it does currently. unknown falls back to the globally set default, never/optional/always are as expected. checkoptions basically controls where we perform checks and what we consider valid. Potential API example: alpm_db_set_checklevel(PM_CHECK_LEVEL_ALWAYS, PM_CHECK_PACKAGE_SIGNATURES | PM_CHECK_DATABASE_SIGNATURES | PM_CHECK_MARGINAL_SIGNATURES_OK); This particular call would enforce that signatures are present for packages AND databases. We would however allow marginally trusted sigs (but not unknown ones). For simplicity, the following rules would also apply, at least initially: * VALID_NEVER, VALIDITY_RED are always failures. * SIG_EXPIRED, KEY_EXPIRED are always failures (we could add a PM_CHECK_EXPIRED_SIGNATURES_OK flag). * Even if PM_CHECK_PACKAGE_SIGNATURES is not provided, we would still check it if present and level is at least OPTIONAL. typedef enum _pmchecklevel_t { PM_CHECK_LEVEL_UNKNOWN = 0, PM_CHECK_LEVEL_NEVER = 1, PM_CHECK_LEVEL_OPTIONAL, PM_CHECK_LEVEL_ALWAYS } pmchecklevel_t; typedef enum _pmcheckoptions_t { PM_CHECK_PACKAGE_SIGNATURES, PM_CHECK_DATABASE_SIGNATURES, PM_CHECK_UNKNOWN_SIGNATURES_OK, PM_CHECK_MARGINAL_SIGNATURES_OK } pmcheckoptions_t; I think this API covers most of the bases we are looking to address. The biggest place we differ is in the accept/warn/abort stuff you proposed. For me I am not a fan of this, only because I know users don't actually read. :/ Feedback welcome, let's work to a reasonable consensus and I should be able to knock this out real quick. Oh, and Kerrick, I don't know if you've figured it out by now but my hacks are usually available in my repo, right now they are on my working branch. http://code.toofishes.net/cgit/dan/pacman.git/log/?h=working. We should probably make a wiki page of our development repo URLs so people can find them. -Dan
I should put in that I'm here for the experience as much as for the sake of implementing this feature. I'd been kicking around the idea of starting a (perhaps ambitious) FOSS project of my own, but since I'd never so much as sent a patch to an existing project before, I thought that was a little foolhardy to set out with no experience and decided to work on pacman because I love Arch, but unsigned packages is a downer for me. So, if there's ever a point where you feel I'm out of line, then by all means say something; the more emphatic, the better. -Kerrick Staley So by all means, On Tue, Jun 14, 2011 at 11:03 AM, Dan McGee <dpmcgee@gmail.com> wrote:
On Mon, Jun 13, 2011 at 6:13 PM, Dan McGee <dpmcgee@gmail.com> wrote:
I have objections on your way of going about business here. I present what i thought was a reasonable approach, got some feedback by Allan, and then you attempt to throw something completely different at us without really responding to what is wrong with my proposal.
I'm trying my hardest to not get seen as a stick in the mud and impediment to progress here, but it is damn hard to keep my cool when my years of experience with this codebase gets shoved aside without so much as a "I don't think you considered this" bit of feedback. My ideas tend to be backed by those years of experience, and adding config options is not something we like to do willy-nilly, nor do we want a complicated config file.
Sorry. I often prefer to write in a very direct way (e.g. less instances of "I think...", "I feel...", etc.), even when I really mean to make a suggestion, because it is terser. I thought it was appropriate in this situation because it's a feature I really want to see implemented, and while "We should..." tends to generate a response along the lines of either "yes" or "no, do this", "Should we..." tends to generate a 20+ message thread. While this isn't a bad thing, I thought it'd be better to just propose a fully-thought-out, workable scheme from the start, so that it could be built off of. I spent several hours devising this scheme and ensuring that it covers all functionality that I thought would be useful, without requiring too much configuration.
Anyway, I think my writing style is somewhat in poor taste; I'll try to tone it down. I'll eat some of my aggressive words as well, I'm sorry. I just come back from taking a break over the weekend and see my own time spent
On Tue, Jun 14, 2011 at 12:47 AM, Kerrick Staley <mail@kerrickstaley.com> wrote: thinking, analyzing the situation, and coding seemingly get tossed aside so I responded with some attitude.
I feel like the config file syntax is mostly a trivial issue anyway; it should fall into place once we have decided (a) which situations we should handle specially, and (b) which actions can be taken in those situations. I feel that having separate directives for each case would be cleaner, but the "CheckLevel" syntax works as well, and since it's a question of conventions, it's squarely your call (as is, ultimately, everything else I suggest).
No marginal signatures should come up, because I don't think we should use such signatures during the interim when not all developers have fully validated keys; a developer's key can just be excluded from pacman-keyring until it is fully validated. That is a pretty bold assumption. You do realize there are more repos out there that are not provided by a distro that people might want to use and have signatures involved, right?
OK, you're right.
SigPreviouslySigned- an interesting thought, but definitely not something we need on first pass- there are also a multitude of issues here considering everyone starts from a different point.
Yeah, I wasn't even planning on implementing it at first; I just thought it'd be something we wanted in the long term, so I put it in among the rest.
On Wed, Jun 1, 2011 at 3:46 PM, Dan McGee <dpmcgee@gmail.com> wrote:
I also think we may need to be a bit more granular than our current Always/Optional/Never trifecta. We have a multitude of possibilities when checking a signature:
* Valid signature, fully trusted (or ultimate, (GPGME_VALIDITY_{ULTIMATE, FULL}).
* Valid signature, unknown trust/unknown key (GPGME_VALIDITY_UNKNOWN, GPGME_SIGSUM_KEY_MISSING). * Valid signature, trust somewhere in between (marginal, GPGME_VALIDITY_MARGINAL).
* Valid signature, user is never valid (GPGME_VALIDITY_NEVER) * Valid signature, signature is however expired (GPGME_SIGSUM_SIG_EXPIRED) * Valid signature, key is however expired (GPGME_SIGSUM_KEY_EXPIRED) * Bad signature, trust level is irrelevant (GPGME_VALIDITY_RED)
It seems that the first 3 in the last group are corner cases (I could not, for the life of me, find a real example of when GPGME_VALIDITY_NEVER would come up). TRUST_NEVER seems to only come up if you do a gpg --edit-key, "trust", and then choose "never" as your level of trust for a given key, from what I can see (option 2).
Please decide how far you trust this user to correctly verify other users' keys (by looking at passports, checking fingerprints from different sources, etc.)
1 = I don't know or won't say 2 = I do NOT trust 3 = I trust marginally 4 = I trust fully 5 = I trust ultimately m = back to the main menu
For the middle two, having a signature with an unknown or nonexistent key doesn't really say anything about the validity of the package, so I think that outcome should just lumped in among the last 4. So, then, this boils down to "the signature is good", "the signature is bad", or "the signature is marginally trusted", but I think there are other cases (hashed, database unsigned) that should be considered.
Looking at this last night, I came up with something like the following. Basically the first enum is what we have now, but moving toward a more standard name. The real change comes with the second enum, which unlike the first, will be bitflags (I just didn't assign values there yet).
checklevel works more or less as it does currently. unknown falls back to the globally set default, never/optional/always are as expected.
checkoptions basically controls where we perform checks and what we consider valid. Potential API example: alpm_db_set_checklevel(PM_CHECK_LEVEL_ALWAYS, PM_CHECK_PACKAGE_SIGNATURES | PM_CHECK_DATABASE_SIGNATURES | PM_CHECK_MARGINAL_SIGNATURES_OK); This particular call would enforce that signatures are present for packages AND databases. We would however allow marginally trusted sigs (but not unknown ones).
For simplicity, the following rules would also apply, at least initially: * VALID_NEVER, VALIDITY_RED are always failures. * SIG_EXPIRED, KEY_EXPIRED are always failures (we could add a PM_CHECK_EXPIRED_SIGNATURES_OK flag). * Even if PM_CHECK_PACKAGE_SIGNATURES is not provided, we would still check it if present and level is at least OPTIONAL.
typedef enum _pmchecklevel_t { PM_CHECK_LEVEL_UNKNOWN = 0, PM_CHECK_LEVEL_NEVER = 1, PM_CHECK_LEVEL_OPTIONAL, PM_CHECK_LEVEL_ALWAYS } pmchecklevel_t;
typedef enum _pmcheckoptions_t { PM_CHECK_PACKAGE_SIGNATURES, PM_CHECK_DATABASE_SIGNATURES, PM_CHECK_UNKNOWN_SIGNATURES_OK, PM_CHECK_MARGINAL_SIGNATURES_OK } pmcheckoptions_t;
I think this API covers most of the bases we are looking to address. The biggest place we differ is in the accept/warn/abort stuff you proposed. For me I am not a fan of this, only because I know users don't actually read. :/
Feedback welcome, let's work to a reasonable consensus and I should be able to knock this out real quick.
Oh, and Kerrick, I don't know if you've figured it out by now but my hacks are usually available in my repo, right now they are on my working branch. http://code.toofishes.net/cgit/dan/pacman.git/log/?h=working. We should probably make a wiki page of our development repo URLs so people can find them.
-Dan
participants (3)
-
Dan McGee
-
Kerrick Staley
-
Rémy Oudompheng