On Sat, 19 Feb 2011 11:21:47 +1000 Allan McRae wrote:

We are using gpgme which is maintained by gpg developers. So we are not reinventing any wheel. If that is an ill-maintained and contains security issues, then why trust gpg at all?

I'm not up on the latest in gpg, what is being developed, or how well it has been cryptographically tested, and it sounds like you're not either to some extent. I was offering general suggestions - try to see the bigger picture and the details won't be so much work. I still think using command line gpg gives users better control over their own security solutions and provides better security. But you're obviously too deeply invested in the libraries now to even consider anything else (which is one reason why I advise against them). So be it.

Have you actually looked at the current implementation at all?

I read some discussions of it, but I have not looked at it. Frankly it interests me far less than having signatures available at this point. I trust that you can implement checking. I think you're putting the cart before the horse, which is why package signing is getting nowhere (no real usable results). But if you have kept it as simple as you describe, good work.

In the end, the only way anything will get implemented is if patches are provided. (That includes the suggestion of providing signatures in repos on Arch Linux - look at that devtools/db-scripts projects). Dan and I have also mentioned our consulting rates in the past, which may or may not increase motivation to finish this...

Surely you have some mechanism in makepkg or elsewhere for generating the signatures used in your test repo for your pacman sig checking features. Why not push this through to the packaging or database tools now while you're still working on pacman? Or have you completely ignored the need to generate signatures (I know you haven't from what I've read, so that's rhetorical). It seems you or others are stalling on providing signatures because you want exclusive control of checking those signatures. It's trivial to implement providing them and surely you know that - man gpg for help. Like everything done by committee, I think you're making things more complicated than they need to be. It's okay, I didn't have high hopes for this communication. This issue is obviously too mired in politics and ego for anything to budge (not you personally).

Finally, *succinct* reports on where the current implementation in pacman can be improved are also appreciated. But that requires actually paying attention to what has already been done.

Agreed. FWIW, I think your pacman changes will see reality (as in popular use) much more quickly if signatures show up on mirrors sooner rather than later. Once the sigs are there, the appetite for your pacman work will be there, and the dev politics of Arch will sway. Simply put, there can be no logical or technical reason for not providing signatures for packages. At the minimum, they can only vastly improve the situation, which currently is a complete abdication of any form of responsible package security, and they are trivial to add. The only reason for not providing them is a human one - IOW irrational. I'm no diplomat - I just make things work. I'll leave you to your politics.

On 19/02/11 12:00, IgnorantGuru wrote:

On Sat, 19 Feb 2011 11:21:47 +1000 Allan McRae wrote:

...

We are using gpgme which is maintained by gpg developers. So we are not reinventing any wheel. If that is an ill-maintained and contains security issues, then why trust gpg at all?

I'm not up on the latest in gpg, what is being developed, or how well it has been cryptographically tested, and it sounds like you're not either to some extent. I was offering general suggestions - try to see the bigger picture and the details won't be so much work. I still think using command line gpg gives users better control over their own security solutions and provides better security. But you're obviously too deeply invested in the libraries now to even consider anything else (which is one reason why I advise against them). So be it.

We are not too deeply invested in anything. No release has been made... I was just pointed out that you appear to have no idea what the library we are using is and what it actually does. It makes absolutely no sense to have pacman manually fork a process to run gpg to verify a package/database and then manually parse the output when the upstream gpg developers provide a library to do just that.

...

Have you actually looked at the current implementation at all?

I read some discussions of it, but I have not looked at it. Frankly it interests me far less than having signatures available at this point. I trust that you can implement checking. I think you're putting the cart before the horse, which is why package signing is getting nowhere (no real usable results). But if you have kept it as simple as you describe, good work.

...

In the end, the only way anything will get implemented is if patches are provided. (That includes the suggestion of providing signatures in repos on Arch Linux - look at that devtools/db-scripts projects). Dan and I have also mentioned our consulting rates in the past, which may or may not increase motivation to finish this...

Surely you have some mechanism in makepkg or elsewhere for generating the signatures used in your test repo for your pacman sig checking features. Why not push this through to the packaging or database tools now while you're still working on pacman? Or have you completely ignored the need to generate signatures (I know you haven't from what I've read, so that's rhetorical).

It seems you or others are stalling on providing signatures because you want exclusive control of checking those signatures. It's trivial to implement providing them and surely you know that - man gpg for help. Like everything done by committee, I think you're making things more complicated than they need to be. It's okay, I didn't have high hopes for this communication. This issue is obviously too mired in politics and ego for anything to budge (not you personally).

Design by committee implies there is an actually committee making decisions around here, which is far from the truth... The reason the makepkg changes have not been pushed yet is because they are not finished. As someone who has run the gpg branch version of makepkg for a while, I found it really annoying to not be able to disable/enable signing from the command line. I see Denis just sent a patch for that so it may be possible to push the makepkg changes in the future. But that is very unlikely to happen before the next release which is scheduled to occur quite soon. Offtopic: Not that this actually matters as far as Arch Linux is concerned. The support needs added to devtools/db-scripts. The package upload script could have a temporary line in it to do the package signing while waiting on makepkg to do that automatically.

...

Finally, *succinct* reports on where the current implementation in pacman can be improved are also appreciated. But that requires actually paying attention to what has already been done.

Agreed. FWIW, I think your pacman changes will see reality (as in popular use) much more quickly if signatures show up on mirrors sooner rather than later. Once the sigs are there, the appetite for your pacman work will be there, and the dev politics of Arch will sway.

Simply put, there can be no logical or technical reason for not providing signatures for packages. At the minimum, they can only vastly improve the situation, which currently is a complete abdication of any form of responsible package security, and they are trivial to add. The only reason for not providing them is a human one - IOW irrational. I'm no diplomat - I just make things work. I'll leave you to your politics.

I'll repeat, talk to the providers of the repos about getting signatures provided. This list is for pacman-development only and thus any distribution specific conversation is off-topic.. But to be the guy who "makes things work", patches will be required. Allan

On 19/02/11 15:18, Daniel Mendler wrote:

The mail by IgnorantGuru is very much what I was going to write. There is no problem in adding signatures to the Arch repositories immediately.

You always say that pacman is not the same as Arch. This might be true, but which major distribution uses pacman? We should not argue about those subtile differences.

I pulled the main pacman branch, merged Allan's gpg-patches and created a signed repository - everything worked fine (Except for example overwriting the db with a unverified one before verifing - I can provide patches for this in one week). You always say that you need patches, but what exactly? You seem to have a working implementation but you don't integrate these into master. Instead you work on minor performance issues (Single file database for example) even though we have a very serious security problem.

I will repeat myself again... Patches for pacman do bugger all for getting signatures into Arch Linux repos. Patches for the Arch Linux devtools/db-scripts packages are needed. And I will once again point to the package signing TODO page for a list of what we need to do at a minimum before this becomes integrated in the main pacman branch: https://wiki.archlinux.org/index.php/User:Allan/Package_Signing As with all feature branches, they integrated into master when they are finished. Otherwise we can not make a release without actually getting it fully completed or backing out the unfinished work. Given the rate this has been developed, the second seems the likely outcome. Finally, "minor" performance issues interest me a hell of a lot more than package signing. Mainly because that actually affects me whereas unsigned packages really does not... That is why I spent my free time implementing them. Thinking about it, improving optdepends handling, transaction hooks, VCS support in makepkg, adding a test suite for makepkg, automatic creation of debug packages, .... all affect me more than package signing does, so I maybe will start work on package signing again once those are finished. Allan

On Sat, 19 Feb 2011 10:25:38 +0100 Pierre Schmitz wrote:

I'd prefer to be pointed at some documents which describe exactly the wrokflow to sign a package with makepkg, upload it, add it to a db, update, replace and delete it.

Once there is a version of pacman which supports signed packages I can start implementing these ideas.

I think it is best not to wait for pacman - that is what is stopping this from reaching usable reality. From the looks of it, pacman is never going to get done until the signatures are available and there's a demand for checking them. If changes need to be made later to make the signatures (file format, etc) compatible with pacman, that should be minor, especially if it is well written. As I said, I'm eager to write and maintain a script that checks the pkg cache signatures, at least until pacman is complete in this area, and I don't mind if the sig db format changes a few times. But I can't check signatures when there are none to check. So I definitely encourage the work you're talking about, and it alone will vastly improve the security situation, regardless of pacman.

Hi Allan

I will repeat myself again... Patches for pacman do bugger all for getting signatures into Arch Linux repos. Patches for the Arch Linux devtools/db-scripts packages are needed.

Well, Pierre says the same for pacman. Someone has to take the first initiative here.

And I will once again point to the package signing TODO page for a list of what we need to do at a minimum before this becomes integrated in the main pacman branch: https://wiki.archlinux.org/index.php/User:Allan/Package_Signing As with all feature branches, they integrated into master when they are finished. Otherwise we can not make a release without actually getting it fully completed or backing out the unfinished work. Given the rate this has been developed, the second seems the likely outcome.

I understand that it should be finished before it is merged. What is missing is a strong statement from the development team that they want signatures asap. I think there are enough people who are willing to provide patches (me included) if you show real interest in package signing.

Finally, "minor" performance issues interest me a hell of a lot more than package signing. Mainly because that actually affects me whereas unsigned packages really does not... That is why I spent my free time implementing them. Thinking about it, improving optdepends handling, transaction hooks, VCS support in makepkg, adding a test suite for makepkg, automatic creation of debug packages, .... all affect me more than package signing does, so I maybe will start work on package signing again once those are finished.

You really have to rethink your priority list here. Those attacks on package managers are known for a long time and the package signing point has come up very often on the pacman mailing list. So there are people who are concerned about it. Daniel

On Sat, 19 Feb 2011 12:38:07 +1000 Allan McRae wrote:

It makes absolutely no sense to have pacman manually fork a process to run gpg to verify a package/database and then manually parse the output when the upstream gpg developers provide a library to do just that.

Perhaps if you look at it from a different perspective, it will make sense to you. This is what I meant by code efficiency not always being optimum in security dev. I still think you've already precluded another solution, but I'll see if I can enlighten you on what I'm getting at. If pacman forks a call to gpg using a command line, it is transparent. I can replace gpg with a script and catch the call, examine it, log it, even alter the text stream returned to see how pacman reacts. I can do this on an executable of unknown origin and with possibly tampered source, without needed the source. I can even use a custom gpg or wrapper script to do other things you and I aren't able to anticipate right now - it is open-ended. The interprocess communication is open, flexible, and transparent. That is why linux programs operate on the command line and on text streams - it makes for transparency and flexibility. Check out http://en.wikipedia.org/wiki/Unix_philosophy#Mike_Gancarz:_The_UNIX_Philosop... In particular, ponder how these might apply, doubly so in security: 4. Choose portability over efficiency. 5. Store data in flat text files. 6. Use shell scripts to increase leverage and portability. 8. Avoid captive user interfaces. 9. Make every program a filter. And especially: 1a. Allow the user to tailor the environment. In addition, if I want to sabotage the source, API calls make for easy hiding of changes that may not be detected, whereas changing command line usage makes a lot of noise and can be detected. Will you notice one value changed in 10,000 lines to create a buffer overrun where I can later inject traceless code? Probably not. In sophisticated security breaches, that's how things are done. They're 'accidents' which can later be denied if discovered. Let's see you deny changing the keyring used on the command line. That's just a small sample of why open and flexible interprocess communication is valuable in security dev. That's not to say it's always done this way - far from it, sometimes with ulterior motives.

Finally, "minor" performance issues interest me a hell of a lot more than package signing.

Obvious Allan, you don't understand the importance of package signing and with that poor attitude and lack of awareness I don't think you should be working on security at all. All you're apparently doing is sabotaging it by procrastinating. Obviously you're the type that needs to see a huge security breach in Arch to understand the importance of this. And even without a huge breach, unsigned packages create a local lack of security for admins. Which part of this aren't you getting? Why do you think Microsoft bothers to sign their updates? Just for the heck of it? (Sad that in this case Microsoft is being used as an example of good security practice - that's how out of line Arch's package security is.) It's great when one of the devs responsible for package signing announces he doesn't care about package signing, it's just stupid. Very professional. I'll not be contracting your services, thanks.

I found it really annoying to not be able to disable/enable signing from the command line.

While I might agree, you seem to find this whole package security subject "really annoying" and too much trouble to bother with. I find a disconcerting lack of good security practices in Arch, and I'm beginning to see why.

But to be the guy who "makes things work", patches will be required.

Interesting that you think so, because patches are the way to make non-secure junk. The way to make things work is for the person most familiar with the code and protocols to make those changes rather than him begging others to write ill-fitting patches. Patches are also a big waste of time because what someone familiar with the code can do in a few minutes (and do well) may take hours of research for someone unfamiliar (to do poorly). Talk about efficiency - try applying it to your work in larger ways. Maybe that's one reason why it takes you so long to develop - you're unwilling to take on responsibility for the project as a whole. You definitely seem a very reluctant and whining freeware developer. Get over yourself and put some quality into your work, regardless of what you're paid or not paid. Or don't - in which case you're not worth your complaining.

On Sat, 19 Feb 2011 23:46:57 +1000 Allan McRae wrote:

Or is it less secure to write our own code (reviewed by perhaps two people total) to launch and parse the output of gpg or use the wrapper provided by the gpgp devs. Note that gpgme just calls gpg, so you can still replace that with a wrapper and do everything you just pointed out.

I actually don't have huge problems with gpgme, but you said you couldn't understand my point, so I explained. Based on what I have seen over the years, I still think parsing the text is wiser. Anything which makes security mechanisms more transparent improves security, in general. But I understand why APIs are so inviting (to developers and hackers alike).

1) I understand its importance

I don't believe so, or you would give it higher priority. Apparently we need a hacker to exploit this and inconvenience huge numbers of people for YOU to see the importance, Microsoft-style, but that's a very lazy and irresponsible approach.

2) I am not "working" on anything. I am volunteering my time.

I find that a poor attitude, as I've always considered freeware (and other volunteer WORK) among the most important WORK I do, but obviously you've got some issues about developing freeware. If you're that miserable, don't do it. A bitter baker bakes a bitter bread. You're taking the joy out of development with your approach IMO. One of the joys of being a freeware developer is that you're free. Turning it into an obligation that you whine about is missing the joy of it. So like I said, if you're that miserable, don't do it - no one is going to make your misery worth it by paying you $1000 for this, like in your 'real work'.

3) I am not sabotaging anything. I have reviewed all patches submitted here for package signing and have pulled them to a git repo and even spent time fixing the current implementation.

I do acknowledge that you've brought this forward a bit, but your attitude about your _work_ gives me great cause for concern. When you work with any area of cryptography, remember that lives and certainly livelihoods can literally depend on your keystrokes (even though you may not want or expect them to), so get behind your work or don't do it. This isn't just a toy, free though it may be.

On Sun 20 Feb 2011 01:24 +1000, Allan McRae wrote:

On 20/02/11 00:33, IgnorantGuru wrote:

...

On Sat, 19 Feb 2011 23:46:57 +1000 Allan McRae wrote:

...

Or is it less secure to write our own code (reviewed by perhaps two people total) to launch and parse the output of gpg or use the wrapper provided by the gpgp devs. Note that gpgme just calls gpg, so you can still replace that with a wrapper and do everything you just pointed out.

I actually don't have huge problems with gpgme, but you said you couldn't understand my point, so I explained. Based on what I have seen over the years, I still think parsing the text is wiser. Anything which makes security mechanisms more transparent improves security, in general. But I understand why APIs are so inviting (to developers and hackers alike).

...

1) I understand its importance

I don't believe so, or you would give it higher priority. Apparently we need a hacker to exploit this and inconvenience huge numbers of people for YOU to see the importance, Microsoft-style, but that's a very lazy and irresponsible approach.

Let me rephrase that: I understand its importance _to other people_.

As I have said, this whole issue does not particularly affect me so I give it low priority. I really do not care if it affects others. I develop pacman and Arch Linux to improve my computing experience. If others get benefit from my work, then that is a bonus.

...

...

2) I am not "working" on anything. I am volunteering my time.

I find that a poor attitude, as I've always considered freeware (and other volunteer WORK) among the most important WORK I do, but obviously you've got some issues about developing freeware. If you're that miserable, don't do it. A bitter baker bakes a bitter bread. You're taking the joy out of development with your approach IMO. One of the joys of being a freeware developer is that you're free. Turning it into an obligation that you whine about is missing the joy of it. So like I said, if you're that miserable, don't do it - no one is going to make your misery worth it by paying you $1000 for this, like in your 'real work'.

I think we have just agreed... in a way. I should focus on the areas that make me a happy contributor. If that does not happen to be package signing, then so be it.

...

...

3) I am not sabotaging anything. I have reviewed all patches submitted here for package signing and have pulled them to a git repo and even spent time fixing the current implementation.

I do acknowledge that you've brought this forward a bit, but your attitude about your _work_ gives me great cause for concern. When you work with any area of cryptography, remember that lives and certainly livelihoods can literally depend on your keystrokes (even though you may not want or expect them to), so get behind your work or don't do it. This isn't just a toy, free though it may be.

I think I know every distribution using pacman as a package manager and (unless there is an enterprise level distro I am missing) if peoples lives depend on one of these distros, then I am sorry to say it but in my opinion they are stupid and deserve to die.

Yeah! Archers deserve to die! But really I'm not convinced by this hyper-paranoia trash. There will always be ways to compromise your machine. Someone who would go through the trouble of setting up a proxy mirror and injecting malicious code into seemingly normal packages is probably going to find other ways. Package signing will not protect you. You will never be safe. The truth is out there.

On Sat, 2011-02-19 at 20:05 +0100, Alf Gaida wrote:

...

Yeah! Archers deserve to die!

But really I'm not convinced by this hyper-paranoia trash. There will always be ways to compromise your machine. Someone who would go through the trouble of setting up a proxy mirror and injecting malicious code into seemingly normal packages is probably going to find other ways. Package signing will not protect you.

You will never be safe. The truth is out there. This is opensource - if you would create real trouble, just help with kernel- modules. ;) The only difference is, in other distributions these errors came through your system signed.

Why hacking, when simple development is so easy?

I don't understand what you are saying, but in short. You can't force Allan / any pacman-dev to create package signing for pacman. If you really want to get this feature into pacman/archlinux (dbscripts etc. needs to be redone too): -read the code -add patches -wait for devs to sign them off on a side note: http://media.ccc.de/browse/congress/2010/27c3-4295-en-high_speed_high_securi... -- Jelle van der Waa

On 02/19/2011 08:38 PM, Alf Gaida wrote:

Maybe i have should use a <ironic> tag. Nothing is secure in the end, if anyone will do harm, he'll find a security hole. Like this: http://www.webhostingtalk.com/showthread.php?t=717240

Exactly, because we cannot reach perfect security, we should not care about it at all!

I agree fully with Allan. For me it makes not a big difference if a package is signed or not. It's a nice to have feature and i would be glad if someone would implement it. But for me it has a very low priority..

It makes a big difference if your system is compromised. And then you will care about it. I don't understand this naive and short-sighted opinion. @Allan: I am a bit disappointed with your opinion that you want to implement only features that you care about. I think there is also a reponsibility if you are one of the main developers of the package manager of a popular distribution. And you don't even have to implement the features yourself - there are people who are willing to help. But those people should also get some support by you. Daniel

I'm not sure I even want to get involved in this thread. :/ On Sat, Feb 19, 2011 at 5:05 PM, Allan McRae wrote:

On 20/02/11 08:42, Daniel Mendler wrote:

...

@Allan: I am a bit disappointed with your opinion that you want to implement only features that you care about. I think there is also a reponsibility if you are one of the main developers of the package manager of a popular distribution.

This is totally false. None of us signed up because we wanted to code stuff other people wanted; we saw an open source project we could contribute to and all the sudden we ended up being the lead developers. If our fellow developers were telling us "hey we really need package signing", we'd probably set aside our having fun to work on it a bit more, but if you notice, none of them are doing that.

Responsibility? I take responsibility for myself and no one else, anything else would be stupid and make me legally liable for work I don't even get paid for.

...

And you don't even have to implement the features yourself - there are people who are willing to help. But those people should also get some support by you.

Those people get full support from me.  You might have seen between these emails that I reviewed the three patches for package signing posted to this list yesterday within 12 hours of them being posted.

I am serious when I say "patches welcome".  I just turns out those people that claim to be willing to help, rare do anything. The other thing we frequently see is work that doesn't come close to meeting our standards, and when we point this out, we get accused of not wanting to implement package signing. At that point, what are we expected to do? Redo the work ourself?

Either way, can we all just relax a bit? This thread is becoming a bitching ground, and nothing productive has come out of it. Act civil and stop using the guise of the internet to say anything you want and attack others. It really isn't appropriate. -Dan

On 20/02/11 10:36, Daniel Mendler wrote:

I think this should also go to a much more technical level. We have the gpg tree in Allan's repository. As I said I tested it with a repository and got it to work. So can you tell me what do you need till this can be merged into master?

1. Design a strategy to manage the keyrings and adapt the tools to it 2. Patches for the issues on the Package Signining Wiki Page 3. Patches to db-scripts to manage the database with gpg signatures

Some of the issues on the wiki page are really minor (e.g. rename option). There are more complex ones (replacing verified db with unverified one, reworking the signature checking code when using pacman -U). And there are already patches for some of the issues.

So what do you say about the code quality of the branch? It it acceptable at this point or is there improvement needed? Are there other blockers preventing you from merging it as soon as the points above are solved?

As far as I am concerned, the major points on the TODO list that need patches are the first five for pacman: TODO: fix (and refactor) reading signatures for packages installed with -U TODO: have a way to force a signature check with -U (i.e. abort if no signature is found) TODO: only replace old database when signature is valid TODO: output when downloading signature file - name when downloaded TODO: output when downloading signature file - "error" when not available The other issues are all fairly minor (and the pacman-key/makepkg ones mostly have patches that just need revised already). So if patches are submitted for those five points, and any criticism followed up, I will commit to then spending the time doing the needed tidying/rebasing of the code on my gpg branch to have it suitable for merging. Allan

On 20/02/11 21:47, Daniel Mendler wrote:

Hi Allan

...

As far as I am concerned, the major points on the TODO list that need patches are the first five for pacman:

TODO: fix (and refactor) reading signatures for packages installed with -U TODO: have a way to force a signature check with -U (i.e. abort if no signature is found) TODO: only replace old database when signature is valid TODO: output when downloading signature file - name when downloaded TODO: output when downloading signature file - "error" when not available

I have a patch for the third point. Can you please clarify the last two points? Do you think the output is too verbose (two download progress bars with the same name etc, and two error messages in case of error)?

Some examples of what those last two points cover: 1) VerifySig = Always, valid signature: :: Synchronizing package databases... pacman 1.0K 381.6K/s 00:00:00 [######################] 100% pacman 0.3K 14.4M/s 00:00:00 [######################] 100% kernel64 1.5K 42.2M/s 00:00:00 [######################] 100% Two download bars with the same name - the second should be something like pacman.sig 2) VerifySig = Always, no signature available: :: Synchronizing package databases... pacman 1.0K 317.1K/s 00:00:00 [######################] 100% error: failed retrieving file 'pacman.db.sig' from disk : No such file or directory error: Failed to download signature for db: No such file or directory error: failed to update pacman (invalid PGP signature) kernel64 1.5K 55.1M/s 00:00:00 [######################] 100% The error messages need reduced to a possibly single, clear message 3) VerifySig = Optional, no signature available: :: Synchronizing package databases... pacman 1.0K 363.2K/s 00:00:00 [######################] 100% error: failed retrieving file 'pacman.db.sig' from disk : No such file or directory kernel64 1.5K 30.5M/s 00:00:00 [######################] 100% That is not an actual error as signature checking is optional

On Sat, Feb 19, 2011 at 2:06 PM, IgnorantGuru wrote:

Interesting that you think so, because patches are the way to make non-secure junk.  The way to make things work is for the person most familiar with the code and protocols to make those changes rather than him begging others to write ill-fitting patches.  Patches are also a big waste of time because what someone familiar with the code can do in a few minutes (and do well) may take hours of research for someone unfamiliar (to do poorly).  Talk about efficiency - try applying it to your work in larger ways.  Maybe that's one reason why it takes you so long to develop - you're unwilling to take on responsibility for the project as a whole.  You definitely seem a very reluctant and whining freeware developer.  Get over yourself and put some quality into your work, regardless of what you're paid or not paid.  Or don't - in which case you're not worth your complaining.

What makes you worth ? You are not only complaining, you are also freely attacking people when you have no idea who they are, what they do, and how things work around here. Ah, the joy of internet. I don't know why you felt the need to write so many mails just to say one thing : it would be nice to have signatures on the mirrors already, regardless of what pacman and repo-add support. And well, we agree, so thanks for your quality contribution !