[arch-dev-public] Can we trust our mirrors?
Hi all, at first: it is really great that the number of mirrors is increasing and I am really thankfull to those who provide one. The point why I feel more and more uncomfortable is that we have no way to ensure tat one will get the same file from a mirror as from archlinux.org. A mirror owner might be a "bad" person himself, his servers might have weak security, the government of their home country cannot be trusted, they might sync from another "bad" mirror. etc... Of course since several years demand package signing. I have even seen some first code, but nothing was ever finished. It should be clear that something has to be done. Manipulating packages is just too easy. The simplest solution would be if we sign the db files (automatically) on gerolde. Of course this is less secure than signing every single package by its packager; but on the other side it would be easy to implement and there would be no overhead for packagers. I am aware that this method would only ensure that packages on a mirror are the same as on gerolde; if our server gets "hacked" we would have lost. But this should be fine and is far more better than just nothing and hoping that there are no "bad guys" out there. Gerhard has written a small patch as a proof of concept. Ignore the details at this point. The idea is as follows: 1) patch repo-add in order to create a .sig file everytime the db file will be changed. For this a private key readable by every dev or just sudo can be used 2) use this version of repo-add on gerolde. So we'll have the sinatures propagated to our mirrors. 3) For testing the whole thing one could just write a small download script which checks the signatures of db files. (Abusing the XferCommand statement in pacman.conf) 4) If all went well we could think about a build-in check in pacman itself. (we might be able to reuse some code here that was written for package signing) 5) Enable those checks by default for all official repos 6) The public key should not be in a package but people have to get it from our website. What do you think about this? Step 1 to 3 could be implemented in a rather short time. Pierre -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
Pierre Schmitz schrieb:
The simplest solution would be if we sign the db files (automatically) on gerolde. Of course this is less secure than signing every single package by its packager; but on the other side it would be easy to implement and there would be no overhead for packagers.
If this is to provide any security, we need to stop using md5! md5 is okay when trying to detect corrupted downloads, however it is possible to find collisions and thus build a "bad" package that has the same md5 as the good package.
On Sat, 29 Nov 2008 15:00:20 +0100, Thomas Bächler <thomas@archlinux.org> wrote:
If this is to provide any security, we need to stop using md5! md5 is okay when trying to detect corrupted downloads, however it is possible to find collisions and thus build a "bad" package that has the same md5 as the good package.
Well, it should be quite easy to use sha instead. I am not an expert but how easy is it to produce a valid package with the same md5sum? I know that creating "some" file is not hard. -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
Pierre Schmitz schrieb:
On Sat, 29 Nov 2008 15:00:20 +0100, Thomas Bächler <thomas@archlinux.org> wrote:
If this is to provide any security, we need to stop using md5! md5 is okay when trying to detect corrupted downloads, however it is possible to find collisions and thus build a "bad" package that has the same md5 as the good package.
Well, it should be quite easy to use sha instead. I am not an expert but how easy is it to produce a valid package with the same md5sum? I know that creating "some" file is not hard.
You can append arbitrary data at the end of a gzip file (this was used in the construction of the "forbidden prime number"). Thus you could create a relatively small malicious package and then try to construct a collision by appending data to the file. It should not be much harder than finding any collision, but I am no expert either.
On Sat, Nov 29, 2008 at 7:48 AM, Pierre Schmitz <pierre@archlinux.de> wrote:
Hi all,
at first: it is really great that the number of mirrors is increasing and I am really thankfull to those who provide one.
The point why I feel more and more uncomfortable is that we have no way to ensure tat one will get the same file from a mirror as from archlinux.org. A mirror owner might be a "bad" person himself, his servers might have weak security, the government of their home country cannot be trusted, they might sync from another "bad" mirror. etc...
Of course since several years demand package signing. I have even seen some first code, but nothing was ever finished. It should be clear that something has to be done. Manipulating packages is just too easy.
The simplest solution would be if we sign the db files (automatically) on gerolde. Of course this is less secure than signing every single package by its packager; but on the other side it would be easy to implement and there would be no overhead for packagers. I am aware that this method would only ensure that packages on a mirror are the same as on gerolde; if our server gets "hacked" we would have lost. But this should be fine and is far more better than just nothing and hoping that there are no "bad guys" out there.
Gerhard has written a small patch as a proof of concept. Ignore the details at this point. The idea is as follows: 1) patch repo-add in order to create a .sig file everytime the db file will be changed. For this a private key readable by every dev or just sudo can be used 2) use this version of repo-add on gerolde. So we'll have the sinatures propagated to our mirrors. 3) For testing the whole thing one could just write a small download script which checks the signatures of db files. (Abusing the XferCommand statement in pacman.conf) 4) If all went well we could think about a build-in check in pacman itself. (we might be able to reuse some code here that was written for package signing) 5) Enable those checks by default for all official repos 6) The public key should not be in a package but people have to get it from our website.
What do you think about this? Step 1 to 3 could be implemented in a rather short time.
Pierre
There's too much talk on this idea. Before we go ahead and do this, could someone submit this patch to the pacman-dev list, so the pacman developers can give it a once-over. Just make sure to let them know that this is a temporary solution. Additionally - where will gpg get the key from on gerolde? Shouldn't this be configurable, or even set via an optarg to the -s param?
On Mon, Dec 1, 2008 at 11:06 AM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On Sat, Nov 29, 2008 at 7:48 AM, Pierre Schmitz <pierre@archlinux.de> wrote:
Hi all,
at first: it is really great that the number of mirrors is increasing and I am really thankfull to those who provide one.
The point why I feel more and more uncomfortable is that we have no way to ensure tat one will get the same file from a mirror as from archlinux.org. A mirror owner might be a "bad" person himself, his servers might have weak security, the government of their home country cannot be trusted, they might sync from another "bad" mirror. etc...
Of course since several years demand package signing. I have even seen some first code, but nothing was ever finished. It should be clear that something has to be done. Manipulating packages is just too easy.
The simplest solution would be if we sign the db files (automatically) on gerolde. Of course this is less secure than signing every single package by its packager; but on the other side it would be easy to implement and there would be no overhead for packagers. I am aware that this method would only ensure that packages on a mirror are the same as on gerolde; if our server gets "hacked" we would have lost. But this should be fine and is far more better than just nothing and hoping that there are no "bad guys" out there.
Gerhard has written a small patch as a proof of concept. Ignore the details at this point. The idea is as follows: 1) patch repo-add in order to create a .sig file everytime the db file will be changed. For this a private key readable by every dev or just sudo can be used 2) use this version of repo-add on gerolde. So we'll have the sinatures propagated to our mirrors. 3) For testing the whole thing one could just write a small download script which checks the signatures of db files. (Abusing the XferCommand statement in pacman.conf) 4) If all went well we could think about a build-in check in pacman itself. (we might be able to reuse some code here that was written for package signing) 5) Enable those checks by default for all official repos 6) The public key should not be in a package but people have to get it from our website.
What do you think about this? Step 1 to 3 could be implemented in a rather short time.
Pierre
There's too much talk on this idea. Before we go ahead and do this, could someone submit this patch to the pacman-dev list, so the pacman developers can give it a once-over. Just make sure to let them know that this is a temporary solution.
Additionally - where will gpg get the key from on gerolde? Shouldn't this be configurable, or even set via an optarg to the -s param?
Yee ha! This one got a bit crazy, didn't it? I decided to let it simmer here until it calmed down enough that my thoughts wouldn't get lost in the mix. First off- stop talking. Start coding. So far, Gerhard has stepped up to the plate on this thread (thanks!), and Geoffroy Carrier has stepped up in the past to start the ball rolling. I'm sick of all this "we need to do this" and not a single person following through with it. Geoffroy Carrier made the biggest push yet in the right direction. This is the best solution I see to this problem and is the one I am willing to put some effort behind. As a matter of fact, we have put effort behind it: http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=shortlog;h=refs/heads/gp... So what is the idea here? Signed *packages*. Why? 1. They are super easy to either leave detached from the package, or integrate right into package databases. This work is already done: Add GPG signature support to makepkg - http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=0bf6375e31a... Add PGPSIG field in repo-add - http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=7e86019e352... 2. Signed packages mean I, as an Arch developer, do not have to sign off on Aaron's or Paul's or Dale's work. I sign off on my *own* work. When Paul (sorry Paul!) decides to go rogue, we drop his signature from the "accepted" keyring (or alternatively, revoke some signature on his key or something) that makes his key no longer a valid signing key for official Arch packages. We can still trust all the other packages in the repositories. 3. Signing databases still makes md5sums a weak point, for all those worried about that. Signing packages mean we don't even have to touch the md5sum stuff, which is great from the pacman perspective. (Remember: the purpose of md5sums is a download sanity check, not a security check.) 4. Joe user can sign individual packages if he wants, and we have a framework in place to honor those signatures and verify them. So where do we stand on all of this? As I showed above, the makepkg and repo-add pieces are pretty much in place. We've done a decent amount of work on the libalpm/pacman side of things, but it isn't complete yet and is very rough around the edges. This is something that is very doable if people want to jump in and help us get this done. Obviously knowledge of C would be helpful, but we can always use some beta testers to help us work out kinks if necessary. This feature request (http://bugs.archlinux.org/task/5331) is quite out of date, but we can use it to start tracking things dealing with signed packages. There have also been numerous discussions on the pacman-dev ML about this stuff (Notably in June, http://archlinux.org/pipermail/pacman-dev/2008-June/thread.html). So who is willing to help? If we can get this done, pacman 3.3 can go out the door with signing support as soon as this is done. -Dan
On Wed, Dec 3, 2008 at 7:40 PM, Dan McGee <dpmcgee@gmail.com> wrote:
So who is willing to help? If we can get this done, pacman 3.3 can go out the door with signing support as soon as this is done.
Sounds like a good Christmas present for all those wanting this functionality.
On Wed, 3 Dec 2008 19:40:59 -0600 "Dan McGee" <dpmcgee@gmail.com> wrote:
On Mon, Dec 1, 2008 at 11:06 AM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On Sat, Nov 29, 2008 at 7:48 AM, Pierre Schmitz <pierre@archlinux.de> wrote:
Hi all,
at first: it is really great that the number of mirrors is increasing and I am really thankfull to those who provide one.
The point why I feel more and more uncomfortable is that we have no way to ensure tat one will get the same file from a mirror as from archlinux.org. A mirror owner might be a "bad" person himself, his servers might have weak security, the government of their home country cannot be trusted, they might sync from another "bad" mirror. etc...
Of course since several years demand package signing. I have even seen some first code, but nothing was ever finished. It should be clear that something has to be done. Manipulating packages is just too easy.
The simplest solution would be if we sign the db files (automatically) on gerolde. Of course this is less secure than signing every single package by its packager; but on the other side it would be easy to implement and there would be no overhead for packagers. I am aware that this method would only ensure that packages on a mirror are the same as on gerolde; if our server gets "hacked" we would have lost. But this should be fine and is far more better than just nothing and hoping that there are no "bad guys" out there.
Gerhard has written a small patch as a proof of concept. Ignore the details at this point. The idea is as follows: 1) patch repo-add in order to create a .sig file everytime the db file will be changed. For this a private key readable by every dev or just sudo can be used 2) use this version of repo-add on gerolde. So we'll have the sinatures propagated to our mirrors. 3) For testing the whole thing one could just write a small download script which checks the signatures of db files. (Abusing the XferCommand statement in pacman.conf) 4) If all went well we could think about a build-in check in pacman itself. (we might be able to reuse some code here that was written for package signing) 5) Enable those checks by default for all official repos 6) The public key should not be in a package but people have to get it from our website.
What do you think about this? Step 1 to 3 could be implemented in a rather short time.
Pierre
There's too much talk on this idea. Before we go ahead and do this, could someone submit this patch to the pacman-dev list, so the pacman developers can give it a once-over. Just make sure to let them know that this is a temporary solution.
Additionally - where will gpg get the key from on gerolde? Shouldn't this be configurable, or even set via an optarg to the -s param?
Yee ha! This one got a bit crazy, didn't it? I decided to let it simmer here until it calmed down enough that my thoughts wouldn't get lost in the mix.
First off- stop talking. Start coding. So far, Gerhard has stepped up to the plate on this thread (thanks!), and Geoffroy Carrier has stepped up in the past to start the ball rolling. I'm sick of all this "we need to do this" and not a single person following through with it.
Geoffroy Carrier made the biggest push yet in the right direction. This is the best solution I see to this problem and is the one I am willing to put some effort behind. As a matter of fact, we have put effort behind it: http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=shortlog;h=refs/heads/gp...
So what is the idea here? Signed *packages*. Why?
1. They are super easy to either leave detached from the package, or integrate right into package databases. This work is already done: Add GPG signature support to makepkg - http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=0bf6375e31a... Add PGPSIG field in repo-add - http://code.toofishes.net/gitweb.cgi?p=pacman.git;a=commitdiff;h=7e86019e352... 2. Signed packages mean I, as an Arch developer, do not have to sign off on Aaron's or Paul's or Dale's work. I sign off on my *own* work. When Paul (sorry Paul!) decides to go rogue, we drop his signature from the "accepted" keyring (or alternatively, revoke some signature on his key or something) that makes his key no longer a valid signing key for official Arch packages. We can still trust all the other packages in the repositories. 3. Signing databases still makes md5sums a weak point, for all those worried about that. Signing packages mean we don't even have to touch the md5sum stuff, which is great from the pacman perspective. (Remember: the purpose of md5sums is a download sanity check, not a security check.) 4. Joe user can sign individual packages if he wants, and we have a framework in place to honor those signatures and verify them.
So where do we stand on all of this? As I showed above, the makepkg and repo-add pieces are pretty much in place. We've done a decent amount of work on the libalpm/pacman side of things, but it isn't complete yet and is very rough around the edges. This is something that is very doable if people want to jump in and help us get this done. Obviously knowledge of C would be helpful, but we can always use some beta testers to help us work out kinks if necessary.
This feature request (http://bugs.archlinux.org/task/5331) is quite out of date, but we can use it to start tracking things dealing with signed packages. There have also been numerous discussions on the pacman-dev ML about this stuff (Notably in June, http://archlinux.org/pipermail/pacman-dev/2008-June/thread.html).
So who is willing to help? If we can get this done, pacman 3.3 can go out the door with signing support as soon as this is done.
-Dan
This is good news, I'll gladly help beta test.
participants (5)
-
Aaron Griffin
-
Dan McGee
-
Jud
-
Pierre Schmitz
-
Thomas Bächler