Re: [arch-general] [arch-dev-public] Can we trust our mirrors?
Am Sat, 29 Nov 2008 15:00:20 +0100 schrieb Thomas Bächler <thomas@archlinux.org>:
Pierre Schmitz schrieb:
The simplest solution would be if we sign the db files (automatically) on gerolde. Of course this is less secure than signing every single package by its packager; but on the other side it would be easy to implement and there would be no overhead for packagers.
If this is to provide any security, we need to stop using md5! md5 is okay when trying to detect corrupted downloads, however it is possible to find collisions and thus build a "bad" package that has the same md5 as the good package.
For myself i don't accept the "md5sum is bad" argument as a "stopper" for each idea to provide a pacman secure concept ;-) Current situation is: Everyone who offers a mirror could provide a manipulated pacman or bash package. He could reduce the content of a binaray to a simple rm -rf, fdisk or something. He only has to tar "his" package and edit the core.db.tar,gz If we sign our db files as a minimum! security: This would make package manipulating more difficult. Content changes of pacman or bash packages (*.pkg.tar.gz) with getting the same md5sum or sha checksum is surely not impossible - but much more difficult as in our current situation. So let's mak a first step! Gerhard
Gerhard Brauer schrieb:
For myself i don't accept the "md5sum is bad" argument as a "stopper" for each idea to provide a pacman secure concept ;-)
I wasn't going to stop you. Signed db files are an important first step. My point is that it was often suggested to move from md5 to something more secure. This suggestion was always rejected because we never used md5 for security. If we are going to implement your suggestion (and I vote that we do), we should really consider a better hash!
On Sat, Nov 29, 2008 at 8:37 AM, Thomas Bächler <thomas@archlinux.org> wrote:
Gerhard Brauer schrieb:
For myself i don't accept the "md5sum is bad" argument as a "stopper" for each idea to provide a pacman secure concept ;-)
I wasn't going to stop you. Signed db files are an important first step.
My point is that it was often suggested to move from md5 to something more secure. This suggestion was always rejected because we never used md5 for security. If we are going to implement your suggestion (and I vote that we do), we should really consider a better hash!
I think we're confusing things here. The checksums in pacman are only used for integrity, not security. I agree that the first step towards super-omg-secure packages would be switching to a different checksum, but sha1 might be deemed insecure soon too. Why not jump over that one to something like sha256? Now, once we do that, we're still stuck with the fact that anyone with a mirror can modify a package, and change the checksum, whatever it may be, in the DB file. So changing checksums isn't doing one iota of good until we can ensure that our list of checksums can't be changed. I talked to Dan about this briefly the other day. What we concluded was that the most proper and secure way to do this is as follows: * gerolde hosts a "developer keyring" * developers sign their individual packages with their own keys * the database file still contains the integrity checksums * the database file also includes package signing data The current proposition, with one key on gerolde used to sign the DB, is insecure in that it makes the developers the weak point. If all of gerolde, or even only those with sudo rights, can see this private key, all it takes is maliciousness on one person's part and all our packages are vulnerable. The whole point of a private key is just that - that it's private. If we suddenly make this key "semi-public", we open the door to lots of other issues, at the human level. When I last spoke to Dan, the biggest issue here was that gpg doesn't have a library interface. We'd have to call the binary directly from pacman. If we look at the above solution, we can leverage the XferCommand to cover this, without affecting pacman at all: * Create a download script that will download a package, check the integrity checksum, and verify the signature in the database using the archlinux.org keyring * Pacman will continue to work as is, considering the checksum is still in the DB All we'd need is to patch repo-add to include signature data in the DB. To do this properly, signatures should be uploaded with the package itself, from the packager's machine... hmmm
Am Sat, 29 Nov 2008 17:24:19 -0600 schrieb "Aaron Griffin" <aaronmgriffin@gmail.com>:
I talked to Dan about this briefly the other day. What we concluded was that the most proper and secure way to do this is as follows: * gerolde hosts a "developer keyring" * developers sign their individual packages with their own keys * the database file still contains the integrity checksums * the database file also includes package signing data
You were right: signing of packages must be done before it gets transported do any destination. That's the part of each developer IMHO. We trust them, we must trust them. Each developers public key must be available in a arch-devel keyring. Pacman itself must pick the right public key from this keyring (based of package signing data from the db) to verify the downloaded package. Arch overlord must have the power to revoke end delete the developers key from this keyring. repo-add at gerolde could check the correct signing of new packages as a "first instance". The public keyring must be available for the user side - best with pacman itself and/or as a seperate package.
The current proposition, with one key on gerolde used to sign the DB, is insecure in that it makes the developers the weak point. If all of gerolde, or even only those with sudo rights, can see this private key, all it takes is maliciousness on one person's part and all our packages are vulnerable.
The whole point of a private key is just that - that it's private. If we suddenly make this key "semi-public", we open the door to lots of other issues, at the human level.
AFAIK from Pierre is that repo-add i neither called directly by the developers - only from scripts when they upload their packages. The private key for signing the database files could be kept private by using a dedicated user which runs repo-add under his uid. In the scripts we could call repo-add with su/sudo and this uid. So the private key is in $HOME, only available for users with super powers on gerolde. If one must call repo-add directly this is also possible with su/sudo. But all in all: signing the database files must be done on gerolde from/during the process when they are created.
When I last spoke to Dan, the biggest issue here was that gpg doesn't have a library interface. We'd have to call the binary directly from pacman.
THAT's sound bad! I'm not a c programmer but is there no useable library from those on: http://www.gnupg.org/related_software/libraries.en.html ? What's with this gpgme also in [extra]?
If we look at the above solution, we can leverage the XferCommand to cover this, without affecting pacman at all: * Create a download script that will download a package, check the integrity checksum, and verify the signature in the database using the archlinux.org keyring * Pacman will continue to work as is, considering the checksum is still in the DB
All we'd need is to patch repo-add to include signature data in the DB. To do this properly, signatures should be uploaded with the package itself, from the packager's machine... hmmm
In the starting mail on arch-dev-public Pierre attached a quick patch and download script that i have tested with my own repo. This is working in this way that a whatever modified database file don't get installed as new data during -Syu when the signature could not be verified. No new database -> no new packages. Dirty things with such a solution (as special download script) is: libalpm/pacman download mechanism gets broken/useless, and the gpg output (and wget/curl output) is bad to integrate this way in pacman. Regards Gerhard
Am Sun, 30 Nov 2008 01:20:13 +0100 schrieb Gerhard Brauer <gerhard.brauer@web.de>:
Am Sat, 29 Nov 2008 17:24:19 -0600 schrieb "Aaron Griffin" <aaronmgriffin@gmail.com>:
All we'd need is to patch repo-add to include signature data in the DB. To do this properly, signatures should be uploaded with the package itself, from the packager's machine... hmmm
In the starting mail on arch-dev-public Pierre attached a quick patch and download script that i have tested with my own repo. This is working in this way that a whatever modified database file don't get installed as new data during -Syu when the signature could not be verified. No new database -> no new packages.
I think i misunderstood Aaron part. He mentioned that repo-add should also add the "signature data" to the db. Maybe we don't need special data to verify a package against a valid signature. We could sign a package inline, that means the tar.gz is enveloped by a signature (default --sign option). After/during verifying the tar.gz could be seperated from the signature with --output. The disadvantage with this methode is maybe that the package archiv could not be used directly without verifying it. On the other side we sign with --detach-sign which provides a .sig File releated to the signed pkg.tar.gz. Upload (and pacman download) then must handle two files. Database signature entries: Maybe we could use existing fields to get the proper key id from the public keyring to verify. Email oder packager data. But this could be only done when package is always and only signed by it's maintainer - and AFAIK we have situations where one developer build packages for a other. So you're right: the key id oder fingerprint of the actually developer/packager/signer must be store in the database file to get the proper key id. Regards Gerhard
Aaron Griffin schrieb:
I think we're confusing things here. The checksums in pacman are only used for integrity, not security. I agree that the first step towards super-omg-secure packages would be switching to a different checksum, but sha1 might be deemed insecure soon too. Why not jump over that one to something like sha256?
Once you sign the repo db file, the checksums are signed as well, so you cannot change the checksum without invalidating the db signature. If you would use a secure hash function, this adds a good layer of security (except for the trust issue).
Am So, 30.11.2008, 00:24, schrieb Aaron Griffin:
All we'd need is to patch repo-add to include signature data in the DB. To do this properly, signatures should be uploaded with the package itself, from the packager's machine... hmmm
perhaps i missed something, but wouldn´t be the easiest way to download the db.tar.gz directly from ftp.archlinux.org or another trusted server and the packages from the mirrors? something like a decentralized system. vlad
perhaps i missed something, but wouldn´t be the easiest way to download the db.tar.gz directly from ftp.archlinux.org or another trusted server and the packages from the mirrors? something like a decentralized system.
sorry. i wasn´t very explicit in my previous mail. my idea is this: first there should 2-3 trusted servers in case one fails or is offline. on these servers there should be a db.tar.gz repository with a hold-back time of 5-10 days. the db.tar.gz files should now look like <repo>-$(date).db.tar.gz. everytime a maintainer updates a package and a new db.tar.gz file is created, it goes to this db repository. when a mirror syncs, the latest <repo>-$(date).db.tar.gz is fetched. now when a user updates his/her system, pacman checks the <repo>-$(date).db.tar.gz file on the mirror, fetches this file form the db repository of a trusted server and then downloads the packages from the mirror. pacman compares the package md5sums with the ones in the db.tar.gz file from the trusted server and proceeds as usual. so one can also see if a mirror is corrupted and change the mirror and perhaps contact the maintainer. vlad
On Sun, 2008-11-30 at 04:22 +0100, vla@uni-bonn.de wrote:
Am So, 30.11.2008, 00:24, schrieb Aaron Griffin:
All we'd need is to patch repo-add to include signature data in the DB. To do this properly, signatures should be uploaded with the package itself, from the packager's machine... hmmm
perhaps i missed something, but wouldn´t be the easiest way to download the db.tar.gz directly from ftp.archlinux.org or another trusted server and the packages from the mirrors? something like a decentralized system.
I think ftp.archlinux.org can be pretty slow sometimes (compared to near-by mirrors), so wouldn't it be equally sufficient to just fetch the DB-checksum from archlinux.org? (Still not as secure as signed DBs though.)
Am Sun, 30 Nov 2008 12:54:34 +0100 schrieb Timm Preetz <timm@preetz.us>:
I think ftp.archlinux.org can be pretty slow sometimes (compared to near-by mirrors), so wouldn't it be equally sufficient to just fetch the DB-checksum from archlinux.org?
This is not possible cause mirrors sync times are different, so the result was: newer package versions in the db - but the package file is not available on users mirror. (I've had make the same request earlier in a bugtracker thread, without thinking a bit deeper... ;-) Regards Gerhard
hi all, On Sun, Nov 30, 2008 at 12:54:34PM +0100, Timm Preetz wrote:
I think ftp.archlinux.org can be pretty slow sometimes (compared to near-by mirrors), so wouldn't it be equally sufficient to just fetch the DB-checksum from archlinux.org?
(Still not as secure as signed DBs though.)
the db.tar.gz file is pretty small. the extra.db.tar.gz file is about 400kb. ok, it is also possible to ge only the db checksum, but the idea is that the db file itself should be in a trusted place. with 2-3 "trusted servers" users can choose a server in a near location (.org, .de or .fr for example). On Sun, Nov 30, 2008 at 01:40:07PM +0100, Gerhard Brauer wrote:
Am Sun, 30 Nov 2008 12:54:34 +0100 schrieb Timm Preetz <timm@preetz.us>:
I think ftp.archlinux.org can be pretty slow sometimes (compared to near-by mirrors), so wouldn't it be equally sufficient to just fetch the DB-checksum from archlinux.org?
This is not possible cause mirrors sync times are different, so the result was: newer package versions in the db - but the package file is not available on users mirror. (I've had make the same request earlier in a bugtracker thread, without thinking a bit deeper... ;-)
Regards Gerhard
yes, i also thought about that. that's why i suggested to establish a db file repository with a file retention of some days (mirrors ususally sync every 2h-24h). the new db.tar.gz now looks like <repo_name>-<file_creation_time>.db.tar.gz and the mirror fetches the latest db file from this repository when sync'ing. pacman checks the time (name) of the db file on the mirror and fetches this file from db file repo of a "trusted server". i don't think this is hard to implement. it's only a file name and md5sum comparison. imo it's the easiest way to do this. it's easier and less work than to sign packages or similar. as a byproduct one can keep track of existing mirrors and users can directly see if mirrors they use are trustworthy or not. just my 2 cents. vlad
Am Sun, 30 Nov 2008 15:09:04 +0100 schrieb vlad <vla@uni-bonn.de>:
yes, i also thought about that. that's why i suggested to establish a db file repository with a file retention of some days (mirrors ususally sync every 2h-24h).
...
just my 2 cents.
IMHO this could not be handled in practice. We will end in a zillion of different *.db.tar.gz files, cause the "main" db file could change every minute/hour by a dev and mirrors don't sync every day in practice. But more important: At the moment we can't guarantee the integrity of *one* db file, with your solution we can't guarantee it for 100 db files. That the file is (maybe) downloaded over a unsecure transport from a mirror (ftp.archlinux.org are mirrors too) that **could** be ftp.archlinux.org make it IMHO not more trusted.
vlad
Regards Gerhard
Aaron Griffin schrieb:
When I last spoke to Dan, the biggest issue here was that gpg doesn't have a library interface. We'd have to call the binary directly from pacman.
1) There is gpgme! But what does it do? It calls the gpg command line tool (iirc). 2) So what? Let's use gnutls or openssl. We could create an arch root certificate and sign the developers' keys with it, and use RSA or (my preference) DSA signatures. Then you can do it all on a library level.
i like the original idea of pierre. i had the same one ;-) because it's easier to implement and could be done quite quickly. it's quite time to shift to something a little more secure, even if it's not the *most* secure one. as soon the db is signed, we have a minimum security (not total i know, i read about the exploit in this thread) package signing could be a second step as it will take even longer to complete (more work to be done in pacman and more things to agree upon) in fact, i suggest a two steps approach.
On Sun, Nov 30, 2008 at 06:56, solsTiCe d'Hiver <solstice.dhiver@gmail.com> wrote:
i like the original idea of pierre. i had the same one ;-)
because it's easier to implement and could be done quite quickly. it's quite time to shift to something a little more secure, even if it's not the *most* secure one. as soon the db is signed, we have a minimum security (not total i know, i read about the exploit in this thread)
package signing could be a second step as it will take even longer to complete (more work to be done in pacman and more things to agree upon)
in fact, i suggest a two steps approach.
I agree. We can talk until we're blue in the face about the "ideal" way to do it, but it doesn't mean a thing if it's not implemented. Let's get *something* done, even if it's not ideal.
Am Sun, 30 Nov 2008 07:06:09 -0500 schrieb "Daenyth Blank" <daenyth+arch@gmail.com>:
On Sun, Nov 30, 2008 at 06:56, solsTiCe d'Hiver <solstice.dhiver@gmail.com> wrote:
i like the original idea of pierre. i had the same one ;-)
I agree. We can talk until we're blue in the face about the "ideal" way to do it, but it doesn't mean a thing if it's not implemented. Let's get *something* done, even if it's not ideal.
You are both right. Let's make a first step with signing the database file - either gpg or RSA/DSA framework. Also let's maybe switch package checksumming from md5 to maybe sha512 to get a higher secure of our than signed db/checksum. In the "ideal solution, the golden way" the database must be signed as well. So let's start with this. We could get expierience with handling in repo-add, pacman etc. In a further step we could think about package signing. Maybe we/you could implement this as a christmas gift to us users? ;-) Regards Gerhard
participants (8)
-
Aaron Griffin
-
Daenyth Blank
-
Gerhard Brauer
-
solsTiCe d'Hiver
-
Thomas Bächler
-
Timm Preetz
-
vla＠uni-bonn.de
-
vlad