On Tue, Jul 21, 2015 at 8:54 PM, Allan McRae <allan@archlinux.org> wrote:
I searched the archives, but I can not find why we stored the package PGP signatures base64'd in the repo database rather than downloading them as needed. Signatures are responsible for ~55% of the Arch repo database size, so I am guessing there must have been a tradeoff.
Can anyone provide insight to this? It was 2008...
2008 or 2011? I see this being read first in commit 39ce9b3afc6. The commit to scripts is authored earlier, but committed much later. Doesn't really matter I suppose. :) I can't be certain what my thinking was, but I can think of a few possible reasons. Not sure of their validity, but: 1) Fewer downloads necessary when installing/upgrading. FTP was still a thing at the time, and it was super-slow by comparison to HTTP on grabbing more files given the way the protocol works. 2) If/when signing databases is a thing, you want to sign the whole database so you can have end-to-end tamper detection. Else anyone could drop a different 'pacman-4.2.1-1' signed package in place, and you wouldn't be able to tell the difference. If I feel confident signing a database, I should feel confident you can't change what that database refers to. With that said, there are checksums in here too, so you couldn't really do this, but we don't currently run the checksum verification if we do signature verification. This could change. 3) When I started work on all this, I had it in my head that signatures were relatively small, so it made sense to inline them. Mine are only 72 bytes, for instance, while other packagers are much longer. Modern keys generate 287 or 543 byte signatures, which are 8 times larger than I originally thought. [1] More random stuff: * https://wiki.debian.org/SecureApt looks like Debian only signs the DB, and then from there, it uses the checksums to verify the packages. Hope that helps. -Dan [1] archweb=# select avg(length(signature_bytes)) as len, packager_str from packages group by packager_str order by 1; len | packager_str -----------------------+---------------------------------------------------------- 71.9500000000000000 | Juergen Hoetzel <juergen@archlinux.org> 71.9789473684210526 | Martin Wimpress <code@flexion.org> 72.0000000000000000 | Massimiliano Torromeo <massimiliano.torromeo@gmail.com> 72.0000000000000000 | Dan McGee <dan@archlinux.org> 72.0000000000000000 | Fabio Castelli (Muflone) <muflone@archlinux.org> 87.9600000000000000 | Thorsten Töpper <atsutane@freethoughts.de> 95.9898648648648649 | Gaetan Bisson <bisson@archlinux.org> 96.0000000000000000 | Guillaume ALAUX <guillaume@archlinux.org> 286.9230769230769231 | Alexandre Filgueira <alexfilgueira@cinnarch.com> 286.9666666666666667 | Connor Behan <connor.behan@gmail.com> 286.9806763285024155 | Balló György <ballogyor+arch@gmail.com> 286.9821428571428571 | Maxime Gauduin <alucryd@gmail.com> 286.9827586206896552 | Jonathan Steel <jsteel@archlinux.org> 286.9836065573770492 | Ronald van Haren <ronald@archlinux.org> 286.9908256880733945 | Laurent Carlier <lordheavym@gmail.com> 286.9911894273127753 | Bartłomiej Piotrowski <bpiotrowski@archlinux.org> 286.9922879177377892 | Eric Belanger <eric@archlinux.org> 286.9945355191256831 | Jan Alexander Steffens (heftig) <jan.steffens@gmail.com> 286.9946070878274268 | Antonio Rojas <arojas@archlinux.org> 286.9956896551724138 | Andreas Radke <andyrtr@archlinux.org> 286.9966499162479062 | Evangelos Foutras <evangelos@foutrelis.com> 286.9968454258675079 | Jan de Groot <jgc@archlinux.org> 287.0000000000000000 | Daniel Isenmann <daniel@archlinux.org> 287.0000000000000000 | Lukas Jirkovsky <l.jirkovsky@gmail.com> 287.0000000000000000 | Tom Gundersen <teg@jklm.no> 287.0000000000000000 | Christian Hesse <arch@eworm.de> 287.0000000000000000 | Dicebot <public@dicebot.lv> 287.0000000000000000 | Giovanni Scafora <giovanni@archlinux.org> 287.0000000000000000 | Kyle Keen <keenerd@gmail.com> 287.0000000000000000 | speps <speps@aur.archlinux.org> 287.0000000000000000 | Bartłomiej Piotrowski <barthalion@gmail.com> 287.0000000000000000 | Jonathan Steel <mail@jsteel.org> 287.0000000000000000 | Pierre Schmitz <pierre@archlinux.de> 287.0000000000000000 | Михаил Страшун <public@dicebot.lv> 287.0000000000000000 | Christian Hesse (leda.eworm.de) <arch@eworm.de> 287.0000000000000000 | Andrzej Giniewicz <gginiu@gmail.com> 287.0000000000000000 | Jelle van der Waa <jelle@vdwaa.nl> 287.0000000000000000 | Ionut Biru <ibiru@archlinux.org> 287.0000000000000000 | Bartłomiej Piotrowski <b@bpiotrowski.pl> 287.0000000000000000 | schuay <jakob.gruber@gmail.com> 287.0000000000000000 | Daniel Wallace <danielwallace at gtmanfred dot com> 287.0000000000000000 | Alexander F Rødseth <rodseth@gmail.com> 287.0000000000000000 | Gerardo Exequiel Pozzi <djgera@archlinux.org> 287.0000000000000000 | Allan McRae <allan@archlinux.org> 287.0000000000000000 | Maxime Gauduin <alucryd@archlinux.org> 287.0000000000000000 | Andrea Scarpino <andrea@archlinux.org> 287.0000000000000000 | Angel Velasquez <angvp@archlinux.org> 287.0000000000000000 | Alexander Rødseth <rodseth@gmail.com> 287.0000000000000000 | Timothy Redaelli <timothy.redaelli@gmail.com> 287.0000000000000000 | Tobias Powalowski <tpowa@archlinux.org> 287.0000000000000000 | Rashif Rahman (Ray) <schiv@archlinux.org> 287.0000000000000000 | Dave Reisner <dreisner@archlinux.org> 386.9024390243902439 | Unknown Packager 538.9859813084112150 | Sébastien Luttringer <seblu@seblu.net> 542.9722222222222222 | Levente Polyak <anthraxx@archlinux.org> 542.9867109634551495 | Anatol Pomozov <anatol.pomozov@gmail.com> 542.9946476360392507 | Felix Yan <felixonmars@archlinux.org> 542.9985337243401760 | Felix Yan <felixonmars@gmail.com> 542.9987021414665801 | Sergej Pupykin <pupykin.s+arch@gmail.com> 543.0000000000000000 | Rémy Oudompheng <remy@archlinux.org> 543.0000000000000000 | Jaroslav Lichtblau<dragonlord@aur.archlinux.org> 543.0000000000000000 | Thomas Bächler <thomas@archlinux.org> 543.0000000000000000 | Jaroslav Lichtblau <dragonlord@aur.archlinux.org> 543.0000000000000000 | Lukas Fleischer <lfleischer@archlinux.org> 543.0000000000000000 | Florian Pritz <bluewind@xinu.at> 543.0000000000000000 | Lukas Fleischer <archlinux@cryptocrack.de> 543.0000000000000000 | Evgeniy Alekseev <arcanis.arch@gmail.com> 543.0000000000000000 | Thomas Dziedzic <gostrc@gmail.com> 543.0000000000000000 | Xyne 543.0000000000000000 | Sven-Hendrik Haase <sh@lutzhaase.com> 543.0000000000000000 | BlackEagle <ike DOT devolder AT gmail DOT com> 543.0000000000000000 | Evgeniy Alekseev <arcanis@archlinux.org> 543.0000000000000000 | Jaroslav Lichtblau <svetlemodry@archlinux.org> 543.0000000000000000 | Daniel Micay <danielmicay@gmail.com> 639.0000000000000000 | Jerome Leclanche <jerome@leclan.ch> 1055.0000000000000000 | Johannes Löthberg <johannes@kyriasis.com> (76 rows)