[pacman-dev] Signature within repo databases?

[Dan McGee]

On Tue, Jul 21, 2015 at 8:54 PM, Allan McRae <allan at archlinux.org> wrote:
> I searched the archives, but I can not find why we stored the package
> PGP signatures base64'd in the repo database rather than downloading
> them as needed.  Signatures are responsible for ~55% of the Arch repo
> database size, so I am guessing there must have been a tradeoff.
>
> Can anyone provide insight to this?   It was 2008...

2008 or 2011? I see this being read first in commit 39ce9b3afc6. The
commit to scripts is authored earlier, but committed much later.
Doesn't really matter I suppose. :)

I can't be certain what my thinking was, but I can think of a few
possible reasons. Not sure of their validity, but:

1) Fewer downloads necessary when installing/upgrading. FTP was still
a thing at the time, and it was super-slow by comparison to HTTP on
grabbing more files given the way the protocol works.
2) If/when signing databases is a thing, you want to sign the whole
database so you can have end-to-end tamper detection. Else anyone
could drop a different 'pacman-4.2.1-1' signed package in place, and
you wouldn't be able to tell the difference. If I feel confident
signing a database, I should feel confident you can't change what that
database refers to. With that said, there are checksums in here too,
so you couldn't really do this, but we don't currently run the
checksum verification if we do signature verification. This could
change.
3) When I started work on all this, I had it in my head that
signatures were relatively small, so it made sense to inline them.
Mine are only 72 bytes, for instance, while other packagers are much
longer. Modern keys generate 287 or 543 byte signatures, which are 8
times larger than I originally thought. [1]

More random stuff:
* https://wiki.debian.org/SecureApt looks like Debian only signs the
DB, and then from there, it uses the checksums to verify the packages.

Hope that helps.

-Dan

[1]
archweb=# select avg(length(signature_bytes)) as len, packager_str
from packages group by packager_str order by 1;
          len          |                       packager_str
-----------------------+----------------------------------------------------------
   71.9500000000000000 | Juergen Hoetzel <juergen at archlinux.org>
   71.9789473684210526 | Martin Wimpress <code at flexion.org>
   72.0000000000000000 | Massimiliano Torromeo <massimiliano.torromeo at gmail.com>
   72.0000000000000000 | Dan McGee <dan at archlinux.org>
   72.0000000000000000 | Fabio Castelli (Muflone) <muflone at archlinux.org>
   87.9600000000000000 | Thorsten Töpper <atsutane at freethoughts.de>
   95.9898648648648649 | Gaetan Bisson <bisson at archlinux.org>
   96.0000000000000000 | Guillaume ALAUX <guillaume at archlinux.org>
  286.9230769230769231 | Alexandre Filgueira <alexfilgueira at cinnarch.com>
  286.9666666666666667 | Connor Behan <connor.behan at gmail.com>
  286.9806763285024155 | Balló György <ballogyor+arch at gmail.com>
  286.9821428571428571 | Maxime Gauduin <alucryd at gmail.com>
  286.9827586206896552 | Jonathan Steel <jsteel at archlinux.org>
  286.9836065573770492 | Ronald van Haren <ronald at archlinux.org>
  286.9908256880733945 | Laurent Carlier <lordheavym at gmail.com>
  286.9911894273127753 | Bartłomiej Piotrowski <bpiotrowski at archlinux.org>
  286.9922879177377892 | Eric Belanger <eric at archlinux.org>
  286.9945355191256831 | Jan Alexander Steffens (heftig)
<jan.steffens at gmail.com>
  286.9946070878274268 | Antonio Rojas <arojas at archlinux.org>
  286.9956896551724138 | Andreas Radke <andyrtr at archlinux.org>
  286.9966499162479062 | Evangelos Foutras <evangelos at foutrelis.com>
  286.9968454258675079 | Jan de Groot <jgc at archlinux.org>
  287.0000000000000000 | Daniel Isenmann <daniel at archlinux.org>
  287.0000000000000000 | Lukas Jirkovsky <l.jirkovsky at gmail.com>
  287.0000000000000000 | Tom Gundersen <teg at jklm.no>
  287.0000000000000000 | Christian Hesse <arch at eworm.de>
  287.0000000000000000 | Dicebot <public at dicebot.lv>
  287.0000000000000000 | Giovanni Scafora <giovanni at archlinux.org>
  287.0000000000000000 | Kyle Keen <keenerd at gmail.com>
  287.0000000000000000 | speps <speps at aur.archlinux.org>
  287.0000000000000000 | Bartłomiej Piotrowski <barthalion at gmail.com>
  287.0000000000000000 | Jonathan Steel <mail at jsteel.org>
  287.0000000000000000 | Pierre Schmitz <pierre at archlinux.de>
  287.0000000000000000 | Михаил Страшун <public at dicebot.lv>
  287.0000000000000000 | Christian Hesse (leda.eworm.de) <arch at eworm.de>
  287.0000000000000000 | Andrzej Giniewicz <gginiu at gmail.com>
  287.0000000000000000 | Jelle van der Waa <jelle at vdwaa.nl>
  287.0000000000000000 | Ionut Biru <ibiru at archlinux.org>
  287.0000000000000000 | Bartłomiej Piotrowski <b at bpiotrowski.pl>
  287.0000000000000000 | schuay <jakob.gruber at gmail.com>
  287.0000000000000000 | Daniel Wallace <danielwallace at gtmanfred dot com>
  287.0000000000000000 | Alexander F Rødseth <rodseth at gmail.com>
  287.0000000000000000 | Gerardo Exequiel Pozzi <djgera at archlinux.org>
  287.0000000000000000 | Allan McRae <allan at archlinux.org>
  287.0000000000000000 | Maxime Gauduin <alucryd at archlinux.org>
  287.0000000000000000 | Andrea Scarpino <andrea at archlinux.org>
  287.0000000000000000 | Angel Velasquez <angvp at archlinux.org>
  287.0000000000000000 | Alexander Rødseth <rodseth at gmail.com>
  287.0000000000000000 | Timothy Redaelli <timothy.redaelli at gmail.com>
  287.0000000000000000 | Tobias Powalowski <tpowa at archlinux.org>
  287.0000000000000000 | Rashif Rahman (Ray) <schiv at archlinux.org>
  287.0000000000000000 | Dave Reisner <dreisner at archlinux.org>
  386.9024390243902439 | Unknown Packager
  538.9859813084112150 | Sébastien Luttringer <seblu at seblu.net>
  542.9722222222222222 | Levente Polyak <anthraxx at archlinux.org>
  542.9867109634551495 | Anatol Pomozov <anatol.pomozov at gmail.com>
  542.9946476360392507 | Felix Yan <felixonmars at archlinux.org>
  542.9985337243401760 | Felix Yan <felixonmars at gmail.com>
  542.9987021414665801 | Sergej Pupykin <pupykin.s+arch at gmail.com>
  543.0000000000000000 | Rémy Oudompheng <remy at archlinux.org>
  543.0000000000000000 | Jaroslav Lichtblau<dragonlord at aur.archlinux.org>
  543.0000000000000000 | Thomas Bächler <thomas at archlinux.org>
  543.0000000000000000 | Jaroslav Lichtblau <dragonlord at aur.archlinux.org>
  543.0000000000000000 | Lukas Fleischer <lfleischer at archlinux.org>
  543.0000000000000000 | Florian Pritz <bluewind at xinu.at>
  543.0000000000000000 | Lukas Fleischer <archlinux at cryptocrack.de>
  543.0000000000000000 | Evgeniy Alekseev <arcanis.arch at gmail.com>
  543.0000000000000000 | Thomas Dziedzic <gostrc at gmail.com>
  543.0000000000000000 | Xyne
  543.0000000000000000 | Sven-Hendrik Haase <sh at lutzhaase.com>
  543.0000000000000000 | BlackEagle <ike DOT devolder AT gmail DOT com>
  543.0000000000000000 | Evgeniy Alekseev <arcanis at archlinux.org>
  543.0000000000000000 | Jaroslav Lichtblau <svetlemodry at archlinux.org>
  543.0000000000000000 | Daniel Micay <danielmicay at gmail.com>
  639.0000000000000000 | Jerome Leclanche <jerome at leclan.ch>
 1055.0000000000000000 | Johannes Löthberg <johannes at kyriasis.com>
(76 rows)