Hello,
I run a repository locally that I would like to share to the public.
The build is mostly automated. That's why I don't want to sign each individual package. The private key is not stored on the build machine and I want to sign the resulting stuff externally.
The easiest way would be actually to just manually sign the database file. As this file includes all checksums of the individual packages, I think this is as secure as signing every package, right?
Thanks in advance
Manuel
On 6/16/19 5:03 AM, Manuel Reimer wrote:
Hello,
I run a repository locally that I would like to share to the public.
The build is mostly automated. That's why I don't want to sign each individual package. The private key is not stored on the build machine and I want to sign the resulting stuff externally.
The easiest way would be actually to just manually sign the database file. As this file includes all checksums of the individual packages, I think this is as secure as signing every package, right?
Thanks in advance
Manuel
theoretically, your thought process is sound. unfortunately, pacman doesn't verify like this (to my knowledge; someone feel free to correct me).
but there's nothing necessitating you sign the package on the build machine, technically. you could fetch the repo DB, grab the checksums inside (i believe they contain a metadata tree and .PKGINFO; been a while since i explored the format), fetch the package itself into memory, and if the checksum matches, you can create a detached signature from that item in memory, then upload that signature. (i think? you might meed to regen the repo.db; not sure if it tracks sigs.) there is, of course, the trouble of not being able to cryptographically verify the integrity of the checksums inside the repo DB (since the packages are being fetches from a remote source and *technically* possibly could have been tampered along with the repo DB). this is why signing is done at build time - it at least removes that vector (notwithstanding local tampering, but that's time-sensitive and a dedicated build box separate from a repo server is a lot more resistant).
i can create a python PoC of this if that'd be easier to understand of the "remote signing".
BUT. TL;DR "pacman doesn't work like that" and it's generally safer practice to build and sign (and build a repo db) on a different box, then push to the repo server.
On 6/16/19 10:44 AM, brent s. wrote:
On 6/16/19 5:03 AM, Manuel Reimer wrote:
Hello,
I run a repository locally that I would like to share to the public.
The build is mostly automated. That's why I don't want to sign each individual package. The private key is not stored on the build machine and I want to sign the resulting stuff externally.
The easiest way would be actually to just manually sign the database file. As this file includes all checksums of the individual packages, I think this is as secure as signing every package, right?
Thanks in advance
Manuel
theoretically, your thought process is sound. unfortunately, pacman doesn't verify like this (to my knowledge; someone feel free to correct me).
but there's nothing necessitating you sign the package on the build machine, technically. you could fetch the repo DB, grab the checksums inside (i believe they contain a metadata tree and .PKGINFO; been a while since i explored the format), fetch the package itself into memory, and if the checksum matches, you can create a detached signature from that item in memory, then upload that signature. (i think? you might meed to regen the repo.db; not sure if it tracks sigs.) there is, of course, the trouble of not being able to cryptographically verify the integrity of the checksums inside the repo DB (since the packages are being fetches from a remote source and *technically* possibly could have been tampered along with the repo DB). this is why signing is done at build time - it at least removes that vector (notwithstanding local tampering, but that's time-sensitive and a dedicated build box separate from a repo server is a lot more resistant).
i can create a python PoC of this if that'd be easier to understand of the "remote signing".
BUT. TL;DR "pacman doesn't work like that" and it's generally safer practice to build and sign (and build a repo db) on a different box, then push to the repo server.
I have no idea what you just said.
...
Manuel -- makepkg has a --sign option, but all it actually does is this:
# pretend it actually uses ${pkgname}-${fullpkgver}-${pkgarch}${PKGEXT} for pkgfile in *.pkg.tar.xz; do gpg --detach-sign --no-armor "${pkgfile}" done
You can run gpg --detach-sign --no-armor yourself right before you repo-add, it makes no difference whether you use makepkg's convenience function for this. As a matter of fact, if you use clean chroot builds then you possibly don't want to copy your private key to the chroot, and anyway there have IIRC been bugs with signing in a chroot, so the devtools scripts do not do signing in the chroot and the official upload scripts for core/extra/community will do gpg --detach--sign --no-armor by hand, so you are in good company!
That being said, if you have signed the repository db then as you mentioned the sha256 checksums for the package file are securely signed, so you are guaranteed that use of pacman -S pkgname will securely verify that it is installing the package the repo-add user expected to provide when running repo-add.
What is your threat model? These things will not be protected against:
- people installing the package file directly, as such: pacman -U https://example.com/foopkg-1-1-x86_64.pkg.tar.xz - An attacker with local filesystem access on the signing/hosting server can retroactively replace *all* packages built at any date, and trick you into signing a new repo DB referencing them. - In shared packaging situations, like when a team of dozens of people all upload packages, you want to be able to verify who signed each package, as opposed to only verifying that the last person to update the repository asserted that all other packages are good and backed by his/her good name -- this does not concern you.
On June 16, 2019 5:57:34 PM GMT+02:00, Eli Schwartz via arch-general arch-general@archlinux.org wrote:
That being said, if you have signed the repository db then as you mentioned the sha256 checksums for the package file are securely signed, so you are guaranteed that use of pacman -S pkgname will securely verify that it is installing the package the repo-add user expected to provide when running repo-add.
What is your threat model? These things will not be protected against:
- people installing the package file directly, as such:
pacman -U https://example.com/foopkg-1-1-x86_64.pkg.tar.xz
- An attacker with local filesystem access on the signing/hosting
server can retroactively replace *all* packages built at any date, and trick you into signing a new repo DB referencing them.
- In shared packaging situations, like when a team of dozens of people
all upload packages, you want to be able to verify who signed each package, as opposed to only verifying that the last person to update the repository asserted that all other packages are good and backed by his/her good name -- this does not concern you.
An important side note: This will only really help if users of the repo have set the repository SigLevel to Required (which is not the default). When using the default of Optional a MitM attacker can just drop signatures for that database, which obviously is much much much easier to achieve for non https mirrors.
Cheers, Levente
On 16.06.19 17:57, Eli Schwartz via arch-general wrote:
As a matter of fact, if you use clean chroot builds then you possibly don't want to copy your private key to the chroot, and anyway there have IIRC been bugs with signing in a chroot, so the devtools scripts do not do signing in the chroot and the official upload scripts for core/extra/community will do gpg --detach--sign --no-armor by hand, so you are in good company!
I don't do builds in "chroot" environment. TBH I never managed to get this to build completely automatic. The problem is that the existing scripts start off in a unprivileged context and then require me to enter the root password when entering the chroot.
My own concept does it the other way around. Main script runs as root and it forks off unprivileged processes to do the build. After building this forked process exits and the "root-privileged" host process takes over again to prepare for the next package build. This includes required dependency installs.
https://github.com/M-Reimer/repo-make
All this runs on a dedicated "Build VM" on Oracle Virtual Box.
Yes, I get "clean" builds with this, as between every build step, all unneeded dependency packages are uninstalled.
Maybe there are other fully automatic systems out there, but my script served me well, so far.
I was just wondering if there is any way to not have to have the private key on this build VM at all. But I guess if something goes wrong on this VM, then security is gone anyway.
But thanks for the information about how the signing works. I think I'll move this out of the "unprivileged build process" and do the signing in the root-privileged main process. This way the dedicated build user of this build VM does not have access to the private key.
Manuel
On 6/17/19 11:11 AM, Manuel Reimer wrote:
On 16.06.19 17:57, Eli Schwartz via arch-general wrote:
As a matter of fact, if you use clean chroot builds then you possibly don't want to copy your private key to the chroot, and anyway there have IIRC been bugs with signing in a chroot, so the devtools scripts do not do signing in the chroot and the official upload scripts for core/extra/community will do gpg --detach--sign --no-armor by hand, so you are in good company!
I don't do builds in "chroot" environment. TBH I never managed to get this to build completely automatic. The problem is that the existing scripts start off in a unprivileged context and then require me to enter the root password when entering the chroot.
My own concept does it the other way around. Main script runs as root and it forks off unprivileged processes to do the build. After building this forked process exits and the "root-privileged" host process takes over again to prepare for the next package build. This includes required dependency installs.
https://github.com/M-Reimer/repo-make
All this runs on a dedicated "Build VM" on Oracle Virtual Box.
Yes, I get "clean" builds with this, as between every build step, all unneeded dependency packages are uninstalled.
This is a perfectly suitable implementation of a clean build, so if it works for you, great!
That being said, it's possible to configure sudo to run makechrootpkg, but only makechrootpkg, as root. Or run SUDO_USER=... SUDO_UID=... makechrootpkg.
Maybe there are other fully automatic systems out there, but my script served me well, so far.
I was just wondering if there is any way to not have to have the private key on this build VM at all. But I guess if something goes wrong on this VM, then security is gone anyway.
Yes -- do all signing locally, after the package leaves the build VM. If something goes wrong on the VM, you can remove the relevant packages without, say, revoking your key, so the security issue is less drastic.
But thanks for the information about how the signing works. I think I'll move this out of the "unprivileged build process" and do the signing in the root-privileged main process. This way the dedicated build user of this build VM does not have access to the private key.
On 17.06.19 18:18, Eli Schwartz via arch-general wrote:
That being said, it's possible to configure sudo to run makechrootpkg, but only makechrootpkg, as root. Or run SUDO_USER=... SUDO_UID=... makechrootpkg.
I've tried several times to just launch makechrootpkg with root privileges directly. As makechrootpkg drops to a unprivileged user inside the chroot, this should be perfectly safe.
But I always ran into errors saying that makepkg is not allowed to be run as root.
Does your SUDO_USER=... SUDO_UID=... command line allow to directly launch as root without needing sudo at all? This is what I would need to make my autobuild work.
Yes -- do all signing locally, after the package leaves the build VM. If something goes wrong on the VM, you can remove the relevant packages without, say, revoking your key, so the security issue is less drastic.
This would also be a possible way. Sign packages where the signature is outdated, delete signatures that don't belong to packages and finally repo-add the whole stuff after deleting the db file.
Is there a better tool as repo-add/repo-remove? I've been searching for some "repo-update" tool for quite a while now. A smart tool which doesn't recreate stuff and just updates a DB file would be pretty handy.
Manuel
On 6/17/19 12:38 PM, Manuel Reimer wrote:
On 17.06.19 18:18, Eli Schwartz via arch-general wrote:
That being said, it's possible to configure sudo to run makechrootpkg, but only makechrootpkg, as root. Or run SUDO_USER=... SUDO_UID=... makechrootpkg.
I've tried several times to just launch makechrootpkg with root privileges directly. As makechrootpkg drops to a unprivileged user inside the chroot, this should be perfectly safe.
But I always ran into errors saying that makepkg is not allowed to be run as root.
Does your SUDO_USER=... SUDO_UID=... command line allow to directly launch as root without needing sudo at all? This is what I would need to make my autobuild work.
makechrootpkg uses the SUDO_USER/SUDO_UID variables to check which user it should use when dropping privileges while running makepkg --verifysource. By setting the variables, you thereby pretend to makechrootpkg that it has been run via sudo.
Not doing *anything* to check which user to drop privileges to, is the reason why running makechrootpkg as root is usually not going to work.
Yes -- do all signing locally, after the package leaves the build VM. If something goes wrong on the VM, you can remove the relevant packages without, say, revoking your key, so the security issue is less drastic.
This would also be a possible way. Sign packages where the signature is outdated, delete signatures that don't belong to packages and finally repo-add the whole stuff after deleting the db file.
Is there a better tool as repo-add/repo-remove? I've been searching for some "repo-update" tool for quite a while now. A smart tool which doesn't recreate stuff and just updates a DB file would be pretty handy.
repo-add generally works pretty well, it doesn't recreate stuff anyway -- it unpacks the DB, adds the files you've specified to the DB, and then repacks the DB. If you're looking for something which scans a directory to find files which need to be updated, you can try "repose", but it has conflicting behavior as compared to repo-add, so you cannot mix and match repo-add and repose.
arch-general@lists.archlinux.org