[pacman-dev] [PATCH 1/2] Note that checksums from "makepkg -g" are not ideal
Generating checksums with "makepkg -g" only determines that the user of a PKGBUILD has the same file as the packager (assuming no collision). This means an upstream source could be maliciously changed and passed on as valid by a PKGBUILD. To avoid this, it is essential that any checksums used in a PKGBUILD are as provided by upstream. Signed-off-by: Allan McRae <allan@archlinux.org> --- doc/PKGBUILD.5.asciidoc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/PKGBUILD.5.asciidoc b/doc/PKGBUILD.5.asciidoc index ef53c0ee..abe2ab52 100644 --- a/doc/PKGBUILD.5.asciidoc +++ b/doc/PKGBUILD.5.asciidoc @@ -152,7 +152,9 @@ contain whitespace characters. file integrity during subsequent builds. If 'SKIP' is put in the array in place of a normal hash, the integrity check for that source file will be skipped. To easily generate md5sums, run ``makepkg -g >> PKGBUILD''. - If desired, move the md5sums line to an appropriate location. + If desired, move the md5sums line to an appropriate location. Note that + checksums generated by "makepkg -g" provide little security benefit. All + checksum values should be as provided by the software developer. *sha1sums, sha224sums, sha256sums, sha384sums, sha512sums, b2sums (arrays)*:: Alternative integrity checks that makepkg supports; these all behave -- 2.25.0
Checksums arrays should be filled with values provided by upstream. We currently have md5 set as an unsecure default, and are constantly asked to change it to sha2. However, just changing the default to a stronger checksum gives the user the impression that "makepkg -g" checksums are perfect. Instead, change the default checksum to a CRC, to make it clear that any checksum generated purely by "makepkg -g" is not ideal. Signed-off-by: Allan McRae <allan@archlinux.org> --- doc/PKGBUILD.5.asciidoc | 14 +++++++------- doc/makepkg.conf.5.asciidoc | 2 +- etc/makepkg.conf.in | 4 ++-- scripts/libmakepkg/util/schema.sh.in | 2 +- scripts/makepkg.sh.in | 4 ++-- 5 files changed, 13 insertions(+), 13 deletions(-) diff --git a/doc/PKGBUILD.5.asciidoc b/doc/PKGBUILD.5.asciidoc index abe2ab52..4d03f8ed 100644 --- a/doc/PKGBUILD.5.asciidoc +++ b/doc/PKGBUILD.5.asciidoc @@ -118,7 +118,7 @@ systems (see below). + Additional architecture-specific sources can be added by appending an underscore and the architecture name e.g., 'source_x86_64=()'. There must be a -corresponding integrity array with checksums, e.g. 'md5sums_x86_64=()'. +corresponding integrity array with checksums, e.g. 'cksums_x86_64=()'. + It is also possible to change the name of the downloaded file, which is helpful with weird URLs and for handling multiple source files with the same @@ -146,19 +146,19 @@ contain whitespace characters. listed here will not be extracted with the rest of the source files. This is useful for packages that use compressed data directly. -*md5sums (array)*:: - This array contains an MD5 hash for every source file specified in the +*cksums (array)*:: + This array contains CRC checksums for every source file specified in the source array (in the same order). makepkg will use this to verify source file integrity during subsequent builds. If 'SKIP' is put in the array in place of a normal hash, the integrity check for that source file will - be skipped. To easily generate md5sums, run ``makepkg -g >> PKGBUILD''. - If desired, move the md5sums line to an appropriate location. Note that + be skipped. To easily generate cksums, run ``makepkg -g >> PKGBUILD''. + If desired, move the cksums line to an appropriate location. Note that checksums generated by "makepkg -g" provide little security benefit. All checksum values should be as provided by the software developer. -*sha1sums, sha224sums, sha256sums, sha384sums, sha512sums, b2sums (arrays)*:: +*md5sums, sha1sums, sha224sums, sha256sums, sha384sums, sha512sums, b2sums (arrays)*:: Alternative integrity checks that makepkg supports; these all behave - similar to the md5sums option described above. To enable use and generation + similar to the cksums option described above. To enable use and generation of these checksums, be sure to set up the `INTEGRITY_CHECK` option in linkman:makepkg.conf[5]. diff --git a/doc/makepkg.conf.5.asciidoc b/doc/makepkg.conf.5.asciidoc index b7496324..04cc5ea9 100644 --- a/doc/makepkg.conf.5.asciidoc +++ b/doc/makepkg.conf.5.asciidoc @@ -192,7 +192,7 @@ Options **INTEGRITY_CHECK=(**check1 ...**)**:: File integrity checks to use. Multiple checks may be specified; this affects both generation and checking. The current valid options are: - `md5`, `sha1`, `sha224`, `sha256`, `sha384`, `sha512`, and `b2`. + `ck`, `md5`, `sha1`, `sha224`, `sha256`, `sha384`, `sha512`, and `b2`. **STRIP_BINARIES=**"--strip-all":: Options to be used when stripping binaries. See linkman:strip[1] diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in index caf5114b..1c7988d2 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -89,8 +89,8 @@ BUILDENV=(!distcc color !ccache check !sign) # OPTIONS=(strip docs libtool staticlibs emptydirs zipman purge !debug) -#-- File integrity checks to use. Valid: md5, sha1, sha224, sha256, sha384, sha512, b2 -INTEGRITY_CHECK=(md5) +#-- File integrity checks to use. Valid: ck, md5, sha1, sha224, sha256, sha384, sha512, b2 +INTEGRITY_CHECK=(ck) #-- Options to be used when stripping binaries. See `man strip' for details. STRIP_BINARIES="@STRIP_BINARIES@" #-- Options to be used when stripping shared libraries. See `man strip' for details. diff --git a/scripts/libmakepkg/util/schema.sh.in b/scripts/libmakepkg/util/schema.sh.in index b2f119cf..02bfdb86 100644 --- a/scripts/libmakepkg/util/schema.sh.in +++ b/scripts/libmakepkg/util/schema.sh.in @@ -26,7 +26,7 @@ LIBRARY=${LIBRARY:-'@libmakepkgdir@'} source "$LIBRARY/util/util.sh" -known_hash_algos=({md5,sha{1,224,256,384,512},b2}) +known_hash_algos=({ck,md5,sha{1,224,256,384,512},b2}) pkgbuild_schema_arrays=(arch backup checkdepends conflicts depends groups license makedepends noextract optdepends options diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index a6de7823..7fa791e1 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1195,8 +1195,8 @@ unset "${known_hash_algos[@]/%/sums}" unset -f pkgver prepare build check package "${!package_@}" unset "${!makedepends_@}" "${!depends_@}" "${!source_@}" "${!checkdepends_@}" unset "${!optdepends_@}" "${!conflicts_@}" "${!provides_@}" "${!replaces_@}" -unset "${!md5sums_@}" "${!sha1sums_@}" "${!sha224sums_@}" "${!sha256sums_@}" -unset "${!sha384sums_@}" "${!sha512sums_@}" "${!b2sums_@}" +unset "${!cksums_@}" "${!md5sums_@}" "${!sha1sums_@}" "${!sha224sums_@}" +unset "${!sha256sums_@}" "${!sha384sums_@}" "${!sha512sums_@}" "${!b2sums_@}" BUILDFILE=${BUILDFILE:-$BUILDSCRIPT} if [[ ! -f $BUILDFILE ]]; then -- 2.25.0
A potentially unforeseen consequence: At present, it is possible (albeit with use of tools that aren't as of present date publicly released, something I hope to change in the future) to use the Nix build system to build Arch packages (with some caveats, but generally manageable ones for folks who don't need these packages to be what Nix calls "pure"). Nix identifies downloaded content by hash -- only build-time processes which can state a cryptographically strong checksum of their intended output prior to time of invocation are allowed to connect to the internet during the build process itself. cksum is not supported by Nix, whereas the other checksums supported by Arch are. Thus, moving to cksum -- quite aside from other concerns, which have been argued outside this thread -- would encourage an increased proportion of Arch packages not be buildable by Nix. On Wed, Jan 22, 2020 at 8:19 PM Allan McRae <allan@archlinux.org> wrote:
Checksums arrays should be filled with values provided by upstream. We currently have md5 set as an unsecure default, and are constantly asked to change it to sha2. However, just changing the default to a stronger checksum gives the user the impression that "makepkg -g" checksums are perfect.
Instead, change the default checksum to a CRC, to make it clear that any checksum generated purely by "makepkg -g" is not ideal.
Signed-off-by: Allan McRae <allan@archlinux.org> --- doc/PKGBUILD.5.asciidoc | 14 +++++++------- doc/makepkg.conf.5.asciidoc | 2 +- etc/makepkg.conf.in | 4 ++-- scripts/libmakepkg/util/schema.sh.in | 2 +- scripts/makepkg.sh.in | 4 ++-- 5 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/doc/PKGBUILD.5.asciidoc b/doc/PKGBUILD.5.asciidoc index abe2ab52..4d03f8ed 100644 --- a/doc/PKGBUILD.5.asciidoc +++ b/doc/PKGBUILD.5.asciidoc @@ -118,7 +118,7 @@ systems (see below). + Additional architecture-specific sources can be added by appending an underscore and the architecture name e.g., 'source_x86_64=()'. There must be a -corresponding integrity array with checksums, e.g. 'md5sums_x86_64=()'. +corresponding integrity array with checksums, e.g. 'cksums_x86_64=()'. + It is also possible to change the name of the downloaded file, which is helpful with weird URLs and for handling multiple source files with the same @@ -146,19 +146,19 @@ contain whitespace characters. listed here will not be extracted with the rest of the source files. This is useful for packages that use compressed data directly.
-*md5sums (array)*:: - This array contains an MD5 hash for every source file specified in the +*cksums (array)*:: + This array contains CRC checksums for every source file specified in the source array (in the same order). makepkg will use this to verify source file integrity during subsequent builds. If 'SKIP' is put in the array in place of a normal hash, the integrity check for that source file will - be skipped. To easily generate md5sums, run ``makepkg -g >> PKGBUILD''. - If desired, move the md5sums line to an appropriate location. Note that + be skipped. To easily generate cksums, run ``makepkg -g >> PKGBUILD''. + If desired, move the cksums line to an appropriate location. Note that checksums generated by "makepkg -g" provide little security benefit. All checksum values should be as provided by the software developer.
-*sha1sums, sha224sums, sha256sums, sha384sums, sha512sums, b2sums (arrays)*:: +*md5sums, sha1sums, sha224sums, sha256sums, sha384sums, sha512sums, b2sums (arrays)*:: Alternative integrity checks that makepkg supports; these all behave - similar to the md5sums option described above. To enable use and generation + similar to the cksums option described above. To enable use and generation of these checksums, be sure to set up the `INTEGRITY_CHECK` option in linkman:makepkg.conf[5].
diff --git a/doc/makepkg.conf.5.asciidoc b/doc/makepkg.conf.5.asciidoc index b7496324..04cc5ea9 100644 --- a/doc/makepkg.conf.5.asciidoc +++ b/doc/makepkg.conf.5.asciidoc @@ -192,7 +192,7 @@ Options **INTEGRITY_CHECK=(**check1 ...**)**:: File integrity checks to use. Multiple checks may be specified; this affects both generation and checking. The current valid options are: - `md5`, `sha1`, `sha224`, `sha256`, `sha384`, `sha512`, and `b2`. + `ck`, `md5`, `sha1`, `sha224`, `sha256`, `sha384`, `sha512`, and `b2`.
**STRIP_BINARIES=**"--strip-all":: Options to be used when stripping binaries. See linkman:strip[1] diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in index caf5114b..1c7988d2 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -89,8 +89,8 @@ BUILDENV=(!distcc color !ccache check !sign) # OPTIONS=(strip docs libtool staticlibs emptydirs zipman purge !debug)
-#-- File integrity checks to use. Valid: md5, sha1, sha224, sha256, sha384, sha512, b2 -INTEGRITY_CHECK=(md5) +#-- File integrity checks to use. Valid: ck, md5, sha1, sha224, sha256, sha384, sha512, b2 +INTEGRITY_CHECK=(ck) #-- Options to be used when stripping binaries. See `man strip' for details. STRIP_BINARIES="@STRIP_BINARIES@" #-- Options to be used when stripping shared libraries. See `man strip' for details. diff --git a/scripts/libmakepkg/util/schema.sh.in b/scripts/libmakepkg/util/schema.sh.in index b2f119cf..02bfdb86 100644 --- a/scripts/libmakepkg/util/schema.sh.in +++ b/scripts/libmakepkg/util/schema.sh.in @@ -26,7 +26,7 @@ LIBRARY=${LIBRARY:-'@libmakepkgdir@'} source "$LIBRARY/util/util.sh"
-known_hash_algos=({md5,sha{1,224,256,384,512},b2}) +known_hash_algos=({ck,md5,sha{1,224,256,384,512},b2})
pkgbuild_schema_arrays=(arch backup checkdepends conflicts depends groups license makedepends noextract optdepends options diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index a6de7823..7fa791e1 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1195,8 +1195,8 @@ unset "${known_hash_algos[@]/%/sums}" unset -f pkgver prepare build check package "${!package_@}" unset "${!makedepends_@}" "${!depends_@}" "${!source_@}" "${!checkdepends_@}" unset "${!optdepends_@}" "${!conflicts_@}" "${!provides_@}" "${!replaces_@}" -unset "${!md5sums_@}" "${!sha1sums_@}" "${!sha224sums_@}" "${!sha256sums_@}" -unset "${!sha384sums_@}" "${!sha512sums_@}" "${!b2sums_@}" +unset "${!cksums_@}" "${!md5sums_@}" "${!sha1sums_@}" "${!sha224sums_@}" +unset "${!sha256sums_@}" "${!sha384sums_@}" "${!sha512sums_@}" "${!b2sums_@}"
BUILDFILE=${BUILDFILE:-$BUILDSCRIPT} if [[ ! -f $BUILDFILE ]]; then -- 2.25.0
On 24/1/20 11:06 am, Charles Duffy wrote:
Thus, moving to cksum -- quite aside from other concerns, which have been argued outside this thread -- would encourage an increased proportion of Arch packages not be buildable by Nix.
If an Arch packager relies on a CRC in their PKGBUILD, then we have bigger issues than these packages not building on Nix. A
On 1/23/20 8:06 PM, Charles Duffy wrote:
A potentially unforeseen consequence:
At present, it is possible (albeit with use of tools that aren't as of present date publicly released, something I hope to change in the future) to use the Nix build system to build Arch packages (with some caveats, but generally manageable ones for folks who don't need these packages to be what Nix calls "pure").
Nix identifies downloaded content by hash -- only build-time processes which can state a cryptographically strong checksum of their intended output prior to time of invocation are allowed to connect to the internet during the build process itself. cksum is not supported by Nix, whereas the other checksums supported by Arch are.
Thus, moving to cksum -- quite aside from other concerns, which have been argued outside this thread -- would encourage an increased proportion of Arch packages not be buildable by Nix. Wait... does that mean Nix considers md5 to be "cryptographically strong"? o_O
-- Eli Schwartz Bug Wrangler and Trusted User
On 1/22/20 9:18 PM, Allan McRae wrote:
Checksums arrays should be filled with values provided by upstream. We currently have md5 set as an unsecure default, and are constantly asked to change it to sha2. However, just changing the default to a stronger checksum gives the user the impression that "makepkg -g" checksums are perfect.
Instead, change the default checksum to a CRC, to make it clear that any checksum generated purely by "makepkg -g" is not ideal.
One reason it is not ideal is due to the fact that in my testing, "time cksum some-large-file" compared to "time md5sum some-large-file" took nearly twice as long. In fact, md5sum, sha1sum and b2sum all took roughly the same time to hash /var/cache/makepkg/srcdest/firefox-72.0.2.source.tar.xz (302MB). I mean, granted we're talking a wall clock time of: 0:00.49 for sha1 0:00.54 for md5 0:00.56 for b2 0:00.92 for ck So these differences don't significantly impact the time spent (regardless of which algorithm you use). On the other hand, it feels silly to move to a slower algorithm. (I would also like to point out for the record I am part of the group of people who would prefer Trust On First Use, but I understand this is not going to be discussed here anymore.) -- Eli Schwartz Bug Wrangler and Trusted User
On 24/1/20 12:37 pm, Eli Schwartz wrote:
On 1/22/20 9:18 PM, Allan McRae wrote:
Checksums arrays should be filled with values provided by upstream. We currently have md5 set as an unsecure default, and are constantly asked to change it to sha2. However, just changing the default to a stronger checksum gives the user the impression that "makepkg -g" checksums are perfect.
Instead, change the default checksum to a CRC, to make it clear that any checksum generated purely by "makepkg -g" is not ideal.
One reason it is not ideal is due to the fact that in my testing, "time cksum some-large-file" compared to "time md5sum some-large-file" took nearly twice as long. In fact, md5sum, sha1sum and b2sum all took roughly the same time to hash /var/cache/makepkg/srcdest/firefox-72.0.2.source.tar.xz (302MB).
I mean, granted we're talking a wall clock time of:
0:00.49 for sha1 0:00.54 for md5 0:00.56 for b2 0:00.92 for ck
So these differences don't significantly impact the time spent (regardless of which algorithm you use).
On the other hand, it feels silly to move to a slower algorithm.
Well... we hope that no-one will ever use this algorithm!
(I would also like to point out for the record I am part of the group of people who would prefer Trust On First Use, but I understand this is not going to be discussed here anymore.)
There is nothing stopping anyone adding sha512sums=() to their PKGBUILD. Running "makepkg -g" only pipes out the default when nothing else is in the PKGBUILD.
Em janeiro 24, 2020 1:56 Allan McRae escreveu:
Well... we hope that no-one will ever use this algorithm!
Then why waste everyone's time going through this process of adding it? Regards, Giancarlo Razzolini
On 25/1/20 12:19 am, Giancarlo Razzolini wrote:
Em janeiro 24, 2020 1:56 Allan McRae escreveu:
Well... we hope that no-one will ever use this algorithm!
Then why waste everyone's time going through this process of adding it?
Because I consider people to be delusional who use "makepkg -g" and think their checksums are fine. I find trust on first use ignorant in terms of software downloads. That is reflected in my decision to include CRC and add it as the default, so PKGBUILDs from people who add checksums without thinking will be visible. Even then, the CRC checksum will only be used by "makepkg -g" if there is no other checksums in a PKGBUILD. So only a portion of the blind trust will be visible. Allan
participants (4)
-
Allan McRae
-
Charles Duffy
-
Eli Schwartz
-
Giancarlo Razzolini