[arch-dev-public] [RFC] Remove svn propset id's
Most of our PKGBUILDs svn propset's break reproducible builds and the pkgbuild_sha256sum in the BUILDINFO file. When building a package before commiting the PKGBUILD the propset $Id will differ since the $Id is set on commit. This has a few implications, pkgbuild_sha256sum is useless and we can't reproduce packages due to the BUILDINFO not matching. Also the reproduce tool uses ASP to retrieve the PKGBUILD and therefore can't verify that it got the correct PKGBUILD (it relies on pkgbuild_sha256sum). To resolve this issue we could simply remove the propset id's, since for me, although not sure about others they don't seem particulary useful. The proof that the sha256sums's don't match: $ extra-x86_64-build $ grep sha256 .BUILDINFO pkgbuild_sha256sum = 8748d60d2c782f477cb7e692a3dad30be90491cdc13fe8951340da4c0bc7f19e $ $repopkg $ sha256sum PKGBUILD d8ab51a983026dd4a6e2f48e9dc66177eca8cf6c1c0ffefb950b093db299e304 PKGBUILD # The git checkout [jelle@helium][/tmp/bar/community/python-psutil/trunk]%sha256sum PKGBUILD ce7f1e68a3b426412a24f46016817d30721860c8ef6b3d0a2dddac8ff2448b84 PKGBUILD [jelle@helium][/tmp/bar/community/python-psutil/trunk]%diff PKGBUILD /tmp/python-psutil/trunk/PKGBUILD 1c1 < # $Id$ ---
# $Id: PKGBUILD 375007 2018-08-28 17:24:26Z jelle $
-- Jelle van der Waa
On 8/29/18 10:23 PM, Jelle van der Waa wrote:
Most of our PKGBUILDs svn propset's break reproducible builds and the pkgbuild_sha256sum in the BUILDINFO file. When building a package before commiting the PKGBUILD the propset $Id will differ since the $Id is set on commit.
This has a few implications, pkgbuild_sha256sum is useless and we can't reproduce packages due to the BUILDINFO not matching. Also the reproduce tool uses ASP to retrieve the PKGBUILD and therefore can't verify that it got the correct PKGBUILD (it relies on pkgbuild_sha256sum).
To resolve this issue we could simply remove the propset id's, since for me, although not sure about others they don't seem particulary useful.
The proof that the sha256sums's don't match:
$ extra-x86_64-build $ grep sha256 .BUILDINFO pkgbuild_sha256sum = 8748d60d2c782f477cb7e692a3dad30be90491cdc13fe8951340da4c0bc7f19e $ $repopkg
$ sha256sum PKGBUILD d8ab51a983026dd4a6e2f48e9dc66177eca8cf6c1c0ffefb950b093db299e304 PKGBUILD
# The git checkout
[jelle@helium][/tmp/bar/community/python-psutil/trunk]%sha256sum PKGBUILD ce7f1e68a3b426412a24f46016817d30721860c8ef6b3d0a2dddac8ff2448b84 PKGBUILD
[jelle@helium][/tmp/bar/community/python-psutil/trunk]%diff PKGBUILD /tmp/python-psutil/trunk/PKGBUILD 1c1 < # $Id$ ---
# $Id: PKGBUILD 375007 2018-08-28 17:24:26Z jelle $
I know there are some people who like them because $reason, but even with svn its not rocket science to get the last author. +1 from me because on top of your reason: - IMO such meta data belongs to the repo history and not the file content itself. - we will purge it anyway if we finally finish the transition to git cheers, Levente
On 8/29/18 4:23 PM, Jelle van der Waa wrote:
Most of our PKGBUILDs svn propset's break reproducible builds and the pkgbuild_sha256sum in the BUILDINFO file. When building a package before commiting the PKGBUILD the propset $Id will differ since the $Id is set on commit.
This has a few implications, pkgbuild_sha256sum is useless and we can't reproduce packages due to the BUILDINFO not matching. Also the reproduce tool uses ASP to retrieve the PKGBUILD and therefore can't verify that it got the correct PKGBUILD (it relies on pkgbuild_sha256sum).
To resolve this issue we could simply remove the propset id's, since for me, although not sure about others they don't seem particulary useful.
I've never been entirely clear on their motivating purpose, in fact. Also to expand on the general issue for people who aren't in #archlinux-reproducible: When you run extra-x86_64-build, you're using the PKGBUILD you're about to commit, which svn will set to the expanded propset of the previous commit... which matches no file ever seen by svn. If you svn commit, and *then* extra-x86_64-build, then svn will actually have the right file. What's the likelihood of people making sure to svn commit before making sure the package actually builds as expected... IIRC at least some packages seem to have been built by the svntogit exported PKGBUILD (e.g. via asp) since their pkgbuild_sha256sum can be obtained from asp. This results in far too many ways to maybe get the actual file used to build, and in the most likely scenario it requires deep forensics of the svn repository. ... svn propsets will die either way whenever we finally manage to migrate away from svn and onto git. -- Eli Schwartz Bug Wrangler and Trusted User
On Wed, Aug 29, 2018 at 10:23:07PM +0200, Jelle van der Waa <jelle@vdwaa.nl> wrote:
To resolve this issue we could simply remove the propset id's, since for me, although not sure about others they don't seem particulary useful.
+1 for getting rid of them. Florian
On 2018-08-29 22:23:07 (+0200), Jelle van der Waa wrote:
This has a few implications, pkgbuild_sha256sum is useless and we can't reproduce packages due to the BUILDINFO not matching. Also the reproduce tool uses ASP to retrieve the PKGBUILD and therefore can't verify that it got the correct PKGBUILD (it relies on pkgbuild_sha256sum). </ insert kill-it-with-fire-meme >
On 08/29/18 at 10:23pm, Jelle van der Waa wrote:
Most of our PKGBUILDs svn propset's break reproducible builds and the pkgbuild_sha256sum in the BUILDINFO file. When building a package before commiting the PKGBUILD the propset $Id will differ since the $Id is set on commit.
So far, I've only gotten positive reactions. If no one objects I propose to remove the propsets treewide after a week, so everyone has time enough to object. Removal will be done as following: $ sed -ri '/\$Id/d' */trunk/PKGBUILD $ svn propdel svn:keywords */trunk/PKGBUILD -- Jelle van der Waa
On 09/04/18 at 08:54pm, Jelle van der Waa wrote:
On 08/29/18 at 10:23pm, Jelle van der Waa wrote:
Most of our PKGBUILDs svn propset's break reproducible builds and the pkgbuild_sha256sum in the BUILDINFO file. When building a package before commiting the PKGBUILD the propset $Id will differ since the $Id is set on commit.
So far, I've only gotten positive reactions. If no one objects I propose to remove the propsets treewide after a week, so everyone has time enough to object. Removal will be done as following:
$ sed -ri '/\$Id/d' */trunk/PKGBUILD $ svn propdel svn:keywords */trunk/PKGBUILD
The propsets have been removed from both community and packages. We can now continue the reproducible build effort knowing that the PKGBUILD hash in trunk and in BUILDINFO matches :) -- Jelle van der Waa
On 12.09.18 - 09:16, Jelle van der Waa wrote:
The propsets have been removed from both community and packages.
We can now continue the reproducible build effort knowing that the PKGBUILD hash in trunk and in BUILDINFO matches :)
-- Jelle van der Waa
Nice job! Could someone please update the info here as well: https://wiki.archlinux.org/index.php/DeveloperWiki:HOWTO_Be_A_Packager "Simple" TUs are lacking the permissions to edit the dev wiki, as most of you know ;) I propose removing the svn propset command from this sections: https://wiki.archlinux.org/index.php/DeveloperWiki:HOWTO_Be_A_Packager#Addin... and removing that $Id$ subsection here: https://wiki.archlinux.org/index.php/DeveloperWiki:HOWTO_Be_A_Packager#SVN_.... Cheers, Thore -- Thore Bödecker GPG ID: 0xD622431AF8DB80F3 GPG FP: 0F96 559D 3556 24FC 2226 A864 D622 431A F8DB 80F3
participants (6)
-
David Runge
-
Eli Schwartz
-
Florian Pritz
-
Jelle van der Waa
-
Levente Polyak
-
Thore Bödecker