[arch-dev-public] [dbscripts] [PATCH] Add signature files to update/move/remove targets

older
[arch-dev-public] [namcap] namcap...

Rémy Oudompheng

31 Mar 2011 31 Mar '11

10:54 p.m.

Signature files are optional and the previous behaviour is unchanged when signatures files do not exist. Signed-off-by: Rémy Oudompheng --- This patch was already posted with very slight differences on the mailing-list by Allan. It needed several changes to fit the current state of dbscripts. I hope this one will allow to move forward. Still nothing in the test suite: we would need to run gpg after the extra-*-build invocation and extra checks for the presence/absence of signature files in the repos. cron-jobs/ftpdir-cleanup | 10 ++++++++-- db-move | 3 +++ db-update | 7 +++++++ 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/cron-jobs/ftpdir-cleanup b/cron-jobs/ftpdir-cleanup index 09e8a49..bb1661a 100755 --- a/cron-jobs/ftpdir-cleanup +++ b/cron-jobs/ftpdir-cleanup @@ -10,9 +10,12 @@ clean_pkg() { if ! ${CLEANUP_DRYRUN}; then for pkg in "$@"; do if [ -h "$pkg" ]; then - rm -f "$pkg" + rm -f "$pkg" "$pkg.sig" else mv -f "$pkg" "$CLEANUP_DESTDIR" + if [ -e "$pkg.sig" ]; then + mv -f "$pkg.sig" "$CLEANUP_DESTDIR" + fi touch "${CLEANUP_DESTDIR}/$(basename ${pkg})" fi done @@ -100,7 +103,10 @@ if [ ${#old_pkgs[@]} -ge 1 ]; then msg "Removing old packages from the cleanup directory..." for old_pkg in ${old_pkgs[@]}; do msg2 "${old_pkg}" - ${CLEANUP_DRYRUN} || rm -f "${CLEANUP_DESTDIR}/${old_pkg}" + if ! ${CLEANUP_DRYRUN}; then + rm -f "${CLEANUP_DESTDIR}/${old_pkg}" + rm -f "${CLEANUP_DESTDIR}/${old_pkg}.sig" + fi done fi diff --git a/db-move b/db-move index ced90a3..ff1b0e2 100755 --- a/db-move +++ b/db-move @@ -96,6 +96,9 @@ for pkgbase in ${args[@]:2}; do cp ${pkgpath} ${FTP_BASE}/${PKGPOOL} fi ln -s "../../../${PKGPOOL}/${pkgfile}" ${ftppath_to}/${tarch}/ + if [ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile}.sig ]; then + ln -s "../../../${PKGPOOL}/${pkgfile}.sig" ${ftppath_to}/${tarch}/ + fi add_pkgs[${tarch}]+="${FTP_BASE}/${PKGPOOL}/${pkgfile} " remove_pkgs[${tarch}]+="${pkgname} " done diff --git a/db-update b/db-update index 5bdce5e..4b9c78f 100755 --- a/db-update +++ b/db-update @@ -64,6 +64,13 @@ for repo in ${repos[@]}; do mv "${pkg}" "$FTP_BASE/${PKGPOOL}" fi ln -s "../../../${PKGPOOL}/${pkgfile}" "$FTP_BASE/$repo/os/${pkgarch}" + # also move signatures + if [ -f "${pkg}.sig" ]; then + mv "${pkg}.sig" "$FTP_BASE/${PKGPOOL}" + fi + if [ -f "$FTP_BASE/${PKGPOOL}/${pkgfile}.sig" ]; then + ln -s "../../../${PKGPOOL}/${pkgfile}.sig" "$FTP_BASE/$repo/os/${pkgarch}" + fi add_pkgs[${#add_pkgs[*]}]=${pkgfile} done if [ ${#add_pkgs[@]} -ge 1 ]; then -- 1.7.4.2

Show replies by date

Pierre Schmitz

1 Apr 1 Apr

7:44 p.m.

New subject: [arch-dev-public] [dbscripts] [PATCH] Add signature files to update/move/remove targets

On Fri, 1 Apr 2011 00:54:57 +0200, Rémy Oudompheng wrote:

...

Signature files are optional and the previous behaviour is unchanged when signatures files do not exist.

Signed-off-by: Rémy Oudompheng --- This patch was already posted with very slight differences on the mailing-list by Allan. It needed several changes to fit the current state of dbscripts. I hope this one will allow to move forward.

Still nothing in the test suite: we would need to run gpg after the extra-*-build invocation and extra checks for the presence/absence of signature files in the repos.

I am a little confused by this patch. If I get the current repo-add code right, gpg signatures will be base64 encoded and added to the db files. So there should be no need to provide .sig files for every single package. -- Pierre Schmitz, https://users.archlinux.de/~pierre

Rémy Oudompheng

7:54 p.m.

New subject: [arch-dev-public] [dbscripts] [PATCH] Add signature files to update/move/remove targets

On 2011/4/1 Pierre Schmitz wrote:

...

On Fri, 1 Apr 2011 00:54:57 +0200, Rémy Oudompheng wrote:

...
Signature files are optional and the previous behaviour is unchanged when signatures files do not exist.

Signed-off-by: Rémy Oudompheng --- This patch was already posted with very slight differences on the mailing-list by Allan. It needed several changes to fit the current state of dbscripts. I hope this one will allow to move forward.

Still nothing in the test suite: we would need to run gpg after the extra-*-build invocation and extra checks for the presence/absence of signature files in the repos.

I am a little confused by this patch. If I get the current repo-add code right, gpg signatures will be base64 encoded and added to the db files. So there should be no need to provide .sig files for every single package.

Then I'm also confused by our current handling of signatures. Do we have decided anything about that? Why did Allan suggest this patch? How can repo-add put signatures in db files if signatures are not available in the package pool? (we probably don't want to extract the signature and copy it when moving packages between repos) In my current understanding: * package pool holds packages and their signature files, and serves as the basis for generating databases * repo directories ($repo/os/$arch) contain symlinks to packages, databases which are generated by repo-add, and the signature file for the database. -- Rémy.

Pierre Schmitz

7:58 p.m.

New subject: [arch-dev-public] [dbscripts] [PATCH] Add signature files to update/move/remove targets

On Fri, 1 Apr 2011 21:54:30 +0200, Rémy Oudompheng wrote:

...

On 2011/4/1 Pierre Schmitz wrote:

...
On Fri, 1 Apr 2011 00:54:57 +0200, Rémy Oudompheng wrote:

...
Signature files are optional and the previous behaviour is unchanged when signatures files do not exist.

Signed-off-by: Rémy Oudompheng --- This patch was already posted with very slight differences on the mailing-list by Allan. It needed several changes to fit the current state of dbscripts. I hope this one will allow to move forward.

Still nothing in the test suite: we would need to run gpg after the extra-*-build invocation and extra checks for the presence/absence of signature files in the repos.

I am a little confused by this patch. If I get the current repo-add code right, gpg signatures will be base64 encoded and added to the db files. So there should be no need to provide .sig files for every single package.

Then I'm also confused by our current handling of signatures. Do we have decided anything about that? Why did Allan suggest this patch? How can repo-add put signatures in db files if signatures are not available in the package pool? (we probably don't want to extract the signature and copy it when moving packages between repos)

I think it will work this way: * you upload the package and its separate signature into your staging dir * repo-add will add the pacakge's meta data and signature into the database file

...

In my current understanding: * package pool holds packages and their signature files, and serves as the basis for generating databases * repo directories ($repo/os/$arch) contain symlinks to packages, databases which are generated by repo-add, and the signature file for the database.

The package's signatures are kept within the db file. The only separate .sig file that will be visible in the repos is the one for the db file itself. -- Pierre Schmitz, https://users.archlinux.de/~pierre

Rémy Oudompheng

8:05 p.m.

New subject: [arch-dev-public] [dbscripts] [PATCH] Add signature files to update/move/remove targets

On 2011/4/1 Pierre Schmitz wrote:

...

On Fri, 1 Apr 2011 21:54:30 +0200, Rémy Oudompheng wrote:

...
Then I'm also confused by our current handling of signatures. Do we have decided anything about that? Why did Allan suggest this patch? How can repo-add put signatures in db files if signatures are not available in the package pool? (we probably don't want to extract the signature and copy it when moving packages between repos)

I think it will work this way: * you upload the package and its separate signature into your staging dir * repo-add will add the pacakge's meta data and signature into the database file

In the current setting (arch_repo_add) in db-functions, repo-add is run from the repo directory ($repo/os/$arch) and would not have access to the signature files. A progress would be to run it from the package pool, so that we don't have to link signature files to repo folders. But not having the signature files in the package pool would make it problematic to use db-move correctly, since signatures would have been deleted from $HOME/staging directories in the meanwhile. -- Rémy.

Dan McGee

8:10 p.m.

New subject: [arch-dev-public] [dbscripts] [PATCH] Add signature files to update/move/remove targets

On Fri, Apr 1, 2011 at 2:58 PM, Pierre Schmitz wrote:

...

On Fri, 1 Apr 2011 21:54:30 +0200, Rémy Oudompheng wrote:

...
On 2011/4/1 Pierre Schmitz wrote:

...
On Fri, 1 Apr 2011 00:54:57 +0200, Rémy Oudompheng wrote:

...
Signature files are optional and the previous behaviour is unchanged when signatures files do not exist.

Signed-off-by: Rémy Oudompheng --- This patch was already posted with very slight differences on the mailing-list by Allan. It needed several changes to fit the current state of dbscripts. I hope this one will allow to move forward.

Still nothing in the test suite: we would need to run gpg after the extra-*-build invocation and extra checks for the presence/absence of signature files in the repos.

I am a little confused by this patch. If I get the current repo-add code right, gpg signatures will be base64 encoded and added to the db files. So there should be no need to provide .sig files for every single package.

Then I'm also confused by our current handling of signatures. Do we have decided anything about that? Why did Allan suggest this patch? How can repo-add put signatures in db files if signatures are not available in the package pool? (we probably don't want to extract the signature and copy it when moving packages between repos)

I think it will work this way: * you upload the package and its separate signature into your staging dir * repo-add will add the pacakge's meta data and signature into the database file

...
In my current understanding: * package pool holds packages and their signature files, and serves as the basis for generating databases * repo directories ($repo/os/$arch) contain symlinks to packages, databases which are generated by repo-add, and the signature file for the database.

The package's signatures are kept within the db file. The only separate .sig file that will be visible in the repos is the one for the db file itself.

No, that is not the intention. We put them in the database as well so you do not have to download each and every .sig file individually, but they have always been intended to be freely available and sitting there as well. It would be quite silly to hide these files away if we have them. For that matter, repo-add doesn't add them *unless* they are sitting next to the package. -Dan

Pierre Schmitz

8:45 p.m.

New subject: [arch-dev-public] [dbscripts] [PATCH] Add signature files to update/move/remove targets

On Fri, 1 Apr 2011 15:10:36 -0500, Dan McGee wrote:

...

On Fri, Apr 1, 2011 at 2:58 PM, Pierre Schmitz wrote:

...
On Fri, 1 Apr 2011 21:54:30 +0200, Rémy Oudompheng wrote:

...
In my current understanding: * package pool holds packages and their signature files, and serves as the basis for generating databases * repo directories ($repo/os/$arch) contain symlinks to packages, databases which are generated by repo-add, and the signature file for the database.

The package's signatures are kept within the db file. The only separate .sig file that will be visible in the repos is the one for the db file itself.

No, that is not the intention. We put them in the database as well so you do not have to download each and every .sig file individually, but they have always been intended to be freely available and sitting there as well. It would be quite silly to hide these files away if we have them.

For that matter, repo-add doesn't add them *unless* they are sitting next to the package.

Thanks for clarifying. Back to the patch: I'll need to have more check for the .sig files and at least a simple test case will be needed. -- Pierre Schmitz, https://users.archlinux.de/~pierre

Allan McRae

9:52 p.m.

New subject: [arch-dev-public] [dbscripts] [PATCH] Add signature files to update/move/remove targets

On 02/04/11 06:45, Pierre Schmitz wrote:

...

On Fri, 1 Apr 2011 15:10:36 -0500, Dan McGee wrote:

...
On Fri, Apr 1, 2011 at 2:58 PM, Pierre Schmitz wrote:

...
On Fri, 1 Apr 2011 21:54:30 +0200, Rémy Oudompheng wrote:

...
In my current understanding: * package pool holds packages and their signature files, and serves as the basis for generating databases * repo directories ($repo/os/$arch) contain symlinks to packages, databases which are generated by repo-add, and the signature file for the database.

The package's signatures are kept within the db file. The only separate .sig file that will be visible in the repos is the one for the db file itself.

No, that is not the intention. We put them in the database as well so you do not have to download each and every .sig file individually, but they have always been intended to be freely available and sitting there as well. It would be quite silly to hide these files away if we have them.

For that matter, repo-add doesn't add them *unless* they are sitting next to the package.

Thanks for clarifying.

Back to the patch: I'll need to have more check for the .sig files and at least a simple test case will be needed.

Just to add another reason to keep the .sig file beside the package, pacman -U http://package/from/mirror.pkg.tar.gz will try and download the signature file and verify it too. Allan

Pierre Schmitz

2 Apr 2 Apr

11:24 a.m.

New subject: [arch-dev-public] [dbscripts] [PATCH] Add signature files to update/move/remove targets

On Fri, 1 Apr 2011 00:54:57 +0200, Rémy Oudompheng wrote:

...

Signature files are optional and the previous behaviour is unchanged when signatures files do not exist.

Signed-off-by: Rémy Oudompheng --- This patch was already posted with very slight differences on the mailing-list by Allan. It needed several changes to fit the current state of dbscripts. I hope this one will allow to move forward.

Still nothing in the test suite: we would need to run gpg after the extra-*-build invocation and extra checks for the presence/absence of signature files in the repos.

I have added a simple test and pushed this patch. There is also a new config var called REQUIRE_SIGNATURE which wont accept new packages if a .sig file is not available. See http://projects.archlinux.org/dbscripts.git/commit/?id=1ce0c6368d0908e25f9bd... -- Pierre Schmitz, https://users.archlinux.de/~pierre

4773

Age (days ago)

4775

Last active (days ago)

List overview

Download

8 comments

4 participants

participants (4)

Allan McRae
Dan McGee
Pierre Schmitz
Rémy Oudompheng

[arch-dev-public] [dbscripts] [PATCH] Add signature files to update/move/remove targets

tags

participants (4)