[pacman-dev] [RFC][PATCH 0/3] Add source package information to repo database

newer
[pacman-dev] [PATCH] log important...

older
[pacman-dev] [PATCH] pacsort: only...

Allan McRae

2 Nov 2013 2 Nov '13

6:21 a.m.

AKA - the I hate ABS patchset... This patchset adds source package information to the sync dbs. Once this is given the all clear, it will be followed by patches reading the information into libalpm and adding a "pacman -B" that will download the source package and extract to /var/build/pacman (or somewhere...). This would be a complete ABS replacement on Arch. Queries: 1) I add a separate file to the db and reuse the variable names from the desc file. This means some probably duplication in the sync db reading code. Or should I add the info to the desc file and give it different names? 2) I add a -S/--source flag to add the source file description. This is similar to --delta and --files. I toyed with the idea of including the source information by default. Opinions on this? Allan McRae (3): makepkg: sign source packages with --sign Always supply base name and version info in .PKGFILE if needed repo-add: add source package information to sync db lib/libalpm/be_package.c | 2 ++ lib/libalpm/be_sync.c | 2 ++ scripts/makepkg.sh.in | 30 +++++++++++++------ scripts/repo-add.sh.in | 75 ++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 99 insertions(+), 10 deletions(-) -- 1.8.4.2

Show replies by date

Allan McRae

2 Nov 2 Nov

6:21 a.m.

New subject: [pacman-dev] [PATCH 1/3] makepkg: sign source packages with --sign

Signed-off-by: Allan McRae --- scripts/makepkg.sh.in | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 446e1de..b04c11b 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -2035,10 +2035,16 @@ create_srcpackage() { exit 1 # TODO: error code fi + create_signature "$pkg_file" + if [[ ! "$SRCPKGDEST" -ef "${startdir}" ]]; then rm -f "${pkg_file/$SRCPKGDEST/$startdir}" ln -s "${pkg_file}" "${pkg_file/$SRCPKGDEST/$startdir}" ret=$? + if [[ -f $pkg_file.sig ]]; then + rm -f "${pkg_file/$PKGDEST/$startdir}.sig" + ln -s "$pkg_file.sig" "${pkg_file/$PKGDEST/$startdir}.sig" + fi fi if (( ret )); then -- 1.8.4.2

Allan McRae

6:21 a.m.

New subject: [pacman-dev] [PATCH 2/3] Always supply base name and version info in .PKGFILE if needed

Provide pkgbase information for non-split packages with pkgbase set. Also record the version of the "base" package. This is useful for matching package files to source packages. Signed-off-by: Allan McRae --- lib/libalpm/be_package.c | 2 ++ scripts/makepkg.sh.in | 24 ++++++++++++++++-------- 2 files changed, 18 insertions(+), 8 deletions(-) diff --git a/lib/libalpm/be_package.c b/lib/libalpm/be_package.c index 4e814f8..c95b6b5 100644 --- a/lib/libalpm/be_package.c +++ b/lib/libalpm/be_package.c @@ -195,6 +195,8 @@ static int parse_descfile(alpm_handle_t *handle, struct archive *a, alpm_pkg_t * /* not used atm */ } else if(strcmp(key, "pkgver") == 0) { STRDUP(newpkg->version, ptr, return -1); + } else if(strcmp(key, "basever") == 0) { + /* not used atm */ } else if(strcmp(key, "pkgdesc") == 0) { STRDUP(newpkg->desc, ptr, return -1); } else if(strcmp(key, "group") == 0) { diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index b04c11b..891e7cf 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1806,9 +1806,18 @@ write_pkginfo() { printf "# Generated by makepkg %s\n" "$makepkg_version" (( INFAKEROOT )) && printf "# using %s\n" "$(fakeroot -v)" printf "# %s\n" "$(LC_ALL=C date -u)" + printf "pkgname = %s\n" "$pkgname" - (( SPLITPKG )) && printf "pkgbase = %s\n" "$pkgbase" - printf "pkgver = %s\n" "$(get_full_version)" + if (( SPLITPKG )) || [[ "$pkgbase" != "$pkgname" ]]; then + printf "pkgbase = %s\n" "$pkgbase" + fi + + local fullver=$(get_full_version) + printf "pkgver = %s\n" "$fullver" + if [[ "$fullver" != "$basever" ]]; then + printf "basever = %s\n" "$basever" + fi + printf "pkgdesc = %s\n" "${pkgdesc//+([[:space:]])/ }" printf "url = %s\n" "$url" printf "builddate = %s\n" "$builddate" @@ -1855,7 +1864,6 @@ create_package() { msg "$(gettext "Creating package \"%s\"...")" "$pkgname" pkgarch=$(get_pkg_arch) - write_pkginfo > .PKGINFO local comp_files=('.PKGINFO') @@ -2808,6 +2816,7 @@ fi # set defaults if they weren't specified in buildfile pkgbase=${pkgbase:-${pkgname[0]}} epoch=${epoch:-0} +basever=$(get_full_version) if [[ $BUILDDIR = "$startdir" ]]; then srcdir="$BUILDDIR/src" @@ -2914,12 +2923,11 @@ if (( INFAKEROOT )); then exit 0 # $E_OK fi -fullver=$(get_full_version) -msg "$(gettext "Making package: %s")" "$pkgbase $fullver ($(date))" +msg "$(gettext "Making package: %s")" "$pkgbase $basever ($(date))" # if we are creating a source-only package, go no further if (( SOURCEONLY )); then - if [[ -f $SRCPKGDEST/${pkgbase}-${fullver}${SRCEXT} ]] \ + if [[ -f $SRCPKGDEST/${pkgbase}-${basever}${SRCEXT} ]] \ && (( ! FORCE )); then error "$(gettext "A source package has already been built. (use %s to overwrite)")" "-f" exit 1 @@ -3007,6 +3015,7 @@ elif (( !REPKG )); then extract_sources if (( PKGVERFUNC )); then update_pkgver + basever=$(get_full_version) check_build_status fi if (( PREPAREFUNC )); then @@ -3061,8 +3070,7 @@ else fi fi -fullver=$(get_full_version) -msg "$(gettext "Finished making: %s")" "$pkgbase $fullver ($(date))" +msg "$(gettext "Finished making: %s")" "$pkgbase $basever ($(date))" install_package -- 1.8.4.2

Allan McRae

6:21 a.m.

New subject: [pacman-dev] [PATCH 3/3] repo-add: add source package information to sync db

When using the -S/--source option, add source package information to the sync db if one is found alongside the package file. TODO: Add documentation Signed-off-by: Allan McRae --- lib/libalpm/be_sync.c | 2 ++ scripts/repo-add.sh.in | 75 ++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 75 insertions(+), 2 deletions(-) diff --git a/lib/libalpm/be_sync.c b/lib/libalpm/be_sync.c index 6ebdf3c..eb60b64 100644 --- a/lib/libalpm/be_sync.c +++ b/lib/libalpm/be_sync.c @@ -674,6 +674,8 @@ static int sync_db_read(alpm_db_t *db, struct archive *archive, /* skip reading delta files if UseDelta is unset */ } else if(strcmp(filename, "files") == 0) { /* currently do nothing with this file */ + } else if(strcmp(filename, "source") == 0) { + /* currently do nothing with this file */ } else { /* unknown database file */ _alpm_log(db->handle, ALPM_LOG_DEBUG, "unknown database file: %s\n", filename); diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index dbb635a..e4a34d1 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -33,6 +33,7 @@ DELTA=0 ONLYADDNEW=0 RMEXISTING=0 WITHFILES=0 +SOURCE=0 SIGN=0 VERIFY=0 REPO_DB_FILE= @@ -61,6 +62,7 @@ Multiple packages to add can be specified on the command line.\n")" printf -- "$(gettext " -n, --new only add packages that are not already in the database\n")" printf -- "$(gettext " -R, --remove remove package file from disk when updating database entry\n")" printf -- "$(gettext " -f, --files update database's file list\n")" + printf -- "$(gettext " -S, --source add source package information to the database\n")" elif [[ $cmd == "repo-remove" ]] ; then printf -- "$(gettext "Usage: repo-remove [options] <path-to-db> ...\n")" printf -- "\n" @@ -124,6 +126,35 @@ find_pkgentry() { return 1 } +match_source_package() { + case $# in + 0) + warning "$(gettext "No source package found")" + return 1 + ;; + [!1]) + warning "$(gettext "Cannot determine source package - multiple matches found:")" + msg2 '%s' "$@" + return 1 + ;; + esac + + echo "$1" +} + +find_source_package() { + local pkgfile pkgname pkgver sourcepkg + + pkgfile=$1 + pkgname=$2 + pkgver=$3 + + local sourcepkg=$(shopt -s nullglob; + match_source_package "$(dirname "$pkgfile")/$pkgname-$pkgver".src.tar!(*.sig)) + + echo "$sourcepkg" +} + funcgrep() { awk -v funcmatch="$1" ' /^[[:space:]]*[[:alnum:]_]+[[:space:]]*$[[:space:]]*$/ { @@ -272,8 +303,9 @@ db_write_entry() { local pkgfile=$1 local -a _groups _licenses _replaces _depends _conflicts _provides \ _optdepends _makedepends _checkdepends - local pkgname pkgver pkgdesc csize size url arch builddate packager \ - md5sum sha256sum pgpsig pgpsigsize install_functions + local pkgname pkgbase pkgver basever pkgdesc csize size url arch builddate \ + packager md5sum sha256sum pgpsig pgpsigsize install_functions \ + spkgfile scsize smd5sum ssha256sum spgpsig # read info from the zipped package local line var val @@ -353,6 +385,31 @@ db_write_entry() { sha256sum=$(openssl dgst -sha256 "$pkgfile") sha256sum=${sha256sum##* } + # gather source package information + if (( SOURCE )); then + msg2 "$(gettext "Adding source package...")" + + spkgfile=$(find_source_package "$pkgfile" ${pkgbase:-$pkgname} ${basever:-$pkgver}) + + if [[ -n "$spkgfile" ]]; then + if [[ -f "$spkgfile.sig" ]]; then + pgpsigsize=$(@SIZECMD@ -L "$spkgfile.sig") + if (( pgpsigsize > 16384 )); then + error "$(gettext "Invalid package signature file '%s'.")" "$spkgfile.sig" + return 1 + fi + spgpsig=$(openssl base64 -in "$spkgfile.sig" | tr -d '\n') + fi + + scsize=$(@SIZECMD@ -L "$spkgfile") + + smd5sum=$(openssl dgst -md5 "$spkgfile") + smd5sum=${md5sum##* } + ssha256sum=$(openssl dgst -sha256 "$spkgfile") + ssha256sum=${sha256sum##* } + fi + fi + # remove an existing entry if it exists, ignore failures db_remove_entry "$pkgname" @@ -371,6 +428,7 @@ db_write_entry() { format_entry "NAME" "$pkgname" format_entry "BASE" "$pkgbase" format_entry "VERSION" "$pkgver" + format_entry "BASEVER" "$basever" format_entry "DESC" "$pkgdesc" format_entry "GROUPS" "${_groups[@]}" format_entry "CSIZE" "$csize" @@ -403,6 +461,18 @@ db_write_entry() { format_entry "CHECKDEPENDS" "${_checkdepends[@]}" } >'depends' + # create source entry + if (( SOURCE )) && [[ -n "$spkgfile" ]]; then + msg2 "$(gettext "Creating '%s' db entry...")" 'source' + { + format_entry "FILENAME" "${spkgfile##*/}" + format_entry "CSIZE" "$scsize" + format_entry "MD5SUM" "$smd5sum" + format_entry "SHA256SUM" "$ssha256sum" + format_entry "PGPSIG" "$spgpsig" + } >'source' + fi + popd >/dev/null popd >/dev/null @@ -640,6 +710,7 @@ while (( $# )); do -n|--new) ONLYADDNEW=1;; -R|--remove) RMEXISTING=1;; -f|--files) WITHFILES=1;; + -S|--source) SOURCE=1;; --nocolor) USE_COLOR='n';; -s|--sign) check_gpg -- 1.8.4.2

Dave Reisner

3 Nov 3 Nov

4:38 a.m.

New subject: [pacman-dev] [PATCH 3/3] repo-add: add source package information to sync db

On Sat, Nov 02, 2013 at 12:21:42PM +1000, Allan McRae wrote:

...

When using the -S/--source option, add source package information to the sync db if one is found alongside the package file.

TODO: Add documentation

Signed-off-by: Allan McRae --- lib/libalpm/be_sync.c | 2 ++ scripts/repo-add.sh.in | 75 ++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 75 insertions(+), 2 deletions(-)

diff --git a/lib/libalpm/be_sync.c b/lib/libalpm/be_sync.c index 6ebdf3c..eb60b64 100644 --- a/lib/libalpm/be_sync.c +++ b/lib/libalpm/be_sync.c @@ -674,6 +674,8 @@ static int sync_db_read(alpm_db_t *db, struct archive *archive, /* skip reading delta files if UseDelta is unset */ } else if(strcmp(filename, "files") == 0) { /* currently do nothing with this file */ + } else if(strcmp(filename, "source") == 0) { + /* currently do nothing with this file */ } else { /* unknown database file */ _alpm_log(db->handle, ALPM_LOG_DEBUG, "unknown database file: %s\n", filename); diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index dbb635a..e4a34d1 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -33,6 +33,7 @@ DELTA=0 ONLYADDNEW=0 RMEXISTING=0 WITHFILES=0 +SOURCE=0 SIGN=0 VERIFY=0 REPO_DB_FILE= @@ -61,6 +62,7 @@ Multiple packages to add can be specified on the command line.\n")" printf -- "$(gettext " -n, --new only add packages that are not already in the database\n")" printf -- "$(gettext " -R, --remove remove package file from disk when updating database entry\n")" printf -- "$(gettext " -f, --files update database's file list\n")" + printf -- "$(gettext " -S, --source add source package information to the database\n")" elif [[ $cmd == "repo-remove" ]] ; then printf -- "$(gettext "Usage: repo-remove [options] <path-to-db> ...\n")" printf -- "\n" @@ -124,6 +126,35 @@ find_pkgentry() { return 1 }

+match_source_package() { + case $# in + 0) + warning "$(gettext "No source package found")" + return 1 + ;;

I think if you check for existence here you make this a more robust function which doesn't silently require nullglob. This case label then simply becomes 0|1.

...

+ [!1]) + warning "$(gettext "Cannot determine source package - multiple matches found:")" + msg2 '%s' "$@" + return 1 + ;; + esac + echo "$1" +} + +find_source_package() { + local pkgfile pkgname pkgver sourcepkg + + pkgfile=$1 + pkgname=$2 + pkgver=$3 + + local sourcepkg=$(shopt -s nullglob; + match_source_package "$(dirname "$pkgfile")/$pkgname-$pkgver".src.tar!(*.sig)) + + echo "$sourcepkg"

You probably don't want to do this if match_source_package failed...

...

+} + funcgrep() { awk -v funcmatch="$1" ' /^[[:space:]]*[[:alnum:]_]+[[:space:]]*$[[:space:]]*$/ { @@ -272,8 +303,9 @@ db_write_entry() { local pkgfile=$1 local -a _groups _licenses _replaces _depends _conflicts _provides \ _optdepends _makedepends _checkdepends - local pkgname pkgver pkgdesc csize size url arch builddate packager \ - md5sum sha256sum pgpsig pgpsigsize install_functions + local pkgname pkgbase pkgver basever pkgdesc csize size url arch builddate \ + packager md5sum sha256sum pgpsig pgpsigsize install_functions \ + spkgfile scsize smd5sum ssha256sum spgpsig

# read info from the zipped package local line var val @@ -353,6 +385,31 @@ db_write_entry() { sha256sum=$(openssl dgst -sha256 "$pkgfile") sha256sum=${sha256sum##* }

+ # gather source package information + if (( SOURCE )); then + msg2 "$(gettext "Adding source package...")" + + spkgfile=$(find_source_package "$pkgfile" ${pkgbase:-$pkgname} ${basever:-$pkgver}) + + if [[ -n "$spkgfile" ]]; then + if [[ -f "$spkgfile.sig" ]]; then + pgpsigsize=$(@SIZECMD@ -L "$spkgfile.sig") + if (( pgpsigsize > 16384 )); then + error "$(gettext "Invalid package signature file '%s'.")" "$spkgfile.sig" + return 1 + fi + spgpsig=$(openssl base64 -in "$spkgfile.sig" | tr -d '\n') + fi + + scsize=$(@SIZECMD@ -L "$spkgfile") + + smd5sum=$(openssl dgst -md5 "$spkgfile") + smd5sum=${md5sum##* } + ssha256sum=$(openssl dgst -sha256 "$spkgfile") + ssha256sum=${sha256sum##* } + fi + fi + # remove an existing entry if it exists, ignore failures db_remove_entry "$pkgname"

@@ -371,6 +428,7 @@ db_write_entry() { format_entry "NAME" "$pkgname" format_entry "BASE" "$pkgbase" format_entry "VERSION" "$pkgver" + format_entry "BASEVER" "$basever" format_entry "DESC" "$pkgdesc" format_entry "GROUPS" "${_groups[@]}" format_entry "CSIZE" "$csize" @@ -403,6 +461,18 @@ db_write_entry() { format_entry "CHECKDEPENDS" "${_checkdepends[@]}" } >'depends'

+ # create source entry + if (( SOURCE )) && [[ -n "$spkgfile" ]]; then + msg2 "$(gettext "Creating '%s' db entry...")" 'source' + { + format_entry "FILENAME" "${spkgfile##*/}" + format_entry "CSIZE" "$scsize" + format_entry "MD5SUM" "$smd5sum" + format_entry "SHA256SUM" "$ssha256sum" + format_entry "PGPSIG" "$spgpsig" + } >'source' + fi + popd >/dev/null popd >/dev/null

@@ -640,6 +710,7 @@ while (( $# )); do -n|--new) ONLYADDNEW=1;; -R|--remove) RMEXISTING=1;; -f|--files) WITHFILES=1;; + -S|--source) SOURCE=1;; --nocolor) USE_COLOR='n';; -s|--sign) check_gpg -- 1.8.4.2

Allan McRae

5 Nov 5 Nov

5:01 a.m.

New subject: [pacman-dev] [PATCH 3/3] repo-add: add source package information to sync db

On 03/11/13 10:38, Dave Reisner wrote:

...

On Sat, Nov 02, 2013 at 12:21:42PM +1000, Allan McRae wrote:

...
When using the -S/--source option, add source package information to the sync db if one is found alongside the package file.

TODO: Add documentation

Signed-off-by: Allan McRae --- lib/libalpm/be_sync.c | 2 ++ scripts/repo-add.sh.in | 75 ++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 75 insertions(+), 2 deletions(-)

diff --git a/lib/libalpm/be_sync.c b/lib/libalpm/be_sync.c index 6ebdf3c..eb60b64 100644 --- a/lib/libalpm/be_sync.c +++ b/lib/libalpm/be_sync.c @@ -674,6 +674,8 @@ static int sync_db_read(alpm_db_t *db, struct archive *archive, /* skip reading delta files if UseDelta is unset */ } else if(strcmp(filename, "files") == 0) { /* currently do nothing with this file */ + } else if(strcmp(filename, "source") == 0) { + /* currently do nothing with this file */ } else { /* unknown database file */ _alpm_log(db->handle, ALPM_LOG_DEBUG, "unknown database file: %s\n", filename); diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index dbb635a..e4a34d1 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -33,6 +33,7 @@ DELTA=0 ONLYADDNEW=0 RMEXISTING=0 WITHFILES=0 +SOURCE=0 SIGN=0 VERIFY=0 REPO_DB_FILE= @@ -61,6 +62,7 @@ Multiple packages to add can be specified on the command line.\n")" printf -- "$(gettext " -n, --new only add packages that are not already in the database\n")" printf -- "$(gettext " -R, --remove remove package file from disk when updating database entry\n")" printf -- "$(gettext " -f, --files update database's file list\n")" + printf -- "$(gettext " -S, --source add source package information to the database\n")" elif [[ $cmd == "repo-remove" ]] ; then printf -- "$(gettext "Usage: repo-remove [options] <path-to-db> ...\n")" printf -- "\n" @@ -124,6 +126,35 @@ find_pkgentry() { return 1 }

+match_source_package() { + case $# in + 0) + warning "$(gettext "No source package found")" + return 1 + ;;

I think if you check for existence here you make this a more robust function which doesn't silently require nullglob. This case label then simply becomes 0|1.

Can you provide more detail here? I am not sure I am understanding where you are directing me...

...

...
+ [!1]) + warning "$(gettext "Cannot determine source package - multiple matches found:")" + msg2 '%s' "$@" + return 1 + ;; + esac + echo "$1" +} + +find_source_package() { + local pkgfile pkgname pkgver sourcepkg + + pkgfile=$1 + pkgname=$2 + pkgver=$3 + + local sourcepkg=$(shopt -s nullglob; + match_source_package "$(dirname "$pkgfile")/$pkgname-$pkgver".src.tar!(*.sig)) + + echo "$sourcepkg"

You probably don't want to do this if match_source_package failed...

...
+} + funcgrep() { awk -v funcmatch="$1" ' /^[[:space:]]*[[:alnum:]_]+[[:space:]]*$[[:space:]]*$/ { @@ -272,8 +303,9 @@ db_write_entry() { local pkgfile=$1 local -a _groups _licenses _replaces _depends _conflicts _provides \ _optdepends _makedepends _checkdepends - local pkgname pkgver pkgdesc csize size url arch builddate packager \ - md5sum sha256sum pgpsig pgpsigsize install_functions + local pkgname pkgbase pkgver basever pkgdesc csize size url arch builddate \ + packager md5sum sha256sum pgpsig pgpsigsize install_functions \ + spkgfile scsize smd5sum ssha256sum spgpsig

# read info from the zipped package local line var val @@ -353,6 +385,31 @@ db_write_entry() { sha256sum=$(openssl dgst -sha256 "$pkgfile") sha256sum=${sha256sum##* }

+ # gather source package information + if (( SOURCE )); then + msg2 "$(gettext "Adding source package...")" + + spkgfile=$(find_source_package "$pkgfile" ${pkgbase:-$pkgname} ${basever:-$pkgver}) + + if [[ -n "$spkgfile" ]]; then + if [[ -f "$spkgfile.sig" ]]; then + pgpsigsize=$(@SIZECMD@ -L "$spkgfile.sig") + if (( pgpsigsize > 16384 )); then + error "$(gettext "Invalid package signature file '%s'.")" "$spkgfile.sig" + return 1 + fi + spgpsig=$(openssl base64 -in "$spkgfile.sig" | tr -d '\n') + fi + + scsize=$(@SIZECMD@ -L "$spkgfile") + + smd5sum=$(openssl dgst -md5 "$spkgfile") + smd5sum=${md5sum##* } + ssha256sum=$(openssl dgst -sha256 "$spkgfile") + ssha256sum=${sha256sum##* } + fi + fi + # remove an existing entry if it exists, ignore failures db_remove_entry "$pkgname"

@@ -371,6 +428,7 @@ db_write_entry() { format_entry "NAME" "$pkgname" format_entry "BASE" "$pkgbase" format_entry "VERSION" "$pkgver" + format_entry "BASEVER" "$basever" format_entry "DESC" "$pkgdesc" format_entry "GROUPS" "${_groups[@]}" format_entry "CSIZE" "$csize" @@ -403,6 +461,18 @@ db_write_entry() { format_entry "CHECKDEPENDS" "${_checkdepends[@]}" } >'depends'

+ # create source entry + if (( SOURCE )) && [[ -n "$spkgfile" ]]; then + msg2 "$(gettext "Creating '%s' db entry...")" 'source' + { + format_entry "FILENAME" "${spkgfile##*/}" + format_entry "CSIZE" "$scsize" + format_entry "MD5SUM" "$smd5sum" + format_entry "SHA256SUM" "$ssha256sum" + format_entry "PGPSIG" "$spgpsig" + } >'source' + fi + popd >/dev/null popd >/dev/null

@@ -640,6 +710,7 @@ while (( $# )); do -n|--new) ONLYADDNEW=1;; -R|--remove) RMEXISTING=1;; -f|--files) WITHFILES=1;; + -S|--source) SOURCE=1;; --nocolor) USE_COLOR='n';; -s|--sign) check_gpg -- 1.8.4.2

Dave Reisner

6:25 p.m.

New subject: [pacman-dev] [PATCH 3/3] repo-add: add source package information to sync db

On Tue, Nov 05, 2013 at 11:01:57AM +1000, Allan McRae wrote:

...

On 03/11/13 10:38, Dave Reisner wrote:

...
On Sat, Nov 02, 2013 at 12:21:42PM +1000, Allan McRae wrote:

...
When using the -S/--source option, add source package information to the sync db if one is found alongside the package file.

TODO: Add documentation

Signed-off-by: Allan McRae --- lib/libalpm/be_sync.c | 2 ++ scripts/repo-add.sh.in | 75 ++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 75 insertions(+), 2 deletions(-)

diff --git a/lib/libalpm/be_sync.c b/lib/libalpm/be_sync.c index 6ebdf3c..eb60b64 100644 --- a/lib/libalpm/be_sync.c +++ b/lib/libalpm/be_sync.c @@ -674,6 +674,8 @@ static int sync_db_read(alpm_db_t *db, struct archive *archive, /* skip reading delta files if UseDelta is unset */ } else if(strcmp(filename, "files") == 0) { /* currently do nothing with this file */ + } else if(strcmp(filename, "source") == 0) { + /* currently do nothing with this file */ } else { /* unknown database file */ _alpm_log(db->handle, ALPM_LOG_DEBUG, "unknown database file: %s\n", filename); diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index dbb635a..e4a34d1 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -33,6 +33,7 @@ DELTA=0 ONLYADDNEW=0 RMEXISTING=0 WITHFILES=0 +SOURCE=0 SIGN=0 VERIFY=0 REPO_DB_FILE= @@ -61,6 +62,7 @@ Multiple packages to add can be specified on the command line.\n")" printf -- "$(gettext " -n, --new only add packages that are not already in the database\n")" printf -- "$(gettext " -R, --remove remove package file from disk when updating database entry\n")" printf -- "$(gettext " -f, --files update database's file list\n")" + printf -- "$(gettext " -S, --source add source package information to the database\n")" elif [[ $cmd == "repo-remove" ]] ; then printf -- "$(gettext "Usage: repo-remove [options] <path-to-db> ...\n")" printf -- "\n" @@ -124,6 +126,35 @@ find_pkgentry() { return 1 }

+match_source_package() { + case $# in + 0) + warning "$(gettext "No source package found")" + return 1 + ;;

I think if you check for existence here you make this a more robust function which doesn't silently require nullglob. This case label then simply becomes 0|1.

Can you provide more detail here? I am not sure I am understanding where you are directing me...

Sure. Here's what I had in mind: match_source_package() { case $# in 0|1) if [[ ! -f $1 ]]; then warning "$(gettext "No source package found")" return 1 fi ;; *) warning "$(gettext "Cannot determine source package - multiple matches found:")" msg2 '%s' "$@" return 1 ;; esac echo "$1" } Now when you call match_source_package(), you don't have to worry about setting nullglob.

...

...
...
+ [!1]) + warning "$(gettext "Cannot determine source package - multiple matches found:")" + msg2 '%s' "$@" + return 1 + ;; + esac + echo "$1" +} + +find_source_package() { + local pkgfile pkgname pkgver sourcepkg + + pkgfile=$1 + pkgname=$2 + pkgver=$3 + + local sourcepkg=$(shopt -s nullglob; + match_source_package "$(dirname "$pkgfile")/$pkgname-$pkgver".src.tar!(*.sig)) + + echo "$sourcepkg"

You probably don't want to do this if match_source_package failed...

...
+} + funcgrep() { awk -v funcmatch="$1" ' /^[[:space:]]*[[:alnum:]_]+[[:space:]]*$[[:space:]]*$/ { @@ -272,8 +303,9 @@ db_write_entry() { local pkgfile=$1 local -a _groups _licenses _replaces _depends _conflicts _provides \ _optdepends _makedepends _checkdepends - local pkgname pkgver pkgdesc csize size url arch builddate packager \ - md5sum sha256sum pgpsig pgpsigsize install_functions + local pkgname pkgbase pkgver basever pkgdesc csize size url arch builddate \ + packager md5sum sha256sum pgpsig pgpsigsize install_functions \ + spkgfile scsize smd5sum ssha256sum spgpsig

# read info from the zipped package local line var val @@ -353,6 +385,31 @@ db_write_entry() { sha256sum=$(openssl dgst -sha256 "$pkgfile") sha256sum=${sha256sum##* }

+ # gather source package information + if (( SOURCE )); then + msg2 "$(gettext "Adding source package...")" + + spkgfile=$(find_source_package "$pkgfile" ${pkgbase:-$pkgname} ${basever:-$pkgver}) + + if [[ -n "$spkgfile" ]]; then + if [[ -f "$spkgfile.sig" ]]; then + pgpsigsize=$(@SIZECMD@ -L "$spkgfile.sig") + if (( pgpsigsize > 16384 )); then + error "$(gettext "Invalid package signature file '%s'.")" "$spkgfile.sig" + return 1 + fi + spgpsig=$(openssl base64 -in "$spkgfile.sig" | tr -d '\n') + fi + + scsize=$(@SIZECMD@ -L "$spkgfile") + + smd5sum=$(openssl dgst -md5 "$spkgfile") + smd5sum=${md5sum##* } + ssha256sum=$(openssl dgst -sha256 "$spkgfile") + ssha256sum=${sha256sum##* } + fi + fi + # remove an existing entry if it exists, ignore failures db_remove_entry "$pkgname"

@@ -371,6 +428,7 @@ db_write_entry() { format_entry "NAME" "$pkgname" format_entry "BASE" "$pkgbase" format_entry "VERSION" "$pkgver" + format_entry "BASEVER" "$basever" format_entry "DESC" "$pkgdesc" format_entry "GROUPS" "${_groups[@]}" format_entry "CSIZE" "$csize" @@ -403,6 +461,18 @@ db_write_entry() { format_entry "CHECKDEPENDS" "${_checkdepends[@]}" } >'depends'

+ # create source entry + if (( SOURCE )) && [[ -n "$spkgfile" ]]; then + msg2 "$(gettext "Creating '%s' db entry...")" 'source' + { + format_entry "FILENAME" "${spkgfile##*/}" + format_entry "CSIZE" "$scsize" + format_entry "MD5SUM" "$smd5sum" + format_entry "SHA256SUM" "$ssha256sum" + format_entry "PGPSIG" "$spgpsig" + } >'source' + fi + popd >/dev/null popd >/dev/null

@@ -640,6 +710,7 @@ while (( $# )); do -n|--new) ONLYADDNEW=1;; -R|--remove) RMEXISTING=1;; -f|--files) WITHFILES=1;; + -S|--source) SOURCE=1;; --nocolor) USE_COLOR='n';; -s|--sign) check_gpg -- 1.8.4.2

3833

Age (days ago)

3836

Last active (days ago)

List overview

Download

6 comments

2 participants

participants (2)

Allan McRae
Dave Reisner