On 10/5/07, Nathan Jones <nathanj@insightbb.com> wrote:
This patch adds delta information to the repository database along with the other information about a package. This would be useful if two or more package versions have been built since a user upgrades.
I don't think it would be too difficult to create an algorithm to decide which delta files, if any, would be downloaded and applied.
Any comments?
Sample output:
$ ls -l *.pkg.tar.gz *.delta -rw-r--r-- 1 nathanj users 1045940 2007-05-12 21:03 pan-devel-0.129-1-i686.pkg.tar.gz -rw-r--r-- 1 nathanj users 583339 2007-10-05 18:38 pan-devel-0.129-1_to_0.130-1-i686.delta -rw-r--r-- 1 nathanj users 1044271 2007-05-19 13:17 pan-devel-0.130-1-i686.pkg.tar.gz -rw-r--r-- 1 nathanj users 425727 2007-10-05 18:38 pan-devel-0.130-1_to_0.131-1-i686.delta -rw-r--r-- 1 nathanj users 1055063 2007-05-26 18:40 pan-devel-0.131-1-i686.pkg.tar.gz -rw-r--r-- 1 nathanj users 680644 2007-10-05 18:38 pan-devel-0.131-1_to_0.132-1-i686.delta -rw-r--r-- 1 nathanj users 1025859 2007-08-15 22:03 pan-devel-0.132-1-i686.pkg.tar.gz -rw-r--r-- 1 nathanj users 1792 2007-10-05 18:38 pan-devel-0.132-1_to_0.132-2-i686.delta -rw-r--r-- 1 nathanj users 1025826 2007-08-01 17:04 pan-devel-0.132-2-i686.pkg.tar.gz $ ~/code/pacman/scripts/repo-add repo.db.tar.gz pan-devel-0.132-2-i686.pkg.tar.gz *.delta ==> Extracting database to a temporary location... ==> Adding package 'pan-devel-0.132-2-i686.pkg.tar.gz' -> Removing existing package 'pan-devel-0.132-2'... -> Creating 'desc' db entry... -> Computing md5 checksums... -> Creating 'depends' db entry... ==> Adding delta 'pan-devel-0.129-1_to_0.130-1-i686.delta' ==> Adding delta 'pan-devel-0.130-1_to_0.131-1-i686.delta' ==> Adding delta 'pan-devel-0.131-1_to_0.132-1-i686.delta' ==> Adding delta 'pan-devel-0.132-1_to_0.132-2-i686.delta' ==> Creating updated database file /home/nathanj/pkgs/pan-devel/repo.db.tar.gz $ tar zxOf repo.db.tar.gz pan-devel-0.132-2/deltas %DELTAS% 0.129-1 0.130-1 583339 0.130-1 0.131-1 425727 0.131-1 0.132-1 680644 0.132-1 0.132-2 1792
Signed-off-by: Nathan Jones <nathanj@insightbb.com> --- scripts/repo-add.sh.in | 64 +++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 63 insertions(+), 1 deletions(-)
diff --git a/scripts/repo-add.sh.in b/scripts/repo-add.sh.in index cb741d7..4a7949e 100644 --- a/scripts/repo-add.sh.in +++ b/scripts/repo-add.sh.in @@ -97,6 +97,61 @@ write_list_entry() { fi }
+# write a delta entry to the pacman database +# arg1 - path to delta +db_write_delta() +{ + # blank out all variables and set deltafile + local deltafile=$(readlink -f "$1") + local pkgname fromver tover arch csize + + pkgname="$(echo "$1" | sed -e 's/\(.*\)-\(.*-.*\)_to_\(.*-.*\)-\(.*\).delta/\1/')" + fromver="$(echo "$1" | sed -e 's/\(.*\)-\(.*-.*\)_to_\(.*-.*\)-\(.*\).delta/\2/')" + tover="$(echo "$1" | sed -e 's/\(.*\)-\(.*-.*\)_to_\(.*-.*\)-\(.*\).delta/\3/')" + arch="$(echo "$1" | sed -e 's/\(.*\)-\(.*-.*\)_to_\(.*-.*\)-\(.*\).delta/\4/')" + + # get size of delta + csize=$(du -b -L "$deltafile" | cut -f 1) + + pushd "$gstmpdir" 2>&1 >/dev/null + + # ensure $pkgname and $pkgver variables were found + if [ -z "$pkgname" -o -z "$fromver" ]; then + error "$(gettext "Invalid delta file '%s'.")" "$deltafile" + popd 2>&1 >/dev/null + return 1 + fi + + # cd into the package's directory + local existing + local gotpackage=f + for existing in *; do + if [ "${existing%-*-*}" = "$pkgname" ]; then + cd "$existing" + gotpackage=t + break + fi + done + + # exit out if the package's directory does not exist + if [ "$gotpackage" == "f" ]; then + error "$(gettext "Cannot find directory for '%s'.")" "$pkgname" + popd 2>&1 >/dev/null + return 1 + fi + + # create deltas file if it doesn't exist + if [ ! -f "deltas" ]; then + echo -e "%DELTAS%" >>deltas + fi + + # add the entry for this delta file + echo -e "$fromver $tover $csize" >>deltas + + popd 2>&1 >/dev/null +} # end db_write_delta + + # write an entry to the pacman database # arg1 - path to package db_write_entry() @@ -242,7 +297,14 @@ for arg in "$@"; do fi else if [ -f "$arg" ]; then - if ! bsdtar -tf "$arg" .PKGINFO 2>&1 >/dev/null; then + # see if extension is .delta ... there's gotta be a better way + if [ "${arg%.delta}" != "$arg" ]; then + msg "$(gettext "Adding delta '%s'")" "$arg" + + if db_write_delta "$arg"; then + success=1 + fi + elif ! bsdtar -tf "$arg" .PKGINFO 2>&1 >/dev/null; then error "$(gettext "'%s' is not a package file, skipping")" "$arg" else msg "$(gettext "Adding package '%s'")" "$arg" -- 1.5.3.4
This idea seems pretty smart to me. A few comments/questions: 1. We probably need a way to remove 'old' deltas. Of course, I don't know what to define as old (probably retain no more than 5 deltas or something). 2. Although sed magic is cool, there has to be a better way to get to that info (pkigname, oldver, etc.) 3. Could delta addition be automatic? Right now the entries would be deleted every time you upgrade a package in the database (since the old entry is removed). Perhaps on an add, we should save the old delta entries, copy them over to the new package, and then add any new deltas that we find? This is meant to be constructive criticism- please don't let that discourage you, I like this idea. -Dan