[pacman-dev] Removing deltas from repos - Was: Allow package to display a brief message before sync install

[Marc - A. Dahlhaus [ Administration | Westermann GmbH ]]

Am Donnerstag, den 17.09.2009, 22:13 +0200 schrieb Marc - A. Dahlhaus:
> Xavier schrieb:
> > On Thu, Sep 17, 2009 at 6:42 PM, Marc - A. Dahlhaus [ Administration |
> > Westermann GmbH ] <mad at wol.de> wrote:
--8<--
> > Both scripts have the same output.
> > For example :
> > 
> > 38866 < 139532 : Ignoring dhcpcd-4.0.2-1-i686.pkg.tar.gz
> > 38866 < 88713 : Ignoring dhcpcd-4.0.2-2-i686.pkg.tar.gz
> > 38866 < 87880 : Ignoring dhcpcd-4.0.3-1-i686.pkg.tar.gz
> > 38866 < 67312 : Ignoring dhcpcd-4.0.4-1-i686.pkg.tar.gz
> > 38866 < 57650 : Ignoring dhcpcd-5.0.2-1-i686.pkg.tar.gz
> > 
> > vs
> > 
> > dhcpcd-3.2.1-1_to_4.0.2-1-i686.delta 139532
> > dhcpcd-4.0.2-1_to_4.0.2-2-i686.delta 88713
> > dhcpcd-4.0.2-2_to_4.0.3-1-i686.delta 87880
> > dhcpcd-4.0.3-1_to_4.0.4-1-i686.delta 67312
> > dhcpcd-4.0.4-1_to_5.0.2-1-i686.delta 57650
> > 
> > but I checked several other packages as well.

Yes, i misplaced the returns inside of the loop and fixed this together
with some other inefficiencies.

I can't work out an efficient implementation to print out the removed
deltas because to create such a list we need to finish the whole tree
walk before. This is because there is always the possibility that you
flag a delta to be removed that is unused in the current branch of the
tree but is useful on another branch of the tree:

| = quota reached
-> = delta file


V7 -> V6 -> V5 -> V4 -> V3 | -> V2 -> V1

V7 -> V4 -> V3 -> V2 | -> V1

As the size of the quota  V7_to_V4 could be smaller than the sum of the
single-version-quotas btween V7 and V4 we would remove a useful V3_to_V2
delta if we settle on a "remove what is detected to be useless"
algorithm...

Do we even want to print out the "useless" deltas in the first place?

I still need to look how this fits into repo-add...

What i have so far:

delta_cleanup() {
  if [ ${2} -gt ${quota} ]; then
    # we exceeded the quota, no need to walk further
    return 1
  else
    # find deltas that match our target on current level of the tree
    grep "${1}$" deltas | while read delta md5sum dsize spkg tpkg; do
      # walk down a level on the branch of each matching source
      if delta_cleanup ${spkg} $(( ${2} + ${dsize} )); then
        # don't add the same delta multiple times
        if ! grep -q "^${delta}" cdeltas &> /dev/null; then
          # add a usefull delta to the list of cleandeltas
          echo "${delta} ${md5sum} ${dsize} ${spkg} ${tpkg}" >> \
           cleandeltas
        fi
      fi
    done
    return 0
  fi
}

# this must be done with the new repos tree as PWD
if [ $CLEAN -eq 1 ]
  ls -1 | while read dir; do
    if [ ! -f ${dir}/desc ]; then
      continue
    fi
    cd ${dir}
      filename=$(grep -A1 FILENAME desc | tail -n1)
      if [ $DELTA -eq 1 -a -f deltas ]; then
        filesize=$(grep -A1 CSIZE desc | tail -n1)
        quota=$(( ${filesize} * 7 / 10 ))
        # start walking the tree for the current package
        delta_cleanup ${filename} 0
        # remove the unclean deltas file
        rm -f deltas
        # move the cleandeltas file into its final position
        [ -f cleandeltas ] && mv -f cleandeltas deltas
      fi
    cd ..
  done
fi


Marc