[pacman-dev] Removing deltas from repos - Was: Allow package to display a brief message before sync install

[Xavier]

On Wed, Sep 16, 2009 at 10:35 PM, Marc - A. Dahlhaus <mad at wol.de> wrote:
>
> I plan to do it based on the accumulated filesize of all deltas that could
> be chained together from the repo package version without gaps. As long as
> this accumulated size is smaller than eg. 90% of the repo packages size (as
> Allan mentioned that pacmans logic was based on that in the other mail) the
> delta will be kept.
>

btw its 70% but it does not change anything.

> It would make sense to use a recursion based implementation.
>
> A function with two params:
> $1: version we want to chain down from
> $2: accumulated size in bytes of deltas above us in the chain
>
> first call would look like:
>
> function currents-package-version 0
>
> inside of function we read all source-versions and filesizes of deltas
> containing "_to_${1}" in the filename and call function for each match with
> the source-version and the filesize+$2. We check every recursions retval. If
> we get a 0 retval for a delta, we would add it to the next repo archive. We
> return 1 if $2 is larger than repo packages filesize*0.9. Last thing in this
> function would be a "return 0".
>
> This would walk down the chain as far as it could and then on its way
> returning up to the first caller it will add the appropriate deltas to the
> repo. If there are deltas in the old repo that can't be chained to other
> deltas and don't have the repo package version as target get removed as they
> will not be found by the recursion.
>
> But i didn't read through the current code in repo-add or pacmans code for
> the incremental delta handling so far, so this is a very early draft of how
> i would do it and i could be very far out in the woods with it... :-D
>

It took me some time to understand how this would work when just
reading that mail, but when trying to implement it it revealed to be
very simple. Attached a quick hack in bash I just wrote. However, my
current implementation is widely inefficient.
And currently I just print the delta which are reachable and too big.
So we don' t get the unreachable ones.
It would indeed be better to just print delta reachable and with a
good size, but then we would have to inverse that list somehow.

I am still not sure what to do.. I am not sure bash is an appropriate
language for this task. And I am not sure this fits well in repo-add.
Currently, repo-add does not do any parsing of the entries.
But half the code for cleaning up delta is actually database parsing
(need filename and csize from desc, and every line from deltas)

I think I still prefer to hack libalpm delta code to do what I want,
just returning a list of bad delta which can then be fed to
repo-remove (and possibly another script taking care of removing the
actual delta files from the FS)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cleanup.sh
Type: application/x-sh
Size: 855 bytes
Desc: not available
URL: <http://mailman.archlinux.org/pipermail/pacman-dev/attachments/20090917/76d9e719/attachment-0001.sh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.db.tar.gz
Type: application/x-gzip
Size: 20268 bytes
Desc: not available
URL: <http://mailman.archlinux.org/pipermail/pacman-dev/attachments/20090917/76d9e719/attachment-0001.bin>