On Sat, Nov 8, 2008 at 12:47 AM, Henning Garus <henning.garus@googlemail.com> wrote:
2. create the package, but don't compress it with bsdtar, use gzip -n instead. This means we have to use gzip again, in libalpm, when we apply the delta.
Seems better than 1, but makes makepkg and libalpm rely on gzip. Not sure if this is a good thing, especially for libalpm.
That sounds alright, I just noticed that xdelta3 has an option to disable the external recompression : -R So we don't even have extra decompression/recompression steps, there is no loss. + snprintf(command, PATH_MAX, "xdelta3 -d -R -c -s %s %s | gzip -n > %s", from, delta, to);
In a previous mail Xavier toyed with the idea to put delta creation into repo-add, I have given this some thought, as it seems nice in principle, but there are drawbacks. For Arch this would mean creating deltas on Gerolde, which seems to be fairly strained already, according to the dev list. Furthermore this introduces some new variables to repo-add (at least repo location and an output location) this would be manageable, but doesn't look very nice.
Delta creation in makepkg seems somehow ok (its already in there after all). But what I would really like is a separate tool for delta creation, which would allow the separation of building packages and creating deltas and setting up a separated delta server. This leaves us with options 2 and 3 and I am not really sure, which way to go.
looking forward to your comments
I just went further than I ever did on this task, I seriously considered a separate tool and spent some times thinking about all the possibilities. I am still not sure what is best. My thought was that it was very easy to generate the database info during the delta creation. However, how do we keep this info? Originally it was stored in the delta filename. But this was before the database change. Now we need two filenames and two md5sums (old pkg and new pkg), it does not seem realistic to store all this in the delta filename. Here are the options I considered : 1) delta support only in repo-add No problem of a temporary storage of the info here, it goes directly into the database. But maybe not flexible enough. 2) embed the .delta files into another format, eg delta.tar.gz archive = delta file + DELTA metafile Might be overkill? And we lost the ability of using xdelta directly 3) a separate tool creates the delta, generates the delta metainfo and stores it in a file This file can then be given to repo-add which basically just add its contents to $pkgname-*/deltas I gave a try to that third option. It's clearly not finished yet but I am attaching the script in its current state to give an idea and to know if I should move forward. Example of usage: $ create-xdelta libxml2-2.7.2-1-x86_64.pkg.tar.gz libxml2-2.7.3-1-x86_64.pkg.tar.gz $ create-xdelta libxml2-2.7.3-1-x86_64.pkg.tar.gz libxml2-2.7.3-1.1-x86_64.pkg.tar.gz (these two commands added one line each in a libxml2.pacdelta file) $ repo-add db.tar.gz libxml2-2.7.3-1.1-x86_64.pkg.tar.gz $ repo-add db.tar.gz libxml2.pacdelta (pkg.tar.gz and pacdelta can be added together, but the order is important, we need a package entry for libxml2 before adding deltas) Now we can upgrade from 2.7.2-1 or 2.7.3-1 to 2.7.3-1.1 using deltas.