[pacman-dev] xdelta status
Today I read a post[1] by Xavier about how deltas cannot currently be used. As I say in the thread, I will work on adding delta creation support to repo-add in the next week. Is there anything else that needs to be changed before this can actually be used? [1] http://bbs.archlinux.org/viewtopic.php?id=43426
On Feb 6, 2008 3:57 AM, Nathan Jones <nathanj@insightbb.com> wrote:
Today I read a post[1] by Xavier about how deltas cannot currently be used. As I say in the thread, I will work on adding delta creation support to repo-add in the next week.
Is there anything else that needs to be changed before this can actually be used?
Quoting Dan from the TODO thread :
xdelta support in repo-add would probably be our next step- we may need to refactor the common code out of repo-add and makepkg into some kind of makepkg lib, which I am not completely sure how to address.
Nathan Jones wrote:
Today I read a post[1] by Xavier about how deltas cannot currently be used. As I say in the thread, I will work on adding delta creation support to repo-add in the next week.
Is there anything else that needs to be changed before this can actually be used?
Another issue is that dbscripts (the database scripts used for core / extra repos) don't use repo-add, even if there is a plan to do that. (see http://www.archlinux.org/pipermail/pacman-dev/2008-January/010999.html) But community already uses repo-add, so maybe once support is added to repo-add, it would be easy to get xdelta in community?
On Feb 6, 2008 7:29 AM, Xavier <shiningxc@gmail.com> wrote:
Another issue is that dbscripts (the database scripts used for core / extra repos) don't use repo-add, even if there is a plan to do that. (see http://www.archlinux.org/pipermail/pacman-dev/2008-January/010999.html)
Don't worry I will have these rewritten soon. They are my number one priority right now. Never fear
Aaron Griffin wrote:
On Feb 6, 2008 7:29 AM, Xavier<shiningxc@gmail.com> wrote:
Another issue is that dbscripts (the database scripts used for core / extra repos) don't use repo-add, even if there is a plan to do that. (see http://www.archlinux.org/pipermail/pacman-dev/2008-January/010999.html)
Don't worry I will have these rewritten soon. They are my number one priority right now. Never fear
Oh great, that's excellent news :)
I was also wondering about the status of xdelta itself. 1) From http://code.google.com/p/xdelta/ :
Xdelta is a tool and library for differential compression. Xdelta release version 3 supports VCDIFF encoding and decoding. This is a BETA release, but almost stable. Supports compressing 64 bit files on Windows, Linux, etc.
Xdelta 1.1.4 is the last stable release. Xdelta 1.1.0 was released in 1999.
So, the arch package will be updated to xdelta 3 only when it is marked stable, right? Since it's almost stable, maybe this could happen in the near future. Will this cause any problems on pacman side? 2) There was also a comparison of both there : http://bbs.archlinux.org/viewtopic.php?pid=255687#p255687 "The only advantages of xdelta3 I see right now is its reduced disk-space usage during delta creation, and the possibility of completely closing the rather unimportant md5 hole without extra patching time/disc-space. That and it seems to be generally a bit quicker in patching." This benchmark was made 8 months ago though, so xdelta3 might be even better now. 3) Also when xdelta 3 is stable, we will have the alternative to use the API. It looks like we could simply do a copy / paste of the code() function there which is ~100 lines : http://code.google.com/p/xdelta/source/browse/trunk/xdelta3/examples/encode_... Then it's just a matter of using fopen, calling that code function, and then fclose. But while using the API looks cleaner, Dan had several arguments against doing it: http://www.archlinux.org/pipermail/pacman-dev/2007-October/009627.html
On Thu, Feb 07, 2008 at 02:03:29PM +0100, Xavier wrote:
I was also wondering about the status of xdelta itself.
1) From http://code.google.com/p/xdelta/ :
Xdelta is a tool and library for differential compression. Xdelta release version 3 supports VCDIFF encoding and decoding. This is a BETA release, but almost stable. Supports compressing 64 bit files on Windows, Linux, etc.
Xdelta 1.1.4 is the last stable release. Xdelta 1.1.0 was released in 1999.
So, the arch package will be updated to xdelta 3 only when it is marked stable, right? Since it's almost stable, maybe this could happen in the near future. Will this cause any problems on pacman side?
Both versions of xdelta can be installed at the same time. xdelta1 consists of /usr/bin/xdelta, libedsio, libxdelta, and include files. xdelta3 is simply /usr/bin/xdelta3. Probably the easiest way to handle this would be to create a seperate package for xdelta3 so pacman can continue to use xdelta1 as long as needed. If we decide to go with the xdelta3 API, users will not actually have to install xdelta at all. See my comments below.
2) There was also a comparison of both there : http://bbs.archlinux.org/viewtopic.php?pid=255687#p255687 "The only advantages of xdelta3 I see right now is its reduced disk-space usage during delta creation, and the possibility of completely closing the rather unimportant md5 hole without extra patching time/disc-space. That and it seems to be generally a bit quicker in patching."
This benchmark was made 8 months ago though, so xdelta3 might be even better now.
3) Also when xdelta 3 is stable, we will have the alternative to use the API. It looks like we could simply do a copy / paste of the code() function there which is ~100 lines : http://code.google.com/p/xdelta/source/browse/trunk/xdelta3/examples/encode_...
Then it's just a matter of using fopen, calling that code function, and then fclose.
But while using the API looks cleaner, Dan had several arguments against doing it: http://www.archlinux.org/pipermail/pacman-dev/2007-October/009627.html
Quoting Dan's email:
1. Using the API does introduce yet another dependency for pacman...our ldd output is starting to get a bit lengthy for what I feel should be a low level tool. If someone doesn't ever use the delta features of pacman, they would still have to have xdelta installed (because of the dynamic link at runtime, and yet some more mem usage).
One of the nice(?) things about xdelta3 is that everything is in one file: xdelta3.c. The encode_decode_test.c posted above does not link with any library; it just includes xdelta3.c. Users will not have to install anything extra to get delta support in pacman. The encode_decode_test is 58K when compiled with -O3.
2. xdelta3 tarball was chmodded all wrong. Please tell me the developer just made a mistake here, otherwise it is very windows focused. And the configure script is no longer there...
The latest tarball looks ok. There is still no configure script, but the Makefile works fine.
participants (3)
-
Aaron Griffin
-
Nathan Jones
-
Xavier