[pacman-dev] Idea: Merge -U and -S (half universal transaction)?
Hi! After killing pmsyncpkg_t, there is not much difference between -S and -U transactions. The only difference is that trans->packages come from pkgcache (-S) or they are loaded from file (-U). And of course, with -S we have an extra step, we have to actually download the packages. However, -U is much more "stupid" than -S atm. Now the removes field is attached to pmpkg_t, so we could implement FS#3492 without pain, the conflict resolving part from sync.c can be reused without any modification*. Or if we allow "mixed" trans->packages list (pkg->origin can indicate that we have a file[-U] or repo[-S] package), even the implementaion of FS#5798 should be also straightforward. The only difference between PKG_FROM_CACHE and PKG_FROM_FILE packages, that PKG_FROM_CACHE needs to be downloaded. Then -U would become a sync transaction (internally), and add_commit would be a helper function only. *: I started to work on the implementation of mixed target list, and I realised that the most annoying "difference" is that PKG_FROM_CACHE packages are not duplicated, so they are mustn't be freed (the new _alpm_pkg_free_trans should be used), but PKG_FROM_FILE packages must be freed. How much memory can we spare by not duplicating sync packages (not calling _alpm_pkg_dup when adding them to transaction target)? The whole base repo is <0.5 MB atm. Does this hack "worth"? With the old behaviour, implementing this idea (and so FS#3492 and FS#5798) would be really nice. With the new behaviour it would not. (And I don't really like the need for _alpm_pkg_free_trans neither.) Bye
On Mon, Jun 8, 2009 at 1:24 PM, Nagy Gabor
Hi!
After killing pmsyncpkg_t, there is not much difference between -S and -U transactions. The only difference is that trans->packages come from pkgcache (-S) or they are loaded from file (-U). And of course, with -S we have an extra step, we have to actually download the packages.
However, -U is much more "stupid" than -S atm.
Now the removes field is attached to pmpkg_t, so we could implement FS#3492 without pain, the conflict resolving part from sync.c can be reused without any modification*. Or if we allow "mixed" trans->packages list (pkg->origin can indicate that we have a file[-U] or repo[-S] package), even the implementaion of FS#5798 should be also straightforward. The only difference between PKG_FROM_CACHE and PKG_FROM_FILE packages, that PKG_FROM_CACHE needs to be downloaded. Then -U would become a sync transaction (internally), and add_commit would be a helper function only.
*: I started to work on the implementation of mixed target list, and I realised that the most annoying "difference" is that PKG_FROM_CACHE packages are not duplicated, so they are mustn't be freed (the new _alpm_pkg_free_trans should be used), but PKG_FROM_FILE packages must be freed. How much memory can we spare by not duplicating sync packages (not calling _alpm_pkg_dup when adding them to transaction target)? The whole base repo is <0.5 MB atm. Does this hack "worth"? With the old behaviour, implementing this idea (and so FS#3492 and FS#5798) would be really nice. With the new behaviour it would not. (And I don't really like the need for _alpm_pkg_free_trans neither.)
I think it'd be neat! That's all I got :)
Hi!
After killing pmsyncpkg_t, there is not much difference between -S and -U transactions. The only difference is that trans->packages come from pkgcache (-S) or they are loaded from file (-U). And of course, with -S we have an extra step, we have to actually download the packages.
However, -U is much more "stupid" than -S atm.
Now the removes field is attached to pmpkg_t, so we could implement FS#3492 without pain, the conflict resolving part from sync.c can be reused without any modification*. Or if we allow "mixed" trans->packages list (pkg->origin can indicate that we have a file[-U] or repo[-S] package), even the implementaion of FS#5798 should be also straightforward. The only difference between PKG_FROM_CACHE and PKG_FROM_FILE packages, that PKG_FROM_CACHE needs to be downloaded. Then -U would become a sync transaction (internally), and add_commit would be a helper function only.
*: I started to work on the implementation of mixed target list, and I realised that the most annoying "difference" is that PKG_FROM_CACHE packages are not duplicated, so they are mustn't be freed (the new _alpm_pkg_free_trans should be used), but PKG_FROM_FILE packages must be freed. How much memory can we spare by not duplicating sync packages (not calling _alpm_pkg_dup when adding them to transaction target)? The whole base repo is <0.5 MB atm. Does this hack "worth"? With the old behaviour, implementing this idea (and so FS#3492 and FS#5798) would be really nice. With the new behaviour it would not. (And I don't really like the need for _alpm_pkg_free_trans neither.)
Bye
A proof-of-concept repo is here: http://repo.or.cz/w/pacman-ng.git?a=shortlog;h=refs/heads/universal This is in alpha state atm, it needs some testing, and probably I will modify the "main patch" later (because of found bugs ;-). Bye
On Mon, Jun 8, 2009 at 8:24 PM, Nagy Gabor
*: I started to work on the implementation of mixed target list, and I realised that the most annoying "difference" is that PKG_FROM_CACHE packages are not duplicated, so they are mustn't be freed (the new _alpm_pkg_free_trans should be used), but PKG_FROM_FILE packages must be freed. How much memory can we spare by not duplicating sync packages (not calling _alpm_pkg_dup when adding them to transaction target)? The whole base repo is <0.5 MB atm. Does this hack "worth"? With the old behaviour, implementing this idea (and so FS#3492 and FS#5798) would be really nice. With the new behaviour it would not. (And I don't really like the need for _alpm_pkg_free_trans neither.)
Yeah it looks quite painful to handle. You should have a careful look at this article : http://www.toofishes.net/blog/valgrind-330-and-new-massif/ It would be nice if you could run these experiments yourself on the current codebase, and check the difference is indeed <0.5MB.
participants (3)
-
Aaron Griffin
-
Nagy Gabor
-
Xavier