[pacman-dev] patch] My speed improvement
Hi! This is in very early stage, just a proof of concept. Just for testing only (not merging to cvs). If you want to test it, after compiling, run 'pacman --repair' <- this creates the extra infos for the local db. Now .groups implemented only, so speedup will be seen with -Qg or similar. Fully compatible with the old db; it checks whether the speedup dotdir exists (if the answer is yes, it uses it). There is no option in "my" pacman to create speedup dirs for sync repos (because it should be downloaded in db.tar.gz), but if you want to play with it for testing, rename the repo to local and run 'pacman --repair' and rename back... Bye, Nagy Gabor My first test results: (after sync; echo 3 > /proc/sys/vm/drop_caches;) Old: real 0m4.522s user 0m0.029s sys 0m0.060s New: real 0m0.788s user 0m0.002s sys 0m0.012s
Hi! Sry, I forgot attaching the patch :-). And my first test was pacman -Qg. Bye
Hi, so this is a index for groupnames implemented by symlinks. How do you ensure the index is up2date? This is only performant because there are few groupnames. How do you want to index pkg names? You will get more files. File-io is already the performance limiting part of of the current backend. Jürgen On Fri, Mar 09, 2007 at 12:32:51AM +0100, Nagy Gabor wrote:
Hi!
This is in very early stage, just a proof of concept. Just for testing only (not merging to cvs). If you want to test it, after compiling, run 'pacman --repair' <- this creates the extra infos for the local db. Now .groups implemented only, so speedup will be seen with -Qg or similar. Fully compatible with the old db; it checks whether the speedup dotdir exists (if the answer is yes, it uses it). There is no option in "my" pacman to create speedup dirs for sync repos (because it should be downloaded in db.tar.gz), but if you want to play with it for testing, rename the repo to local and run 'pacman --repair' and rename back...
Bye, Nagy Gabor
My first test results: (after sync; echo 3 > /proc/sys/vm/drop_caches;)
Old:
real 0m4.522s user 0m0.029s sys 0m0.060s
New:
real 0m0.788s user 0m0.002s sys 0m0.012s
_______________________________________________ pacman-dev mailing list pacman-dev@archlinux.org http://www.archlinux.org/mailman/listinfo/pacman-dev
Hi,
so this is a index for groupnames implemented by symlinks. How do you ensure the index is up2date?
I implemented it yesterday, but you can imagine that only ~1 extra symlink creation/remove is needed in case of _alpm_db_write and _alpm_db_remove, so this doesn't count.
This is only performant because there are few groupnames. How do you want to index pkg names? You will get more files. File-io is already the performance limiting part of of the current backend.
Yes. I tested with the case, when all file is a group (I put all files to fakegroup) and there is not much difference. Because: In the current state you _must_ read the desc files and my solution does only "ls" (yes, in this twice but this is even much faster). The only thing remained, that free_grpcache is used too often now (because its creating is fast from groupcache) so it should be replaced with remove_from_grpcache and add_pkg_in_grpcache.
Jürgen
Other benefits: 1. groups are categorised now, so we can implement that load only groupnames to grpcache and we fill in the package list on demand 2. you needn't load pkgcache if you do -Qg or -Sg But this was only the first part of my ideas, I think .providers and .conflicts will be effective (I plan store them in a "pkgcache"). Yes, this will be useful if few packages have CONFLICTS and PROVIDES filled in, but this is always true I think. Summary: I think we can skip "brute force scan" for providers, groups and conflicts with this "dotdir way". One case, when you need to read the whole database: -Ss and similars <- we should use the db.tar file (which we store instead of delete after syncing) in these cases for sync repos. Nagy Gabor P.S.: I'm not so enthusiastic now, because overall we can reduce pacman's running time by about 50% only, because the ldconfig is really slow and we must run it in case of most of the packages.
participants (2)
-
Jürgen Hötzel
-
Nagy Gabor