[arch-dev-public] [PATCH 0/7] *** SUBJECT HERE ***
In the spirit of Eli making a bunch of patches for the AUR, I finally decided to sit down tonight and figure out why the heck namcap was sucking it up, and did a little code cleanup along the way. namcap.py is now a bit cleaner and separated into functions, and the real treat is namcap is a hell of a lot faster now that I found the bottleneck, which was the depends hook. The first 6 patches in this series lead up to the 7th, which is where the speed increase is found. Let me know what you see, otherwise it would be cool to get this in and a new namcap release made, as it has rather dramatic effects with regards to speed. If you don't like patches, you can get all this from my git tree as well: http://code.toofishes.net/cgit/dan/namcap.git/log/?h=working -Dan Dan McGee (7): Rename 'tags' to 'namcap-tags' Only process tags if necessary Move extracted variable to the correct scope Only do active_modules check once Move PKGBUILD processing to a function Move real package processing to a function Make the depends module not suck Namcap/depends.py | 104 +++++++++++++++++++------------- README | 10 ++-- namcap-tags | 65 ++++++++++++++++++++ namcap.py | 173 +++++++++++++++++++++++++++-------------------------- setup.py | 2 +- tags | 65 -------------------- tests/tags-check | 4 +- 7 files changed, 224 insertions(+), 199 deletions(-) create mode 100644 namcap-tags delete mode 100644 tags
It was a rather generic name for a file. More importantly, it prevented the
use of ctags in this projects since ctags creates a file named 'tags' by
default for storing its information.
Signed-off-by: Dan McGee
We don't even need to look at the tags file if we are outputting machine
readable tags, so don't bother. Make process_tags just return the lambda
directly and even find a valid case for using a closure!
Signed-off-by: Dan McGee
Signed-off-by: Dan McGee
Signed-off-by: Dan McGee
Signed-off-by: Dan McGee
Signed-off-by: Dan McGee
Namcap being slow pissed me off enough that I wanted to figure out what
could be taking so long. As is usual with software, 1 module was causing
about 90% of the slowdown. Compare the before and after with this patch:
$ time namcap -m -r depends {4 packages} >/dev/null
real 0m18.001s
user 0m9.313s
sys 0m7.966s
$ time ./namcap.py -m -r depends {4 packages }>/dev/null
real 0m4.512s
user 0m3.770s
sys 0m0.740s
This patch addresses a few issue noticed when crawling and profiling the code
using cProfile. First, we were doing O(n*m) string compares of files in the
local database with libraries in the package. We can use a few tricks to cut
down the number of checks- namely, once a library has been found we don't need
to look for it anymore, and we can also ignore any non-'.so' files.
Another part where we were really shooting ourselves in the foot was invoking
readelf on *every single file* in the package. Cut that down so we only call
readelf on files with the ELF magic bytes, which gives us a dramatic savings as
we no longer fork for files in /usr/share, for instance.
Bonus material: script checking left out perl, and script dependencies were
formerly listed as link-level dependencies as well. Both of these have been
fixed.
The sad part is there is still room to improve here, but I decided enough
was enough for now. :)
Signed-off-by: Dan McGee
On Mon, Sep 28, 2009 at 6:16 AM, Dan McGee
In the spirit of Eli making a bunch of patches for the AUR, I finally decided to sit down tonight and figure out why the heck namcap was sucking it up, and did a little code cleanup along the way. namcap.py is now a bit cleaner and separated into functions, and the real treat is namcap is a hell of a lot faster now that I found the bottleneck, which was the depends hook. The first 6 patches in this series lead up to the 7th, which is where the speed increase is found.
Let me know what you see, otherwise it would be cool to get this in and a new namcap release made, as it has rather dramatic effects with regards to speed.
If you don't like patches, you can get all this from my git tree as well: http://code.toofishes.net/cgit/dan/namcap.git/log/?h=working
That sounds pretty cool. Just yesterday, I was complaining to myself that namcap was too slow. I will give this a try :)
participants (2)
-
Dan McGee
-
Xavier