[arch-dev-public] Namcap 3.0 coming soon
Hello people, Namcap can still be found at the usual Git repository. I should release version 3.0 in a few days. Here are some highlights of what's new: * it works with Python 3.x (tested with Python 3.2) * it no longer outputs crazy contradicting messages about dependencies missing/unneeded at the same time (FS #15591 and #17166) * it finally can read split PKGBUILDs : a very basic rule has been implemented (FS #15027). It is however as poorly useful as the old depends module * it has a fairly extensive test suite (with currently 3 expected failures) * new rules have been implemented (FS#23003, FS#22881, FS#22929, FS#18852) * the help message gives more useful information about available options * the undocumented option "-r list" giving the rule list is now the documented option "-L" Big changes in code have been made : * it works with Python 3 * it uses the distribute module to build * the rules and the tests are derived from abstract base classes to help with code factorization * rules are in the rules/ subdirectory * the PacmanPackage class has a dictionary-API which avoids using hasattr/getattr/setattr everywhere and gives more Python-looking source code * none of the rules needs a complete extraction of a tarball. Only ELF files are written to disk because neither objdump nor readelf wantes to read from stdin. * parsepkgbuild is now a wrapper in /usr/bin and an auxiliary script in /usr/share which is run under "bash -r". -- Rémy.
On 2011/2/27 Rémy Oudompheng <remyoudompheng@gmail.com> wrote:
* it finally can read split PKGBUILDs : a very basic rule has been implemented (FS #15027). It is however as poorly useful as the old depends module
It behaves now better. A preliminary built package can be found at http://dev.archlinux.org/~remy/namcap/namcap-3.0-1-any.pkg.tar.xz -- Rémy.
And additionally, don't swallow error output from parsepkgbuild calls- this helped debug the next issue with a patch to come. Signed-off-by: Dan McGee <dan@archlinux.org> --- namcap.py | 13 +++++++------ pacman.py | 7 ++++++- 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/namcap.py b/namcap.py index 69787db..44440a9 100755 --- a/namcap.py +++ b/namcap.py @@ -103,7 +103,7 @@ def process_realpackage(package, modules): pkgtar = verify_package(package) if not pkgtar: - print("Error: " + package + " is empty or is not a valid package") + print("Error: %s is empty or is not a valid package" % package) return 1 pkginfo = pacman.load(package) @@ -167,11 +167,12 @@ def process_pkginfo(pkginfo, modules): def process_pkgbuild(package, modules): """Runs namcap checks over a PKGBUILD""" - # We might want to do some verifying in here... but really... isn't that what pacman.load is for? + # We might want to do some verifying in here... but really... isn't that + # what pacman.load is for? pkginfo = pacman.load(package) if pkginfo == None: - print("Error: " + package + " is not a valid PKGBUILD") + print("Error: %s is not a valid PKGBUILD" % package) return 1 # apply global PKGBUILD rules @@ -227,7 +228,7 @@ for i, k in optlist: if j in modules: active_modules[j] = modules[j] else: - print("Error: Rule '" + j + "' does not exist") + print("Error: Rule '%s' does not exist" % j) usage() # Used to exclude some rules from the check @@ -238,7 +239,7 @@ for i, k in optlist: if j in modules: active_modules.pop(j) else: - print("Error: Rule '" + j + "' does not exist") + print("Error: Rule '%s' does not exist" % j) usage() if i in ('-i', '--info'): @@ -266,7 +267,7 @@ if len(active_modules) == 0: # Go through each package, get the info, and apply the rules for package in packages: if not os.access(package, os.R_OK): - print("Error: Problem reading " + package) + print("Error: Problem reading %s" % package) usage() if os.path.isfile(package) and tarfile.is_tarfile(package): diff --git a/pacman.py b/pacman.py index eecc6c5..9473f6e 100644 --- a/pacman.py +++ b/pacman.py @@ -18,6 +18,7 @@ # import tarfile, os, re, subprocess +import sys from Namcap.package import PacmanPackage pacmandb = '/var/lib/pacman/local/' @@ -52,11 +53,15 @@ def load(package, root=None): if workingdir == '': workingdir = None filename = os.path.basename(package) - process = subprocess.Popen(['parsepkgbuild',filename], + process = subprocess.Popen(['parsepkgbuild', filename], stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=workingdir) data = process.communicate() # this means parsepkgbuild returned an error, so we are not valid if process.returncode > 0: + if data[0]: + print("Error:", data[0].decode("utf-8", "ignore")) + if data[1]: + print("Error:", data[1].decode("utf-8", "ignore"), file=sys.stdout) return None ret = PacmanPackage(db = data[0].decode('utf-8', 'ignore')) -- 1.7.4.1
A lot clearer and not magical. Signed-off-by: Dan McGee <dan@archlinux.org> --- namcap.py | 2 +- pacman.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/namcap.py b/namcap.py index 44440a9..8f69f74 100755 --- a/namcap.py +++ b/namcap.py @@ -272,7 +272,7 @@ for package in packages: if os.path.isfile(package) and tarfile.is_tarfile(package): process_realpackage(package, active_modules) - elif package[-8:] == 'PKGBUILD': + elif package.endswith('PKGBUILD'): process_pkgbuild(package, active_modules) else: print("Error: Cannot process %s" % package) diff --git a/pacman.py b/pacman.py index 9473f6e..db45539 100644 --- a/pacman.py +++ b/pacman.py @@ -47,7 +47,7 @@ def load(package, root=None): return ret # Ooooo, it's a PKGBUILD - elif package[-8:] == 'PKGBUILD': + elif package.endswith('PKGBUILD'): # Load all the data like we normally would workingdir = os.path.dirname(package) if workingdir == '': -- 1.7.4.1
We regressed when splitting the package parser into two scripts; add a variable to configure the path it is found on and add a 'namcap-devel' script that allows running the whole deal from the current git checkout. Signed-off-by: Dan McGee <dan@archlinux.org> --- namcap-devel | 9 +++++++++ parsepkgbuild | 3 ++- 2 files changed, 11 insertions(+), 1 deletions(-) create mode 100755 namcap-devel diff --git a/namcap-devel b/namcap-devel new file mode 100755 index 0000000..8619a67 --- /dev/null +++ b/namcap-devel @@ -0,0 +1,9 @@ +#!/bin/sh +PATH="$(pwd):$PATH" +PARSE_PKGBUILD_PATH="$(pwd)" +export PARSE_PKGBUILD_PATH + +PYTHONPATH="$(pwd)" +export PYTHONPATH + +./namcap -t namcap-tags $@ diff --git a/parsepkgbuild b/parsepkgbuild index 28fde8e..8f0f67d 100755 --- a/parsepkgbuild +++ b/parsepkgbuild @@ -6,6 +6,7 @@ source /etc/makepkg.conf export CARCH export PATH=/tmp/parsepkgbuild +PARSE_PKGBUILD_PATH=${PARSE_PKGBUILD_PATH:-/usr/share/namcap} -exec /bin/bash --noprofile --norc -r /usr/share/namcap/parsepkgbuild.sh $1 +exec /bin/bash --noprofile --norc -r "$PARSE_PKGBUILD_PATH"/parsepkgbuild.sh $1 -- 1.7.4.1
On 2011/2/27 Rémy Oudompheng <remyoudompheng@gmail.com> wrote:
Hello people,
Namcap can still be found at the usual Git repository. I should release version 3.0 in a few days. Here are some highlights of what's new:
Namcap 3.0 has been released in [testing], including Dan's latest patchset. -- Rémy.
On 13/03/11 22:29, Rémy Oudompheng wrote:
On 2011/2/27 Rémy Oudompheng<remyoudompheng@gmail.com> wrote:
Hello people,
Namcap can still be found at the usual Git repository. I should release version 3.0 in a few days. Here are some highlights of what's new:
Namcap 3.0 has been released in [testing], including Dan's latest patchset.
Looks good to me. I just ran it on a whole heap of PKGBUILDs and packages and nothing looked too wrong. There is a bunch of false positives in the form of: W: Non standard variable 'foo' doesn't start with an underscore e.g. the perl PKGBUILD. I'm not sure if they are new or not. What is more concerning (and unrelated to namcap...) is the number of PKGBUILDs giving this error even though they were recently updated: Use $srcdir instead of $startdir/src Allan
On Sun, Mar 13, 2011 at 8:30 AM, Allan McRae <allan@archlinux.org> wrote:
On 13/03/11 22:29, Rémy Oudompheng wrote:
On 2011/2/27 Rémy Oudompheng<remyoudompheng@gmail.com> wrote:
Hello people,
Namcap can still be found at the usual Git repository. I should release version 3.0 in a few days. Here are some highlights of what's new:
Namcap 3.0 has been released in [testing], including Dan's latest patchset.
Looks good to me. I just ran it on a whole heap of PKGBUILDs and packages and nothing looked too wrong.
There is a bunch of false positives in the form of: W: Non standard variable 'foo' doesn't start with an underscore e.g. the perl PKGBUILD. I'm not sure if they are new or not.
I'm not so sure these are false positives- the rule just didn't run at all before, if I remember one of Rémy's emails, so you never saw this trigger a warning. -Dan
On 2011/3/13 Dan McGee <dpmcgee@gmail.com> wrote:
On Sun, Mar 13, 2011 at 8:30 AM, Allan McRae <allan@archlinux.org> wrote:
Looks good to me. I just ran it on a whole heap of PKGBUILDs and packages and nothing looked too wrong.
There is a bunch of false positives in the form of: W: Non standard variable 'foo' doesn't start with an underscore e.g. the perl PKGBUILD. I'm not sure if they are new or not.
I'm not so sure these are false positives- the rule just didn't run at all before, if I remember one of Rémy's emails, so you never saw this trigger a warning.
By the way if someone could explain the exact purpose of the "CARCH" rule, which does not work currently with the various syntaxes ($CARCH, ${CARCH}, "$CARCH"), I could try to fix it, but I really don't know its purpose. -- Rémy.
participants (4)
-
Allan McRae
-
Dan McGee
-
Dan McGee
-
Rémy Oudompheng