[arch-dev-public] Unit tests for python packages
Hi, Does anyone have experience with unit tests for python packages? It's really useful to spot missing runtime dependencies, but it's a pain to get to work. Here is what I saw: 1) just running "python -m unittest discover" or "nosetests" or equivalent in the check() function does not work, because the package is not yet installed (so all imports will fail) 2) setuptools is supposed to have a unittest integration such that "python setup.py test" should work out of the box, but in my case it never finds any tests to run 3) I've seen some more... "creative" solutions :) https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packa... So far, the only working solution I found is playing with PYTHONPATH: cd "$srcdir/$pkgname-$pkgver/tests" export PYTHONPATH="$srcdir/$pkgname-$pkgver/src" python -m unittest discover But it's a hack, and it probably won't work with things like 2to3. Any better ideas? Baptiste
On April 18, 2018 11:53:01 AM GMT+02:00, Baptiste Jonglez <baptiste@bitsofnetworks.org> wrote:
Hi,
Does anyone have experience with unit tests for python packages? It's really useful to spot missing runtime dependencies, but it's a pain to get to work.
Here is what I saw:
1) just running "python -m unittest discover" or "nosetests" or equivalent in the check() function does not work, because the package is not yet installed (so all imports will fail)
2) setuptools is supposed to have a unittest integration such that "python setup.py test" should work out of the box, but in my case it never finds any tests to run
3) I've seen some more... "creative" solutions :) https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packa...
So far, the only working solution I found is playing with PYTHONPATH:
cd "$srcdir/$pkgname-$pkgver/tests" export PYTHONPATH="$srcdir/$pkgname-$pkgver/src" python -m unittest discover
But it's a hack, and it probably won't work with things like 2to3.
Any better ideas? Baptiste
If you use a build function there shouldn't be any problems like 2to3 while running tests, everything should be converted / build and generated for the use of tests. For certain upstream test setups there won't be much you can do other then PYTHONPATH but as mentioned with a build function it should always work. You can give python-pytest-runner a go, it does proper resolution while running "python setup.py test" but requires certain ways the whole test suites are wired. Cheers Levente
On 18-04-18, Levente Polyak wrote:
On April 18, 2018 11:53:01 AM GMT+02:00, Baptiste Jonglez <baptiste@bitsofnetworks.org> wrote:
So far, the only working solution I found is playing with PYTHONPATH:
cd "$srcdir/$pkgname-$pkgver/tests" export PYTHONPATH="$srcdir/$pkgname-$pkgver/src" python -m unittest discover
If you use a build function there shouldn't be any problems like 2to3 while running tests, everything should be converted / build and generated for the use of tests.
I do use a build function: https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packa... The problem is that this package is written in python2 and relies on 2to3. If I just run the tests with python3 in the source directory, like this: cd "$srcdir/pybtex-$pkgver" export PYTHONPATH="$srcdir/pybtex-$pkgver" nosetests it fails with lots of syntax errors, because the code is python2. I tried to run the tests in the build directory, where 2to3 has been applied: cd "$srcdir/pybtex-$pkgver/build/lib" export PYTHONPATH="$srcdir/pybtex-$pkgver/build/lib" nosetests It works better, but a lot of tests fail because they can't load some stuff from the package: https://paste.aliens-lyon.fr/vam The same tests with python2 work fine in the source dir, and fail in the build/lib dir. So there's something strange going on with the build dir.
For certain upstream test setups there won't be much you can do other then PYTHONPATH but as mentioned with a build function it should always work. You can give python-pytest-runner a go, it does proper resolution while running "python setup.py test" but requires certain ways the whole test suites are wired.
It looks like this will only work for packages that use pytest? Thanks, Baptiste
For testing with not installed python modules, invoking setup.py commands are often preferable and addresses both PYTHONPATH and 2to3. For nosetests: Use "python setup.py nosetests" instead. For pytest: Use "python setup.py pytest" instead. Note that "python-pytest-runner" needs to be in checkdepends. There are additional common hacks if the tests need to invoke an entry points or requires the versioned distribution object. Either: 1) Installing it in a temporary place and hack PYTHONPATH/PATH. 2) Create a sitecustomize.py and add the build dir to site. (See python2-faulthandler for an example) And finally if above still didn't solve it, you may choose to skip the problematic part of the tests or use "heavier" workarounds like a venv. -- Regards, Felix Yan
On Wed, Apr 18, 2018 at 11:19:33PM +0800, Felix Yan via arch-dev-public wrote:
For testing with not installed python modules, invoking setup.py commands are often preferable and addresses both PYTHONPATH and 2to3.
For nosetests: Use "python setup.py nosetests" instead.
For pytest: Use "python setup.py pytest" instead. Note that "python-pytest-runner" needs to be in checkdepends.
There are additional common hacks if the tests need to invoke an entry points or requires the versioned distribution object. Either:
1) Installing it in a temporary place and hack PYTHONPATH/PATH. 2) Create a sitecustomize.py and add the build dir to site. (See python2-faulthandler for an example)
And finally if above still didn't solve it, you may choose to skip the problematic part of the tests or use "heavier" workarounds like a venv.
I think we should be better at documenting these things. This is super usefull to know about and not something that is obvious to everyone. I'll work on the package guideline pages for Python and Golang whenever I have some spare energy left and will be sure to include this. -- Morten Linderud PGP: 9C02FF419FECBE16
participants (4)
-
Baptiste Jonglez
-
Felix Yan
-
Levente Polyak
-
Morten Linderud