[aur-general] Exact purpose of check()
Hi, There is a bit of debate at the borgmatic package ( https://aur.archlinux.org/packages/borgmatic) about what check() should do. The upstream borgmatic project uses tox to execute its tests. tox creates an isolated python virtualenv with the correct dependency versions and executes tests in there. The original maintainer thinks check() should not use tox so that tests can be run against the system installed dependencies. But this is not so easy to do, an attempt was made by installing the python package into an isolated directory just for tests, but even then it seems to conflict with an existing borgmatic installation. Another way of seeing things is that check() should just run tests the way it is intended by upstream, it is for testing the build artifacts are correct, not for testing it will run correctly on the system where it is built. By this logic check() should just run tox, and correct dependency versions can be enforced by using version ranges in "depends". Any advice on this ? Cheers, Julien
On 1/2/19 9:44 AM, Julien Nicoulaud via aur-general wrote:
Hi,
There is a bit of debate at the borgmatic package ( https://aur.archlinux.org/packages/borgmatic) about what check() should do. The upstream borgmatic project uses tox to execute its tests. tox creates an isolated python virtualenv with the correct dependency versions and executes tests in there.
The original maintainer thinks check() should not use tox so that tests can be run against the system installed dependencies. But this is not so easy to do, an attempt was made by installing the python package into an isolated directory just for tests, but even then it seems to conflict with an existing borgmatic installation.
Another way of seeing things is that check() should just run tests the way it is intended by upstream, it is for testing the build artifacts are correct, not for testing it will run correctly on the system where it is built. By this logic check() should just run tox, and correct dependency versions can be enforced by using version ranges in "depends".
Any advice on this ?
1) But upstream already runs tox in order to check that the source code is well-formed. 2) The PKGBUILD check() function is part of the packaging process for packaging software with Arch Linux, it is not part of the upstream development process for creating generic, cross-platform, PyPI uploads. -- Eli Schwartz Bug Wrangler and Trusted User
Tox should never ever be used for check() exactly for the reasons your pointed out. It is supposed to check the build/package works without regression in the environment it will run in. Running the test against system libraries that are later after install used by the runtime is the way check() should behave otherwise it's pointless in the context of checking the package.
On Wed, Jan 02, 2019 at 03:44:46PM +0100, Julien Nicoulaud via aur-general wrote:
There is a bit of debate at the borgmatic package ( https://aur.archlinux.org/packages/borgmatic) about what check() should do. The upstream borgmatic project uses tox to execute its tests. tox creates an isolated python virtualenv with the correct dependency versions and executes tests in there.
Don't use tox. We are checking of the dependecies we correct as well. Installing deps with tox from pypi defeats this purpose.
The original maintainer thinks check() should not use tox so that tests can be run against the system installed dependencies. But this is not so easy to do, an attempt was made by installing the python package into an isolated directory just for tests, but even then it seems to conflict with an existing borgmatic installation.
The original maintainer is correct. This is also why you should be running clean chroots. But if it's conflicting you are doing something wrong
Another way of seeing things is that check() should just run tests the way it is intended by upstream, it is for testing the build artifacts are correct, not for testing it will run correctly on the system where it is built. By this logic check() should just run tox, and correct dependency versions can be enforced by using version ranges in "depends".
Using tox is for convenience, but if tox is using nose or pytest then use that. Don't use version ranges to check if dependencies are correct, run the tests. We shouldn't download anything after dependencies and sources have been fetched. -- Morten Linderud PGP: 9C02FF419FECBE16
We can also add a more explicit warning in the package guidelines. https://wiki.archlinux.org/index.php/Python_package_guidelines#Check -- Morten Linderud PGP: 9C02FF419FECBE16
On 1/2/19 10:02 AM, Morten Linderud via aur-general wrote:
We can also add a more explicit warning in the package guidelines.
https://wiki.archlinux.org/index.php/Python_package_guidelines#Check
Let's do this anyway: https://wiki.archlinux.org/index.php?title=Python_package_guidelines&diff=56... -- Eli Schwartz Bug Wrangler and Trusted User
participants (4)
-
Eli Schwartz
-
Julien Nicoulaud
-
Levente Polyak
-
Morten Linderud