TAP testing in Python

[Filipe Laíns]

On Sat, 2020-03-21 at 16:51 +0100, Frédéric Mangano-Tarumi wrote:
> In order to write tests in Python, we first need to make one important
> choice: the test framework. Our main constraint is the output format,
> which should be TAP to run along with the existing scripts.
> 
> My favorite candidate is pycotap, which holds in a 157-line
> no-dependency module that we could put in our tests directory, pretty
> much like we did for `test/sharness.sh`. It is implemented as a producer
> for Python’s standard unittest module. https://el-tramo.be/pycotap/
> 
> Its main opponent is tappy, a set of tools for working with TAP. It
> integrates with unittest, pytest and nose. From the documentation, my
> impression is that it expects to be invoked with nose or pytest’s
> harness, which would be inconsistent with usual TAP test invocation
> consisting of calling scripts directly and relying on the shebang.
> https://tappy.readthedocs.io/en/latest/producers.html
> 
> More alternatives at
> https://tappy.readthedocs.io/en/latest/alternatives.html
> 
> Unless we plan to use pytest or nose over unittest, I suggest we use use
> pycotap. Except for a couple of lines, tests written with pycotap are
> standard unittest tests, which nose and pytest can collect results from
> too. If our needs evolve, we could easily upgrade to another framework.

Why should we use unittest over pytest? pytest is easier to read and
write, more extensible/has more integrations and has a larger userbase.
The only caveat is that people would have to fo a quick read on
fixtures before they start contributing.

Choosing pycotap over tappy/pytest-tap would make sense if we already
had a codebase written in unittest, but we don't.

Lukas, what do you think?

Cheers,
Filipe Laíns
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <https://lists.archlinux.org/pipermail/aur-dev/attachments/20200321/a4f8eafb/attachment.sig>