On Fri, 21 Feb 2020 at 23:41:06, Frédéric Mangano-Tarumi wrote:
prove comes with its conventions, which are from the original TAP. First, it expects test programs to end with `.t`, which is restrictive enough to exclude non-test files, but doesn\u2019t bind to any specific language since the shebang is used to know how to run each test. This is by the way a great opportunity to start writing tests in Python too.
The current tests mostly execute commands and compare their outputs, and I don't see much benefit in rewriting them in Python. That being said, I think we could benefit from additional unit tests.
Another convention is to name the test directory `t` and to call prove from the root of the project. This one is harder to use since all the existing tests assume they\u2019re run from the `test` directory. A simple cd at the beginning of every script is probably a good solution, but let\u2019s leave that problem open for now. While we can run prove from the `test` directory, running tests from the root has the notable advantage of letting us import aurweb Python modules without hacking PYTHONPATH.
Here are various suggestions:
1. Change the extension of all the test programs to `.t`.
Sounds good.
2. Create `test\README` to tell developers about TAP and prove.
Great idea!
3. Make `make check` call prove.
I like the idea of being able to use prove but I'd prefer still having a way to run tests without having prove installed. How about adding a "prove" target to the Makefile instead of changing `make check`?
4. Delete the `make t1234-xxx.sh` rules. 5. Delete the whole Makefile altogether.
Why? Those are small, super low maintenance, and provide the benefit of being able to easily run the test suite without prove.
6. Run the test programs from the project root. 7. Rename the `test` directory to `t`.
Sounds good.
1 to 4 are cheap and have a significant added value in my opinion. 5 to 7 less so, but I think that if we want to use TAP, we should stick to its conventions.
I agree with sticking to TAP conventions but it should be easy to keep almost everything working as-is at the same time without any extra work. Thanks for bringing this up! Lukas