On Wed, 15 Apr 2020 at 08:44:44, Frédéric Mangano-Tarumi wrote:
Lukas Fleischer [2020-04-15 08:08:08 -0400]
Our test suite already uses a separate configuration which is auto-generated in test/setup.sh, so it is already self-contained in that sense; we only need to update aur_location there.
I\u2019m hesitant about using setup.sh because it does more than we need. It aims at being universal but that means it prepares an environment suitable for all tests, but run for every test. That\u2019s quite a waste of resources. If it\u2019s just for the configuration file, we could make a config.test.defaults and make setup.sh use it except for the few variables it needs to override.
The new tests are going to require mock data in the database, too, right? Do you intend to populate the database separately for each test? Note that many tests require a common basic set of data, such as user accounts and packages. If we want different initialization procedures for each test, we should at least have some way of including this common data set. The setup script only takes a couple of milliseconds, so having to execute it multiple times doesn't bother me too much. If you have any suggestions on how to reduce some of the duplicate steps, feel free to post them here, though.
Still, I wonder if a skip is that bad an option. It\u2019s true it\u2019s not as explicit as a fail, but a skipped test is a common convention for the test suite to indicate it\u2019s missing something, isn\u2019t it?
Yeah, I guess you are right, especially since there probably is an option to make skipped tests fail the test suite. Lukas