[pacman-dev] [PATCH 0/7] integrate test suite with automake

Allan McRae allan at archlinux.org
Mon Aug 5 00:18:13 EDT 2013

On 05/08/13 14:16, Andrew Gregory wrote:
> On 08/05/13 at 10:52am, Allan McRae wrote:
>> On 02/08/13 22:34, Andrew Gregory wrote:
>>> This patchset converts the output of all of our tests to tap [1] and fully
>>> integrates them with automake so that tests can be run in parallel with `make
>>> check`.  The test suite may also be run with other test harnesses such as
>>> perl's prove which can do such interesting things as remember which tests
>>> failed and run only those on subsequent invocations.  The documentation for
>>> integrating tests is here [2].
>>> [1] http://podwiki.hexten.net/TAP/TAP.html?page=TAP
>>> [2] http://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html
>> Have you any ideas on how to fix the "unexpected" pass on the time test
>> for x86_64 to not have the test suite return non-zero?  I believe this
>> is essential.
>> Allan
> I think that "unexpected" passes are rightly considered failures.  The test
> should reflect what we actually expect to happen.  We should either update the
> test so that it succeeds or fails uniformly on all systems or set expectfailure
> only on systems where we actually expect it to fail.  Personally, I would
> prefer that the test use the maximum values that the testing system could be
> expected to support and unset expectfailure, but the easier solution is to just
> set expectfailure only on 32 bit systems.

Setting expected failure on 32bit systems would actually be my preferred
solution in this case.  Can our test suite handle that?

More information about the pacman-dev mailing list