[pacman-dev] [PATCH 0/7] integrate test suite with automake
andrew.gregory.8 at gmail.com
Mon Aug 5 00:16:34 EDT 2013
On 08/05/13 at 10:52am, Allan McRae wrote:
> On 02/08/13 22:34, Andrew Gregory wrote:
> > This patchset converts the output of all of our tests to tap  and fully
> > integrates them with automake so that tests can be run in parallel with `make
> > check`. The test suite may also be run with other test harnesses such as
> > perl's prove which can do such interesting things as remember which tests
> > failed and run only those on subsequent invocations. The documentation for
> > integrating tests is here .
> >  http://podwiki.hexten.net/TAP/TAP.html?page=TAP
> >  http://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html
> Have you any ideas on how to fix the "unexpected" pass on the time test
> for x86_64 to not have the test suite return non-zero? I believe this
> is essential.
I think that "unexpected" passes are rightly considered failures. The test
should reflect what we actually expect to happen. We should either update the
test so that it succeeds or fails uniformly on all systems or set expectfailure
only on systems where we actually expect it to fail. Personally, I would
prefer that the test use the maximum values that the testing system could be
expected to support and unset expectfailure, but the easier solution is to just
set expectfailure only on 32 bit systems.
More information about the pacman-dev