[arch-dev-public] Package checking

[Allan McRae]

On 02/06/10 23:02, Jan de Groot wrote:
> A lot of projects include a "check" target in their makefiles. Right
> now, we don't use these checks and just upload a package after compiling
> it, sometimes even without a runtime test.
>
> I would propose to add checks to important packages. Examples include
> glibc, glib2, cairo, MySQL, Mozilla stuff and more. Running make check
> can result in better package quality. Upstream usually runs these tests
> before releasing, but they don't test every possible combinations of
> packages that a distribution ships.
>
> To implement this, I would propose to have some kind of check function
> inside the PKGBUILD, which can be called by makepkg if "check" is in the
> options array. Other option is to include make check inside the build()
> function, but that will make it non-optional.
>
> This package checking stuff could get implemented in an automatic build
> bot in the future. Not only will that build bot detect packages which
> don't build anymore, but it will also detect regressions caused by
> package updates. A package like xulrunner or firefox could pass the
> check without errors on package upload, but it could break after
> updating cairo or gtk2 for example.
>

I agree and have filed a bug a while back 
(http://bugs.archlinux.org/task/15145) - I am still thinking about how 
to invoke the check function (all the time if it is there, global config 
option, flag?), but the same consideration needs to be address for 
(e.g.) splitting debug symbols into a separate package.

I do run the test suites within the build function for the numeric 
libraries used by gcc (gmp/mpfr/ppl/cloog-ppl/libmpc).  For the 
toolchain, I usually manually run the test suite in the build directory 
(requires stopping makechrootpkg from deleting it...).  The gcc test 
suite requires some packages not in [core] so I did not want to add them 
as makedepends.

Allan