[pacman-dev] makepkg test suite

[Dan McGee]

On Mon, Oct 19, 2009 at 7:23 AM, Allan McRae <allan at archlinux.org> wrote:
> Dan McGee wrote:
>>
>> On Sat, Aug 8, 2009 at 1:04 AM, Allan McRae <allan at archlinux.org> wrote:
>>
>>>
>>> Hi all,
>>>
>>> I am starting to write a test suite for makepkg.  Hopefully this will
>>> make
>>> it more difficult to break things in the future...
>>>
>>> I have written a brief outline of how I have started implementing this
>>> here:
>>> http://wiki.archlinux.org/index.php/User:Allan/Makepkg_Test_Suite
>>>
>>> I realize that most issues with the implementation will be discovered as
>>> it
>>> goes on but any comments at this stage would be useful.
>>>
>>
>> Any forward progress on this? Don't want to rush you, but haven't
>> heard anything lately on this front.
>>
>> On a side note, I wonder if this is one of those "I'm new to pacman
>> development how can I help" type projects- obviously it is a big task,
>> but having a few people hacking and getting this working would be
>> sweet.
>>
>
> I have written the .PKGINFO parser (posted on this list a while ago) and
> made a rough start on the PKGBUILD parser.  And I have one test case
> (absolutely minimum PKGBUILD).
>
> I'm still not sure what the test suite should look like in terms of the way
> to convey information about the test (see the example I gave on the wiki
> page).  And since I have received a total of zero comments on that,
> everything has stalled in favour of me fixing other things.
>
> Allan

One naive thought is to just have a PKGBUILD+ format. This would
entail a semi-normal PKGBUILD, that also had some sort of
makepkg_test() function defined. The test harness would source the
PKGBUILD (and thus get a side benefit of knowing the pkgname, pkgver,
etc. variables) and then would invoke the makepkg_test() function,
which would be responsible for invoking makepkg with whatever options,
and verifying the output code, produced files, etc. as necessary.

Two thoughts:
* There would be a defined $MAKEPKG_OPTS that would always get
included in the makepkg calls made by the test function, to ensure
things like the config file location were set.
* Each test PKGBUILD could have multiple makepkg_test_XXX() functions
defined; we could find all of these via $(declare -F) or something.
* The test harness could predefine some super-helpful check functions
so we aren't reinventing the wheel

-Dan

pkgname=foobar
pkgver=1.0
pkgrel=1
pkgdesc="Basic tests"
arch=('any')

build() {
  touch $srcdir/foobar
}

package() {
  cd $pkgdir
  mkdir -p usr/local/
  touch usr/local/omgtest
  cp $srcdir/foobar usr/local/foobar
}

makepkg_test_files_exist() {
  makepkg $MAKEPKG_OPTS
  file_exists usr/local/omgtest
  file_exists usr/local/foobar
}