[pacman-dev] A makepkg test suite

[Ashley Whetter]

On 12 July 2013 20:00, Ashley Whetter <awhetter.2011 at my.bristol.ac.uk> wrote:
> I've recently come across issue FS#15645 ([makepkg] needs a test
> suite) and I'd like to have a bash at implementing it. It seems like a
> good idea to get peoples ideas and thoughts first so what do you
> think?
>
> My first reaction is to do something similar to the pacman tests and
> just write a custom framework in python. Also have the output the same
> as the pacman tests.
>
> I think keeping the idea an environment (like in the pacman tests)
> seems a bit heavy handed because the only configuration done is in
> makepkg.conf and we can probably fake a pacman environment using a
> stub pacman. Or maybe we could utilise the pacman test environments!
>
>
> Unit testing should be easy enough because we can just call specific
> functions in makepkg and check the output.
>
> For functional testing I'm not really sure.
>
> As for acceptance testing we can just put different flag combinations
> in different test environments, and check the return code. Check the
> .PKGINFO. Maybe that files were put where they should be in pkg and
> src, as well as in the package itself.
> Have some PKGBUILDs that fail for various reasons like syntax, a
> dependency package doesn't exist. Make sure missing
> {,make,check}dependencies are installed.
>
>
> Any suggestions or ideas?
> Ashley

Thanks for everyone's suggestions. Sorry if my initial proposal seemed
a bit ropey. I've never done bash scripting on this scale before, and
I'm not as experienced with testing as I'd like. So I appreciate
everyone's patience.

I've started taking a look at splitting up makepkg into libraries.
I've created a (pretty messy) dependency graph that shows what
functions call what, and my initial idea of what functions go in what
libraries (http://underrun.org/~gadget/makepkg_resolved_deps.png).
Is it worth me creating and emailing a plain text list for the benefit
of the mailing list archives?

There are 5 main libraries:
makepkg: For "makepkg specific" stuff.
PKGBUILD: For PKGBUILD specific stuff.
sources: For downloading and extracting sources.
deps: Handles dependencies and pacman stuff.
utils: For general useful functions.

On the graph there's also 'misc' and 'Messages'.
Misc is just general makepkg code that isn't currently in a function
and depends on all other libraries. It also calls any functions that
don't currently have incoming dependencies, as well as some others.
Messages contains just the 5 functions that print messages and
everything depends on it.

I'm bit a unsure about what to do with a couple of functions though,
and I've marked these as orange on the graph.
check_option feels to me like it does both 'makepkg' and 'PKGBUILD'
specific stuff, and shouldn't really go in utils. Also run_function
depends on it.
I think get_full_version is a PKGBUILD specific function, but
run_function (in utils) depends on it to create the log file. Maybe
logging stuff could be split into another library?

Other points:
get_downloadclient and get_url could be moved to PKGBUILD.
extract_sources to check_build_status is the only dependency between
'sources' and 'makepkg'. I feel like it's worth getting rid of this
dependency somehow?

I was a bit unclear as to where some functions should go, so if
anything seems odd then let me know.

Thanks,
Ashley