[pacman-dev] A makepkg test suite

[Allan McRae]

On 28/07/13 06:57, Ashley Whetter wrote:
> On 12 July 2013 20:00, Ashley Whetter <awhetter.2011 at my.bristol.ac.uk> wrote:
>> I've recently come across issue FS#15645 ([makepkg] needs a test
>> suite) and I'd like to have a bash at implementing it. It seems like a
>> good idea to get peoples ideas and thoughts first so what do you
>> think?
>>
>> My first reaction is to do something similar to the pacman tests and
>> just write a custom framework in python. Also have the output the same
>> as the pacman tests.
>>
>> I think keeping the idea an environment (like in the pacman tests)
>> seems a bit heavy handed because the only configuration done is in
>> makepkg.conf and we can probably fake a pacman environment using a
>> stub pacman. Or maybe we could utilise the pacman test environments!
>>
>>
>> Unit testing should be easy enough because we can just call specific
>> functions in makepkg and check the output.
>>
>> For functional testing I'm not really sure.
>>
>> As for acceptance testing we can just put different flag combinations
>> in different test environments, and check the return code. Check the
>> .PKGINFO. Maybe that files were put where they should be in pkg and
>> src, as well as in the package itself.
>> Have some PKGBUILDs that fail for various reasons like syntax, a
>> dependency package doesn't exist. Make sure missing
>> {,make,check}dependencies are installed.
>>
>>
>> Any suggestions or ideas?
>> Ashley
> 
> Thanks for everyone's suggestions. Sorry if my initial proposal seemed
> a bit ropey. I've never done bash scripting on this scale before, and
> I'm not as experienced with testing as I'd like. So I appreciate
> everyone's patience.
> 
> I've started taking a look at splitting up makepkg into libraries.
> I've created a (pretty messy) dependency graph that shows what
> functions call what, and my initial idea of what functions go in what
> libraries (http://underrun.org/~gadget/makepkg_resolved_deps.png).
> Is it worth me creating and emailing a plain text list for the benefit
> of the mailing list archives?
> 
> There are 5 main libraries:
> makepkg: For "makepkg specific" stuff.
> PKGBUILD: For PKGBUILD specific stuff.
> sources: For downloading and extracting sources.
> deps: Handles dependencies and pacman stuff.
> utils: For general useful functions.
> 
> On the graph there's also 'misc' and 'Messages'.
> Misc is just general makepkg code that isn't currently in a function
> and depends on all other libraries. It also calls any functions that
> don't currently have incoming dependencies, as well as some others.
> Messages contains just the 5 functions that print messages and
> everything depends on it.
> 
> I'm bit a unsure about what to do with a couple of functions though,
> and I've marked these as orange on the graph.
> check_option feels to me like it does both 'makepkg' and 'PKGBUILD'
> specific stuff, and shouldn't really go in utils. Also run_function
> depends on it.
> I think get_full_version is a PKGBUILD specific function, but
> run_function (in utils) depends on it to create the log file. Maybe
> logging stuff could be split into another library?
> 
> Other points:
> get_downloadclient and get_url could be moved to PKGBUILD.
> extract_sources to check_build_status is the only dependency between
> 'sources' and 'makepkg'. I feel like it's worth getting rid of this
> dependency somehow?
> 
> I was a bit unclear as to where some functions should go, so if
> anything seems odd then let me know.
> 

The things I think make the most sense to separate out the source
downloading/extracting functions first because they are a clear
grouping.  The rest can be considered later once we get the initial
steps completed.

Allan