[pacman-dev] [PATCH] makepkg: parallelize integrity checks

[Allan McRae]

On 23/04/11 02:22, Dan McGee wrote:
> On Fri, Apr 22, 2011 at 11:16 AM, Dan McGee<dan at archlinux.org>  wrote:
>> This enables parallel integrity checks in makepkg within a given family
>> of integrity sums. Subshell jobs for each source file are kicked off and
>> run in parallel, and then we wait for each of them in turn to complete
>> and print the same information as before.
>>
>> Note that programming sense says this loop should be done differently
>> for filesystem access reasons; doing all checks for a given file would
>> make more sense rather than running through the filelist multiple times.
>> However, that would be a very different patch than what this is trying
>> to accomplish.
>
> Two other things worth mentioning:
> 1. We don't limit the number of jobs here in any way, so in theory you
> could have a lot...
> 2. Applying this to source file extraction would be the next logical
> step, as that is a much slower part than this, and we might as well
> use more cores since all extraction programs we use are
> single-threaded.
>

I'd be very careful about applying this to extraction.  The main package 
I maintain where this would be useful is gcc, but there the source files 
gave lots of overlapping directories and I would want to be sure no race 
condition occurred in extracting them.

Allan