[pacman-dev] [PATCH] [RFC] makepkg: extract sources in parallel

[Allan McRae]

On 23/04/11 08:04, Dan McGee wrote:
> This one could definitely benefit from limiting the number of parallel
> threads, especially when dealing with the GCC PKGBUILD. Definitely a lot
> of contention and times varied from twice as fast to a few seconds
> slower, depending on if the cache decided it needed to flush out some
> data. Limiting the number of threads to the number of CPUs would
> probably go a long way to resolving some of this contention for IO
> bandwidth.
>
> Signed-off-by: Dan McGee<dan at archlinux.org>

Took this for a spin and it does make a difference for the GCC PKGBUILD. 
  Reduced total extraction time from 35sec to 25sec on my laptop.  I was 
actually surprised it made that much difference given I figured this 
would be more disk speed bound that cpu bound...

My main concern is still what happens if two processes try to extract 
the same directory at the same time.  I guess such an occurrence would 
be very rare, and perhaps bstar actually would gracefully handle this, 
but it is something to consider.

Also, the extraction time actually seemed slower despite not being so 
due to the output.  On the non-parallel version, you get a visual cue on 
how far through the extraction process you are (in terms of number of 
files extracted), but with the parallel extraction, all "Extracting" 
output is printed at once and then there is a big wait.  I guess that 
could be adjusted.

So overall, I am +/-0 on this...

Allan