[pacman-dev] [PATCH] makepkg: parallelize integrity checks

[Dan McGee]

On Fri, Apr 22, 2011 at 3:35 PM, Allan McRae <allan at archlinux.org> wrote:
> On 23/04/11 02:22, Dan McGee wrote:
>>
>> On Fri, Apr 22, 2011 at 11:16 AM, Dan McGee<dan at archlinux.org>  wrote:
>>>
>>> This enables parallel integrity checks in makepkg within a given family
>>> of integrity sums. Subshell jobs for each source file are kicked off and
>>> run in parallel, and then we wait for each of them in turn to complete
>>> and print the same information as before.
>>>
>>> Note that programming sense says this loop should be done differently
>>> for filesystem access reasons; doing all checks for a given file would
>>> make more sense rather than running through the filelist multiple times.
>>> However, that would be a very different patch than what this is trying
>>> to accomplish.
>>
>> Two other things worth mentioning:
>> 1. We don't limit the number of jobs here in any way, so in theory you
>> could have a lot...
>> 2. Applying this to source file extraction would be the next logical
>> step, as that is a much slower part than this, and we might as well
>> use more cores since all extraction programs we use are
>> single-threaded.
>>
>
> I'd be very careful about applying this to extraction.  The main package I
> maintain where this would be useful is gcc, but there the source files gave
> lots of overlapping directories and I would want to be sure no race
> condition occurred in extracting them.

Wait, seriously? So both archive A and archive B have a file that
extracts to the same path? Is it so insane that each archive doesn't
end up in it's own folder anyway?

-Dan