[pacman-dev] [PATCH v2 1/3] bacman: allow for parallel packaging

[Gordian Edenhofer]

On Tue, 2016-08-30 at 20:30 +1000, Allan McRae wrote:
> On 15/08/16 06:39, Gordian Edenhofer wrote:
> > 
> > * move the actual assembly process into its own function
> > * allow for packaging multiple packages with one command
> > * handle SIGHUP SIGINT SIGTERM and remove working dirs accordingly
> > * add some comments
> > 
> > Signed-off-by: Gordian Edenhofer <gordian.edenhofer at gmail.com>
> > ---
> > * Clean up working directories with force on abort
> > * Sleep for 100ms in between parallelization work
> 
> Why sleep?

Sleep is invoked to reduce the CPU usage caused by busy-waiting.
The approach for parallelization is straight forward and simple. It
checks every n seconds whether enough jobs are running and in case not
it starts a new one. Since this is not an elegant way to do it, I added
a note which incorporates Dave Reisner hint to use wait -n at some
point in the future when bash 4.3 is available at every system.

> Two patches are needed:
> 
> 1) allow bacman to take multiple package arguments
> 2) parallelize...

I will submit a second patch very soon which will address your
complaints concerning my commit division.

> The second probably would not be accepted...

I urge you to reconsider. Parallelization increases the speed of this
tool immensely. To proof my point I made some test on an Intel NUC with
an i5-4250U dual-core CPU. The results [1] clearly show that when using
all available four threads the time needed to assemble various chunks
of packages is sliced in half.
The graphs show the time required for a group of packages to be
recreated with bacman. To show the potential of parallelization I
chose: base + base-devel, base + base-devel + gnome and finally I
assembled all packages installed on the system. Packages contained in
multiple groups were only assembled once. All chosen package groups
were recreated using 1-6 jobs.
The respective measured values and code can be seen here [2].

Best Regards,
Gordian Edenhofer

[1] http://edh.ddns.net/pacman_ml_bacman_benchmarks/bacman:%20simple%20
benchmark.svg
[2] http://edh.ddns.net/pacman_ml_bacman_benchmarks/bacman:%20simple%20
benchmark.R.txt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: This is a digitally signed message part
URL: <https://lists.archlinux.org/pipermail/pacman-dev/attachments/20160831/0e24d611/attachment.asc>