[pacman-dev] [PATCH] Support parallel download with xfercommand

[lesto fante]

> So, you are just passing the full list of files to download to a
> download script.  Downloads are not managed by pacman at all?

Exactly, my understanding is that with XferCommand we delegate an
external command to manage the downloads.
The advantage of having a dedicated wait command/parameter for the
last packet to download is that this final command can act as a
collection of information.

The only reason I do not call XferCommand with a list of all the
packages, server, and other options like ParalleDownalods as parameter
is because i fear to hit the parameter limit on some supported
OS/kernel I am not aware of;
but i feel maybe this is the best option, give to the download program
all he has to known.

The idea of having a single command or a dedicate wait
command/parameter imho is very important as this will help to have a
decent output of the external program progress, as it will be managed
by the program itself

> Just add three more lines to your script:
>
> pacman -Sy
> pacman -Sup --noconfirm
> <downloads here>
> pacman -Su

Yes, this is more or less what my actual script is doing.
I don't think it is the right solution as it became a multistage
process (error prone) or a pacman wrapper (don't like too much as this
is adding only a small modification, and will interfere with all the
other wrappers and possibly command line options) .

> I don't see the point of implementing parallel XferCommand like that
> within pacman at all.

Convenience and integration with other wrappers.

> Pacman currently monitors a single download in a portable way.  I see no
> reason it could not monitor more than one.  Then it could use
> ParallelDownloads and provided some consistency across download methods.

because I am not sure if tracking PID is portable and would require a
deeper modification of `fetchcb`, probably to return a PID handle.
Pacman could track every process alive/dead, but would not know any
other information like internal prograss.
If you think this is the best way, I will implement it.

> It does not...  I'd expect it would after an addition to XferCommand to
> support parallel downloads.

ok, so i can implement the tracking of the PID, but before writing any
more code I want to make sure this solution is acceptable;
and if it is, any implementation suggestion is welcome, if not, what
you think is the solution.

so to recap:

- solution 1 -
XferCommand called multiple times, non blocking, and a final
XferCommand with special parameter/XferLockCommand to wait for output.
We trust XferCommand to start only ParallelDownloads download.

- solution 2 -
We call XferCommand ParallelDownloads times, and wait for process to
complete before calling again

- solution 3 -
XferParallelCommand is added, it will be called with a list of all
packages, servers and options like ParallelDownloads.
We trust XferCommand to start only ParallelDownloads download.