[pacman-dev] [PATCH] Added mirror support to makepkg's source array.

[Allan McRae]

On 16/11/13 07:53, Ido Rosen wrote:
> Allan,
> 
> On Fri, Nov 15, 2013 at 12:06 PM, Allan McRae <allan at archlinux.org> wrote:
> 
>> On 01/11/13 20:00, Florian Pritz wrote:
>>> From: Ido Rosen <code at idorosen.com>
>>>
>>> To specify multiple mirrors, simply add multiple files to the source
>> array
>>> that have the same downloaded filename, e.g.:
>>>
>>> source=("file.tar.gz::http://mirror1.example.com/file.tar.gz"
>>>         "file.tar.gz::http://mirror2.example.com/file.tar.gz")
>>>
>>> ...makepkg will try them all, and if all fail, will abort.  This also
>> applies
>>> to VCS repositories and local files references in the source array, so
>> for
>>> example:
>>>
>>> source=("git+https://github.com/torvalds/linux.git
>>>         "git+
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git")
>>>
>>> ...will allow cloning the git repository from GitHub if kernel.org is
>> down or
>>> fails.
>>>
>>> Signed-off-by: Florian Pritz <bluewind at xinu.at>
>>
>>
>> We discussed this on IRC.   I really dislike the multiple source lines
>> "downloading" the same file approach.
>>
>> An idea is doing:
>>
>> source=("mirror://file.tar.gz")
>> mirror=("http://foo.com/" "http://bar.com/")
>>
>> makepkg would replace "mirror://" in the source line with a value from
>> the "mirror" array and try downloading from each one until success or
>> total failure.
>>
>> To allow multiple sources to have different mirrors, I am proposing we
>> detect an unknown protocol in the url and look for the corresponding
>> array.  e.g. we could have source=("sourceforge://file.tar.gz") and look
>> for the "sourceforge" array.
>>
>> Comments or other ideas?
>>
> 
> Would this then implement the mirror:// or sourceforge:// protocol as a new
> DLAGENT?  

No.

My proposal had nothing to do with DLAGENT.  We call get_protocol() and
see an unrecognised protocol.  We see that the specified "protocol"
corresponds to an array in the PKGBUILD.  So we know we are dealing with
a mirror source list.

> It seems to me that DLAGENTs should serve a different (but also
> important) purpose and should be conceptually separate - I would avoid
> overloading the scheme/protocol field for this case.  Or am I
> misunderstanding your suggestion?  Also, would this mean adding a new
> variable for each mirrored site?  That seems less flexible than Jerome's
> formatting that I implemented.
> 
> I really like the simplicity of the original patch, and it doesn't break
> existing PKGBUILDs AFAICT...  If you're vetoing that, here is a compromise:

I'm definitely vetoing the original proposal - having multiple source
lines for one file is really ugly and I did not consider it simple at all.

> we could create a new DLAGENT called mirrorlist that allows specifying the
> variable name in which the mirror list is stored for that file, and an
> optional suffix:
>
> source=("destinationfile.tar.gz::mirrorlist://_variablename/suffix/file.tar.gz")
> _variablename=("http://foo.com/" "http://bar.com/")
> # ... would download from http://foo.com/suffix/file.tar.gz, then try the
> same for bar.com upon failure, and put it in destinationfile.tar.gz.

That is essentially what I proposed.  Having a constant protocol string
makes sense.  I'd use "mirror://" because it is shorter.   Implement
this one...

> source=("destinationfile.tar.gz::mirrorlist://_variablename")
> _variablename=("http://foo.com/suffix/file.tar.gz" "
> http://bar.com/suffix/file.tar.gz")
> # ... same as above.
> # (destinationfile.tar.gz:: is optional as usual...)

... and not that one.   We should be able to extract the filename from
the source URL (if the file name is not given in the prefix).

Allan