[pacman-dev] [PATCH] makepkg: Support for resuming source downloads
Hi! The inability of makepkg to resume broken source downloads annoyed me pretty much, so here's a patch attached. ;-) Before applying the patch: $ makepkg -o ==> Making package: audacious-itouch-control 0.1.1-2 (неділя, 10 лютого 2008 17:23:53 +0200) ==> Retrieving Sources... -> Downloading itouch-control-0.1.1.tar.gz... [ . . . skipped . . . ] 13% [====> ] 44 560 24,7K/s ==> ERROR: Aborted by user! Exiting... [roman@home audacious-itouch-control]$ makepkg -o ==> Making package: audacious-itouch-control 0.1.1-2 (неділя, 10 лютого 2008 17:24:04 +0200) ==> Retrieving Sources... -> Using cached copy of itouch-control-0.1.1.tar.gz ==> Validating source files with md5sums... itouch-control-0.1.1.tar.gz ... FAILED ==> ERROR: One or more files did not pass the validity check! This is because makepkg doesn't know if file is complete, or not. But pacman is able to resume downloads. This inspired me to do a very similar implementation in makepkg. After applying the patch: [roman@home audacious-itouch-control]$ makepkg -o ==> Making package: audacious-itouch-control 0.1.1-2 (неділя, 10 лютого 2008 17:28:22 +0200) ==> Retrieving Sources... -> Downloading itouch-control-0.1.1.tar.gz... [ . . . skipped . . . ] 12% [====> ] 41 672 24,2K/s ==> ERROR: Aborted by user! Exiting... [roman@home audacious-itouch-control]$ makepkg -o ==> Making package: audacious-itouch-control 0.1.1-2 (неділя, 10 лютого 2008 19:04:24 +0200) ==> Retrieving Sources... -> Downloading itouch-control-0.1.1.tar.gz... [ . . . skipped . . . ] 80% [+++++++========================> ] 259 593 --.-K/s in 16m 40s 2008-02-10 19:21:07 (197 B/s) - Read error at byte 259593/323774 (Connection timed out). Продовження спроб. --2008-02-10 19:21:08-- (спроба: 2) [ . . . skipped . . . ] 100%[++++++++++++++++++++++++++++++++=======>] 323 774 18,1K/s in 3,5s 2008-02-10 19:21:12 (18,1 KB/s) - `itouch-control-0.1.1.tar.gz.part' saved [323774/323774] ==> Validating source files with md5sums... itouch-control-0.1.1.tar.gz ... Passed ==> Extracting Sources... -> bsdtar -x -f itouch-control-0.1.1.tar.gz ==> Sources are ready. The patch also includes changes to man makepkg.conf, and makepkg.conf itself, because of introduced %o and %u pseudo-variables (same as in XferCommand). -- Roman Kyrylych (Роман Кирилич)
2008/2/10, Roman Kyrylych <roman.kyrylych@gmail.com>:
Hi!
The inability of makepkg to resume broken source downloads annoyed me pretty much, so here's a patch attached. ;-)
And with the help of Xavier I've remembered the bug that can be re-opened and closed as "Fixed" now. ;-) http://bugs.archlinux.org/task/9299 -- Roman Kyrylych (Роман Кирилич)
2008/2/11, Roman Kyrylych <roman.kyrylych@gmail.com>:
2008/2/10, Roman Kyrylych <roman.kyrylych@gmail.com>:
Hi!
The inability of makepkg to resume broken source downloads annoyed me pretty much, so here's a patch attached. ;-)
And with the help of Xavier I've remembered the bug that can be re-opened and closed as "Fixed" now. ;-) http://bugs.archlinux.org/task/9299
Previous patch didn't care about path with spaces, here is a fixed version which restores previous behaviour of putting paths/urls in quotes ("). -- Roman Kyrylych (Роман Кирилич)
On Feb 11, 2008 3:33 AM, Roman Kyrylych <roman.kyrylych@gmail.com> wrote:
Previous patch didn't care about path with spaces, here is a fixed version which restores previous behaviour of putting paths/urls in quotes (").
Can you please resend this as inline so we can comment on it easier? -Dan
2008/2/11, Dan McGee <dpmcgee@gmail.com>:
On Feb 11, 2008 3:33 AM, Roman Kyrylych <roman.kyrylych@gmail.com> wrote:
Previous patch didn't care about path with spaces, here is a fixed version which restores previous behaviour of putting paths/urls in quotes (").
Can you please resend this as inline so we can comment on it easier?
OK, I just had some bad experience with Gmail in past. From 1196a6b6a97c17027608d6c92a3df647deab114c Mon Sep 17 00:00:00 2001 From: Roman Kyrylych <roman@archlinux.org> Date: Sun, 10 Feb 2008 18:56:47 +0200 Subject: [PATCH] makepkg: Support for resuming source downloads download to *.part and remove the suffix if successful, thus md5sums will be checked only for complete files, and next time makepkg will resume downloading of partial file instead of complaining about invalid md5sums Signed-off-by: Roman Kyrylych <roman@archlinux.org> --- doc/makepkg.conf.5.txt | 3 +++ etc/makepkg.conf.in | 10 +++++----- scripts/makepkg.sh.in | 13 ++++++++++++- 3 files changed, 20 insertions(+), 6 deletions(-) diff --git a/doc/makepkg.conf.5.txt b/doc/makepkg.conf.5.txt index f6e4b38..ec72e29 100644 --- a/doc/makepkg.conf.5.txt +++ b/doc/makepkg.conf.5.txt @@ -37,6 +37,9 @@ Options well; the download URL is placed on the end of the command. This is more flexible than the former `FTPAGENT` variable, as any protocol can have a download agent. Several examples are provided in the default makepkg.conf. + All instances of `%u` will be replaced with the download URL. If present, + instances of `%o` will be replaced with the local filename, plus a ``.part'' + extension, which allows yo do file resumes properly. **CARCH=**"carch":: Specifies your computer architecture; possible values include such things diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in index d808270..e8ac6a1 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -8,11 +8,11 @@ # #-- The download utilities that makepkg should use to acquire sources # Format: 'protocol::agent' -DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3' - 'http::/usr/bin/wget -c -t 3 --waitretry=3' - 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate' - 'rsync::/usr/bin/rsync -z' - 'scp::/usr/bin/scp -C') +DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3 -O %o %u' + 'http::/usr/bin/wget -c -t 3 --waitretry=3 -O %o %u' + 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate -O %o %u' + 'rsync::/usr/bin/rsync -z %u %o' + 'scp::/usr/bin/scp -C %u %o') # Other common tools: # /usr/bin/snarf diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index cecda1d..27bf7fa 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -299,6 +299,16 @@ get_downloadclient() { echo "$agent" } +get_downloadcmd() { + local dlagent=$1 + local netfile=$2 + local file=$3 + + local dlcmd=$(echo $dlagent | sed "s|%o|\"$file.part\"|" | sed "s|%u|\"$netfile\"|") + + echo "$dlcmd" +} + check_deps() { [ $# -gt 0 ] || return @@ -476,12 +486,13 @@ download_sources() { msg2 "$(gettext "Downloading %s...")" "$file" # fix flyspray bug #3289 local ret=0 - $dlclient "$netfile" || ret=$? + $(get_downloadcmd "$dlclient" "$netfile" "$file") || ret=$? if [ $ret -gt 0 ]; then error "$(gettext "Failure while downloading %s")" "$file" plain "$(gettext "Aborting...")" exit 1 fi + mv -f "$SRCDEST/$file.part" "$SRCDEST/$file" cp -s --remove-destination "$SRCDEST/$file" "$srcdir/" done -- 1.5.3.7 -- Roman Kyrylych (Роман Кирилич)
On Mon, Feb 11, 2008 at 04:10:51PM +0200, Roman Kyrylych <roman.kyrylych@gmail.com> wrote:
+ extension, which allows yo do file resumes properly.
typo? - VMiklos
2008/2/11, Miklos Vajna <vmiklos@frugalware.org>:
On Mon, Feb 11, 2008 at 04:10:51PM +0200, Roman Kyrylych <roman.kyrylych@gmail.com> wrote:
+ extension, which allows yo do file resumes properly.
typo?
Yep. I've took that part from the description of XferCommand and must have broken the sentence while editing. -- Roman Kyrylych (Роман Кирилич)
From 1196a6b6a97c17027608d6c92a3df647deab114c Mon Sep 17 00:00:00 2001 From: Roman Kyrylych <roman@archlinux.org> Date: Sun, 10 Feb 2008 18:56:47 +0200 Subject: [PATCH] makepkg: Support for resuming source downloads
download to *.part and remove the suffix if successful, thus md5sums will be checked only for complete files, and next time makepkg will resume downloading of partial file instead of complaining about invalid md5sums
Signed-off-by: Roman Kyrylych <roman@archlinux.org> --- doc/makepkg.conf.5.txt | 3 +++ etc/makepkg.conf.in | 10 +++++----- scripts/makepkg.sh.in | 13 ++++++++++++- 3 files changed, 20 insertions(+), 6 deletions(-)
diff --git a/doc/makepkg.conf.5.txt b/doc/makepkg.conf.5.txt index f6e4b38..ec72e29 100644 --- a/doc/makepkg.conf.5.txt +++ b/doc/makepkg.conf.5.txt @@ -37,6 +37,9 @@ Options well; the download URL is placed on the end of the command. This is more flexible than the former `FTPAGENT` variable, as any protocol can have a download agent. Several examples are provided in the default makepkg.conf. + All instances of `%u` will be replaced with the download URL. If present, + instances of `%o` will be replaced with the local filename, plus a ``.part'' + extension, which allows yo do file resumes properly. Typo, but no big deal. :P
index d808270..e8ac6a1 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -8,11 +8,11 @@ # #-- The download utilities that makepkg should use to acquire sources # Format: 'protocol::agent' -DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3' - 'http::/usr/bin/wget -c -t 3 --waitretry=3' - 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate' - 'rsync::/usr/bin/rsync -z' - 'scp::/usr/bin/scp -C') +DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3 -O %o %u' + 'http::/usr/bin/wget -c -t 3 --waitretry=3 -O %o %u' + 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate -O %o %u' + 'rsync::/usr/bin/rsync -z %u %o' + 'scp::/usr/bin/scp -C %u %o') Damn, I just wish we could leave what works alone, but that is too hard. :P
Is there anyway we can slightly rework the below so that if neither %o or %u are found in the DLAGENT, we basically do the former behavior? Not sure if this is completely feasible, but it would be nice to have the simple option still available (and especially if someone is using a download client that doesn't need all this complexity).
diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index cecda1d..27bf7fa 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -299,6 +299,16 @@ get_downloadclient() { echo "$agent" }
+get_downloadcmd() { + local dlagent=$1 + local netfile=$2 + local file=$3 + + local dlcmd=$(echo $dlagent | sed "s|%o|\"$file.part\"|" | sed "s|%u|\"$netfile\"|") + + echo "$dlcmd" +} + check_deps() { [ $# -gt 0 ] || return
@@ -476,12 +486,13 @@ download_sources() { msg2 "$(gettext "Downloading %s...")" "$file" # fix flyspray bug #3289 local ret=0 - $dlclient "$netfile" || ret=$? + $(get_downloadcmd "$dlclient" "$netfile" "$file") || ret=$? if [ $ret -gt 0 ]; then error "$(gettext "Failure while downloading %s")" "$file" plain "$(gettext "Aborting...")" exit 1 fi + mv -f "$SRCDEST/$file.part" "$SRCDEST/$file" cp -s --remove-destination "$SRCDEST/$file" "$srcdir/" done
-- 1.5.3.7
Someone doesn't run testing! -Dan
2008/2/12, Dan McGee <dpmcgee@gmail.com>:
index d808270..e8ac6a1 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -8,11 +8,11 @@ # #-- The download utilities that makepkg should use to acquire sources # Format: 'protocol::agent' -DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3' - 'http::/usr/bin/wget -c -t 3 --waitretry=3' - 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate' - 'rsync::/usr/bin/rsync -z' - 'scp::/usr/bin/scp -C') +DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3 -O %o %u' + 'http::/usr/bin/wget -c -t 3 --waitretry=3 -O %o %u' + 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate -O %o %u' + 'rsync::/usr/bin/rsync -z %u %o' + 'scp::/usr/bin/scp -C %u %o') Damn, I just wish we could leave what works alone, but that is too hard. :P
Is there anyway we can slightly rework the below so that if neither %o or %u are found in the DLAGENT, we basically do the former behavior? Not sure if this is completely feasible, but it would be nice to have the simple option still available (and especially if someone is using a download client that doesn't need all this complexity).
Yeah, I thought about that. Will try to do something in the next version.
diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index cecda1d..27bf7fa 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -299,6 +299,16 @@ get_downloadclient() { echo "$agent" }
+get_downloadcmd() { + local dlagent=$1 + local netfile=$2 + local file=$3 + + local dlcmd=$(echo $dlagent | sed "s|%o|\"$file.part\"|" | sed "s|%u|\"$netfile\"|") + + echo "$dlcmd" +} + check_deps() { [ $# -gt 0 ] || return
@@ -476,12 +486,13 @@ download_sources() { msg2 "$(gettext "Downloading %s...")" "$file" # fix flyspray bug #3289 local ret=0 - $dlclient "$netfile" || ret=$? + $(get_downloadcmd "$dlclient" "$netfile" "$file") || ret=$? if [ $ret -gt 0 ]; then error "$(gettext "Failure while downloading %s")" "$file" plain "$(gettext "Aborting...")" exit 1 fi + mv -f "$SRCDEST/$file.part" "$SRCDEST/$file" cp -s --remove-destination "$SRCDEST/$file" "$srcdir/" done
-- 1.5.3.7
Someone doesn't run testing!
I did test the first version (without \" added to dlcmd= and it worked flawlessly. The addition of \" in the second patch broke it :-) So please use the version attached in my first message to test that it works. ;-) -- Roman Kyrylych (Роман Кирилич)
2008/2/12, Roman Kyrylych <roman.kyrylych@gmail.com>:
I did test the first version (without \" added to dlcmd= and it worked flawlessly. The addition of \" in the second patch broke it :-) So please use the version attached in my first message to test that it works. ;-)
Ok, I'm running the patched makepkg for a couple of days already. It turned out that "path with spaces" are non-issue due to a simple reason: all spaces in URLs are converted to %20 anyway. :-) -- Roman Kyrylych (Роман Кирилич)
Roman Kyrylych wrote:
2008/2/12, Roman Kyrylych<roman.kyrylych@gmail.com>:
I did test the first version (without \" added to dlcmd= and it worked flawlessly. The addition of \" in the second patch broke it :-) So please use the version attached in my first message to test that it works. ;-)
Ok, I'm running the patched makepkg for a couple of days already. It turned out that "path with spaces" are non-issue due to a simple reason: all spaces in URLs are converted to %20 anyway. :-)
Oh, of course. And since the filename is taken from the url anyway, well it shouldn't be a problem indeed.
2008/2/12, Dan McGee <dpmcgee@gmail.com>:
From 1196a6b6a97c17027608d6c92a3df647deab114c Mon Sep 17 00:00:00 2001 From: Roman Kyrylych <roman@archlinux.org> Date: Sun, 10 Feb 2008 18:56:47 +0200 Subject: [PATCH] makepkg: Support for resuming source downloads
download to *.part and remove the suffix if successful, thus md5sums will be checked only for complete files, and next time makepkg will resume downloading of partial file instead of complaining about invalid md5sums
Signed-off-by: Roman Kyrylych <roman@archlinux.org> --- doc/makepkg.conf.5.txt | 3 +++ etc/makepkg.conf.in | 10 +++++----- scripts/makepkg.sh.in | 13 ++++++++++++- 3 files changed, 20 insertions(+), 6 deletions(-)
diff --git a/doc/makepkg.conf.5.txt b/doc/makepkg.conf.5.txt index f6e4b38..ec72e29 100644 --- a/doc/makepkg.conf.5.txt +++ b/doc/makepkg.conf.5.txt @@ -37,6 +37,9 @@ Options well; the download URL is placed on the end of the command. This is more flexible than the former `FTPAGENT` variable, as any protocol can have a download agent. Several examples are provided in the default makepkg.conf. + All instances of `%u` will be replaced with the download URL. If present, + instances of `%o` will be replaced with the local filename, plus a ``.part'' + extension, which allows yo do file resumes properly.
Typo, but no big deal. :P
index d808270..e8ac6a1 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -8,11 +8,11 @@ # #-- The download utilities that makepkg should use to acquire sources # Format: 'protocol::agent' -DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3' - 'http::/usr/bin/wget -c -t 3 --waitretry=3' - 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate' - 'rsync::/usr/bin/rsync -z' - 'scp::/usr/bin/scp -C') +DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3 -O %o %u' + 'http::/usr/bin/wget -c -t 3 --waitretry=3 -O %o %u' + 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate -O %o %u' + 'rsync::/usr/bin/rsync -z %u %o' + 'scp::/usr/bin/scp -C %u %o')
Damn, I just wish we could leave what works alone, but that is too hard. :P
Is there anyway we can slightly rework the below so that if neither %o or %u are found in the DLAGENT, we basically do the former behavior? Not sure if this is completely feasible, but it would be nice to have the simple option still available (and especially if someone is using a download client that doesn't need all this complexity).
Here's the latest version with all bugs fixed and backwards compatibility introduced. ;-) From 6fc40a691f6c2d2b11fd8c3f9fe0648aead34484 Mon Sep 17 00:00:00 2001 From: Roman Kyrylych <roman@archlinux.org> Date: Sun, 10 Feb 2008 18:56:47 +0200 Subject: [PATCH] makepkg: Support for resuming source downloads Download to *.part and remove the suffix if successful, thus md5sums will be checked only for complete files, and next time makepkg will resume downloading of partial file instead of complaining about invalid md5sums. Old DLAGENTS format is still supported. Signed-off-by: Roman Kyrylych <roman@archlinux.org> --- doc/makepkg.conf.5.txt | 3 +++ etc/makepkg.conf.in | 10 +++++----- scripts/makepkg.sh.in | 19 ++++++++++++++++++- 3 files changed, 26 insertions(+), 6 deletions(-) diff --git a/doc/makepkg.conf.5.txt b/doc/makepkg.conf.5.txt index f6e4b38..3f6d74c 100644 --- a/doc/makepkg.conf.5.txt +++ b/doc/makepkg.conf.5.txt @@ -37,6 +37,9 @@ Options well; the download URL is placed on the end of the command. This is more flexible than the former `FTPAGENT` variable, as any protocol can have a download agent. Several examples are provided in the default makepkg.conf. + All instances of `%u` will be replaced with the download URL. If present, + instances of `%o` will be replaced with the local filename, plus a ``.part'' + extension, which allows to do file resumes properly. **CARCH=**"carch":: Specifies your computer architecture; possible values include such things diff --git a/etc/makepkg.conf.in b/etc/makepkg.conf.in index 4da4a63..a034b51 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -8,11 +8,11 @@ # #-- The download utilities that makepkg should use to acquire sources # Format: 'protocol::agent' -DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3' - 'http::/usr/bin/wget -c -t 3 --waitretry=3' - 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate' - 'rsync::/usr/bin/rsync -z' - 'scp::/usr/bin/scp -C') +DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3 -O %o %u' + 'http::/usr/bin/wget -c -t 3 --waitretry=3 -O %o %u' + 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate -O %o %u' + 'rsync::/usr/bin/rsync -z %u %o' + 'scp::/usr/bin/scp -C %u %o') # Other common tools: # /usr/bin/snarf diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 598e687..0e28719 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -299,6 +299,20 @@ get_downloadclient() { echo "$agent" } +get_downloadcmd() { + local dlagent=$1 + local netfile=$2 + local file=$3 + + if echo "$dlagent" | grep "%u" ; then + local dlcmd=$(echo $dlagent | sed "s|%o|$file.part|" | sed "s|%u|$netfile|") + else + local dlcmd="$dlagent $netfile" + fi + + echo "$dlcmd" +} + check_deps() { [ $# -gt 0 ] || return @@ -476,12 +490,15 @@ download_sources() { msg2 "$(gettext "Downloading %s...")" "$file" # fix flyspray bug #3289 local ret=0 - $dlclient "$netfile" || ret=$? + $(get_downloadcmd "$dlclient" "$netfile" "$file") || ret=$? if [ $ret -gt 0 ]; then error "$(gettext "Failure while downloading %s")" "$file" plain "$(gettext "Aborting...")" exit 1 fi + if echo "$dlclient" | grep "%o" ; then + mv -f "$SRCDEST/$file.part" "$SRCDEST/$file" + fi cp -s --remove-destination "$SRCDEST/$file" "$srcdir/" done -- 1.5.3.7 -- Roman Kyrylych (Роман Кирилич)
2008/3/1, Roman Kyrylych <roman.kyrylych@gmail.com>:
2008/2/12, Dan McGee <dpmcgee@gmail.com>:
From 1196a6b6a97c17027608d6c92a3df647deab114c Mon Sep 17 00:00:00 2001 From: Roman Kyrylych <roman@archlinux.org> Date: Sun, 10 Feb 2008 18:56:47 +0200 Subject: [PATCH] makepkg: Support for resuming source downloads
download to *.part and remove the suffix if successful, thus md5sums will be checked only for complete files, and next time makepkg will resume downloading of partial file instead of complaining about invalid md5sums
Signed-off-by: Roman Kyrylych <roman@archlinux.org> --- doc/makepkg.conf.5.txt | 3 +++ etc/makepkg.conf.in | 10 +++++----- scripts/makepkg.sh.in | 13 ++++++++++++- 3 files changed, 20 insertions(+), 6 deletions(-)
diff --git a/doc/makepkg.conf.5.txt b/doc/makepkg.conf.5.txt index f6e4b38..ec72e29 100644 --- a/doc/makepkg.conf.5.txt +++ b/doc/makepkg.conf.5.txt @@ -37,6 +37,9 @@ Options well; the download URL is placed on the end of the command. This is more flexible than the former `FTPAGENT` variable, as any protocol can have a download agent. Several examples are provided in the default makepkg.conf. + All instances of `%u` will be replaced with the download URL. If present, + instances of `%o` will be replaced with the local filename, plus a ``.part'' + extension, which allows yo do file resumes properly.
Typo, but no big deal. :P
index d808270..e8ac6a1 100644 --- a/etc/makepkg.conf.in +++ b/etc/makepkg.conf.in @@ -8,11 +8,11 @@ # #-- The download utilities that makepkg should use to acquire sources # Format: 'protocol::agent' -DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3' - 'http::/usr/bin/wget -c -t 3 --waitretry=3' - 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate' - 'rsync::/usr/bin/rsync -z' - 'scp::/usr/bin/scp -C') +DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3 -O %o %u' + 'http::/usr/bin/wget -c -t 3 --waitretry=3 -O %o %u' + 'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate -O %o %u' + 'rsync::/usr/bin/rsync -z %u %o' + 'scp::/usr/bin/scp -C %u %o')
Damn, I just wish we could leave what works alone, but that is too hard. :P
Is there anyway we can slightly rework the below so that if neither %o or %u are found in the DLAGENT, we basically do the former behavior? Not sure if this is completely feasible, but it would be nice to have the simple option still available (and especially if someone is using a download client that doesn't need all this complexity).
Here's the latest version with all bugs fixed and backwards compatibility introduced. ;-)
From 6fc40a691f6c2d2b11fd8c3f9fe0648aead34484 Mon Sep 17 00:00:00 2001
From: Roman Kyrylych <roman@archlinux.org> Date: Sun, 10 Feb 2008 18:56:47 +0200 Subject: [PATCH] makepkg: Support for resuming source downloads
Download to *.part and remove the suffix if successful, thus md5sums will be checked only for complete files, and next time makepkg will resume downloading of partial file
instead of complaining about invalid md5sums.
Old DLAGENTS format is still supported.
. . . Dan, Xavier, Nagy, did you try this? ;-) -- Roman Kyrylych (Роман Кирилич)
Dan, Xavier, Nagy, did you try this? ;-)
To be honest: I didn't (I have no net at home atm, so I'm an active -Sup user now ;-P), but I find resumedownload useful; so if that works, I vote for implement it. Bye ---------------------------------------------------- SZTE Egyetemi Könyvtár - http://www.bibl.u-szeged.hu This mail sent through IMP: http://horde.org/imp/
Roman Kyrylych wrote:
2008/2/11, Roman Kyrylych<roman.kyrylych@gmail.com>:
2008/2/10, Roman Kyrylych<roman.kyrylych@gmail.com>:
Hi!
The inability of makepkg to resume broken source downloads annoyed me pretty much, so here's a patch attached. ;-)
And with the help of Xavier I've remembered the bug that can be re-opened and closed as "Fixed" now. ;-) http://bugs.archlinux.org/task/9299
Previous patch didn't care about path with spaces, here is a fixed version which restores previous behaviour of putting paths/urls in quotes (").
For some reason, adding these quotes broke it. -> Downloading opera-9.50-20080131.6-shared-qt.i386-1786.tar.bz2... "http://snapshot.opera.com/unix/snapshot-1786/intel-linux/opera-9.50-20080131...": Unsupported scheme. ==> ERROR: Failure while downloading opera-9.50-20080131.6-shared-qt.i386-1786.tar.bz2 The original version works fine though. But I didn't check what happens with spaces..
participants (5)
-
Dan McGee
-
Miklos Vajna
-
Nagy Gabor
-
Roman Kyrylych
-
Xavier