[pacman-dev] [PATCH] pacman: add option --max-recv-speed
My dad was watching internet connected television, while I decided to check fixes for Arch. There were new kernel, some libreoffice stuff, and so on making the download to be more than few megabytes. When I downloaded the packages television sound track was affected annoying way. Obviosly my download was affecting a stream that had greater service quality expectancy. This patch will allow an user to polite and throttle down bandwidth usage in benefit for others. Signed-off-by: Sami Kerola <kerolasa@iki.fi> --- doc/pacman.8.txt | 5 +++++ lib/libalpm/alpm.h | 5 +++++ lib/libalpm/dload.c | 4 ++++ lib/libalpm/handle.c | 13 +++++++++++++ lib/libalpm/handle.h | 1 + src/pacman/conf.c | 1 + src/pacman/conf.h | 2 ++ src/pacman/pacman.c | 6 ++++++ 8 files changed, 37 insertions(+) diff --git a/doc/pacman.8.txt b/doc/pacman.8.txt index 574995c..da8fa1b 100644 --- a/doc/pacman.8.txt +++ b/doc/pacman.8.txt @@ -362,6 +362,11 @@ Sync Options[[SO]] packages that are no longer installed; use two to remove all files from the cache. In both cases, you will have a yes or no option to remove packages and/or unused downloaded databases. + +*\--max-recv-speed* <bits>:: + Define maximum transfer rate in bits per seconds. This option is + useful when network bandwidth is scarce resource, and other users + of the network has greater service expecations. + If you use a network shared cache, see the 'CleanMethod' option in linkman:pacman.conf[5]. diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h index b049007..bc77641 100644 --- a/lib/libalpm/alpm.h +++ b/lib/libalpm/alpm.h @@ -533,6 +533,11 @@ int alpm_option_get_usesyslog(alpm_handle_t *handle); /** Sets whether to use syslog (0 is FALSE, TRUE otherwise). */ int alpm_option_set_usesyslog(alpm_handle_t *handle, int usesyslog); +/** Returns maximum download receive speed. */ +int alpm_option_get_max_recv_speed(alpm_handle_t *handle); +/** Sets maximum download receive speed. */ +int alpm_option_set_max_recv_speed(alpm_handle_t *handle, int rate); + /** @name Accessors to the list of no-upgrade files. * These functions modify the list of files which should * not be updated by package installation. diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 8537b3d..25f2423 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -331,6 +331,10 @@ static void curl_set_handle_opts(struct dload_payload *payload, (intmax_t)st.st_size); payload->initial_size = st.st_size; } + + if(payload->handle->max_recv_speed) { + curl_easy_setopt(curl, CURLOPT_MAX_RECV_SPEED_LARGE, payload->handle->max_recv_speed); + } } static void mask_signal(int signum, void (*handler)(int), diff --git a/lib/libalpm/handle.c b/lib/libalpm/handle.c index 53c86c5..b061efd 100644 --- a/lib/libalpm/handle.c +++ b/lib/libalpm/handle.c @@ -181,6 +181,12 @@ alpm_cb_progress SYMEXPORT alpm_option_get_progresscb(alpm_handle_t *handle) return handle->progresscb; } +int SYMEXPORT alpm_option_get_max_recv_speed(alpm_handle_t *handle) +{ + CHECK_HANDLE(handle, return -1); + return handle->max_recv_speed; +} + const char SYMEXPORT *alpm_option_get_root(alpm_handle_t *handle) { CHECK_HANDLE(handle, return NULL); @@ -314,6 +320,13 @@ int SYMEXPORT alpm_option_set_progresscb(alpm_handle_t *handle, alpm_cb_progress return 0; } +int SYMEXPORT alpm_option_set_max_recv_speed(alpm_handle_t *handle, int rate) +{ + CHECK_HANDLE(handle, return -1); + handle->max_recv_speed = rate; + return 0; +} + static char *canonicalize_path(const char *path) { char *new_path; diff --git a/lib/libalpm/handle.h b/lib/libalpm/handle.h index 5e84d58..f5b4baf 100644 --- a/lib/libalpm/handle.h +++ b/lib/libalpm/handle.h @@ -90,6 +90,7 @@ struct __alpm_handle_t { char *arch; /* Architecture of packages we should allow */ double deltaratio; /* Download deltas if possible; a ratio value */ int usesyslog; /* Use syslog instead of logfile? */ /* TODO move to frontend */ + int max_recv_speed; /* Maximum download receive speed. */ int checkspace; /* Check disk space before installing */ alpm_siglevel_t siglevel; /* Default signature verification level */ alpm_siglevel_t localfilesiglevel; /* Signature verification level for local file diff --git a/src/pacman/conf.c b/src/pacman/conf.c index cd357ab..e6112f1 100644 --- a/src/pacman/conf.c +++ b/src/pacman/conf.c @@ -682,6 +682,7 @@ static int setup_libalpm(void) alpm_option_set_eventcb(handle, cb_event); alpm_option_set_questioncb(handle, cb_question); alpm_option_set_progresscb(handle, cb_progress); + alpm_option_set_max_recv_speed(handle, config->max_recv_speed); config->logfile = config->logfile ? config->logfile : strdup(LOGFILE); ret = alpm_option_set_logfile(handle, config->logfile); diff --git a/src/pacman/conf.h b/src/pacman/conf.h index dcd3204..30ccd60 100644 --- a/src/pacman/conf.h +++ b/src/pacman/conf.h @@ -47,6 +47,7 @@ typedef struct __config_t { unsigned short checkspace; unsigned short usesyslog; unsigned short color; + unsigned int max_recv_speed; double deltaratio; char *arch; char *print_format; @@ -143,6 +144,7 @@ enum { OP_ASEXPLICIT, OP_ARCH, OP_PRINTFORMAT, + OP_MAXRECVSPEED, OP_GPGDIR, OP_DBONLY, OP_FORCE, diff --git a/src/pacman/pacman.c b/src/pacman/pacman.c index 24fd57f..c1c6705 100644 --- a/src/pacman/pacman.c +++ b/src/pacman/pacman.c @@ -177,6 +177,7 @@ static void usage(int op, const char * const myname) addlist(_(" --force force install, overwrite conflicting files\n")); addlist(_(" --asdeps install packages as non-explicitly installed\n")); addlist(_(" --asexplicit install packages as explicitly installed\n")); + addlist(_(" --max-recv-speed <bits> limit download speed to bits per second (libcurl)\n")); addlist(_(" --ignore <pkg> ignore a package upgrade (can be used more than once)\n")); addlist(_(" --ignoregroup <grp>\n" " ignore a group upgrade (can be used more than once)\n")); @@ -510,6 +511,10 @@ static int parsearg_trans(int opt) check_optarg(); config->print_format = strdup(optarg); break; + case OP_MAXRECVSPEED: + check_optarg(); + config->max_recv_speed = (unsigned int)atoi(optarg); + break; default: return 1; } return 0; @@ -645,6 +650,7 @@ static int parseargs(int argc, char *argv[]) {"asexplicit", no_argument, 0, OP_ASEXPLICIT}, {"arch", required_argument, 0, OP_ARCH}, {"print-format", required_argument, 0, OP_PRINTFORMAT}, + {"max-recv-speed", required_argument, 0, OP_MAXRECVSPEED}, {"gpgdir", required_argument, 0, OP_GPGDIR}, {"dbonly", no_argument, 0, OP_DBONLY}, {"color", required_argument, 0, OP_COLOR}, -- 1.8.4
On Sat, Sep 07, 2013 at 10:29:31PM +0100, Sami Kerola wrote:
My dad was watching internet connected television, while I decided to check fixes for Arch. There were new kernel, some libreoffice stuff, and so on making the download to be more than few megabytes. When I downloaded the packages television sound track was affected annoying way. Obviosly my download was affecting a stream that had greater service quality expectancy. This patch will allow an user to polite and throttle down bandwidth usage in benefit for others.
Signed-off-by: Sami Kerola <kerolasa@iki.fi> ---
I've been extremely hesistant to accept patches that tune the downloader -- it really ought to be an invisible thing that "just works". If you really need this functionality, I'd suggest using XferCommand in /etc/pacman.conf.
doc/pacman.8.txt | 5 +++++ lib/libalpm/alpm.h | 5 +++++ lib/libalpm/dload.c | 4 ++++ lib/libalpm/handle.c | 13 +++++++++++++ lib/libalpm/handle.h | 1 + src/pacman/conf.c | 1 + src/pacman/conf.h | 2 ++ src/pacman/pacman.c | 6 ++++++ 8 files changed, 37 insertions(+)
diff --git a/doc/pacman.8.txt b/doc/pacman.8.txt index 574995c..da8fa1b 100644 --- a/doc/pacman.8.txt +++ b/doc/pacman.8.txt @@ -362,6 +362,11 @@ Sync Options[[SO]] packages that are no longer installed; use two to remove all files from the cache. In both cases, you will have a yes or no option to remove packages and/or unused downloaded databases. + +*\--max-recv-speed* <bits>:: + Define maximum transfer rate in bits per seconds. This option is + useful when network bandwidth is scarce resource, and other users + of the network has greater service expecations.
One might argue that QoS on a network device is a far better mechanism for dictating this rather than hoping that clients tune it on their own.
+ If you use a network shared cache, see the 'CleanMethod' option in linkman:pacman.conf[5]. diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h index b049007..bc77641 100644 --- a/lib/libalpm/alpm.h +++ b/lib/libalpm/alpm.h @@ -533,6 +533,11 @@ int alpm_option_get_usesyslog(alpm_handle_t *handle); /** Sets whether to use syslog (0 is FALSE, TRUE otherwise). */ int alpm_option_set_usesyslog(alpm_handle_t *handle, int usesyslog);
+/** Returns maximum download receive speed. */ +int alpm_option_get_max_recv_speed(alpm_handle_t *handle); +/** Sets maximum download receive speed. */ +int alpm_option_set_max_recv_speed(alpm_handle_t *handle, int rate); + /** @name Accessors to the list of no-upgrade files. * These functions modify the list of files which should * not be updated by package installation. diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c index 8537b3d..25f2423 100644 --- a/lib/libalpm/dload.c +++ b/lib/libalpm/dload.c @@ -331,6 +331,10 @@ static void curl_set_handle_opts(struct dload_payload *payload, (intmax_t)st.st_size); payload->initial_size = st.st_size; } + + if(payload->handle->max_recv_speed) { + curl_easy_setopt(curl, CURLOPT_MAX_RECV_SPEED_LARGE, payload->handle->max_recv_speed); + } }
static void mask_signal(int signum, void (*handler)(int), diff --git a/lib/libalpm/handle.c b/lib/libalpm/handle.c index 53c86c5..b061efd 100644 --- a/lib/libalpm/handle.c +++ b/lib/libalpm/handle.c @@ -181,6 +181,12 @@ alpm_cb_progress SYMEXPORT alpm_option_get_progresscb(alpm_handle_t *handle) return handle->progresscb; }
+int SYMEXPORT alpm_option_get_max_recv_speed(alpm_handle_t *handle) +{ + CHECK_HANDLE(handle, return -1); + return handle->max_recv_speed; +} + const char SYMEXPORT *alpm_option_get_root(alpm_handle_t *handle) { CHECK_HANDLE(handle, return NULL); @@ -314,6 +320,13 @@ int SYMEXPORT alpm_option_set_progresscb(alpm_handle_t *handle, alpm_cb_progress return 0; }
+int SYMEXPORT alpm_option_set_max_recv_speed(alpm_handle_t *handle, int rate) +{ + CHECK_HANDLE(handle, return -1); + handle->max_recv_speed = rate; + return 0; +} + static char *canonicalize_path(const char *path) { char *new_path; diff --git a/lib/libalpm/handle.h b/lib/libalpm/handle.h index 5e84d58..f5b4baf 100644 --- a/lib/libalpm/handle.h +++ b/lib/libalpm/handle.h @@ -90,6 +90,7 @@ struct __alpm_handle_t { char *arch; /* Architecture of packages we should allow */ double deltaratio; /* Download deltas if possible; a ratio value */ int usesyslog; /* Use syslog instead of logfile? */ /* TODO move to frontend */ + int max_recv_speed; /* Maximum download receive speed. */ int checkspace; /* Check disk space before installing */ alpm_siglevel_t siglevel; /* Default signature verification level */ alpm_siglevel_t localfilesiglevel; /* Signature verification level for local file diff --git a/src/pacman/conf.c b/src/pacman/conf.c index cd357ab..e6112f1 100644 --- a/src/pacman/conf.c +++ b/src/pacman/conf.c @@ -682,6 +682,7 @@ static int setup_libalpm(void) alpm_option_set_eventcb(handle, cb_event); alpm_option_set_questioncb(handle, cb_question); alpm_option_set_progresscb(handle, cb_progress); + alpm_option_set_max_recv_speed(handle, config->max_recv_speed);
config->logfile = config->logfile ? config->logfile : strdup(LOGFILE); ret = alpm_option_set_logfile(handle, config->logfile); diff --git a/src/pacman/conf.h b/src/pacman/conf.h index dcd3204..30ccd60 100644 --- a/src/pacman/conf.h +++ b/src/pacman/conf.h @@ -47,6 +47,7 @@ typedef struct __config_t { unsigned short checkspace; unsigned short usesyslog; unsigned short color; + unsigned int max_recv_speed; double deltaratio; char *arch; char *print_format; @@ -143,6 +144,7 @@ enum { OP_ASEXPLICIT, OP_ARCH, OP_PRINTFORMAT, + OP_MAXRECVSPEED, OP_GPGDIR, OP_DBONLY, OP_FORCE, diff --git a/src/pacman/pacman.c b/src/pacman/pacman.c index 24fd57f..c1c6705 100644 --- a/src/pacman/pacman.c +++ b/src/pacman/pacman.c @@ -177,6 +177,7 @@ static void usage(int op, const char * const myname) addlist(_(" --force force install, overwrite conflicting files\n")); addlist(_(" --asdeps install packages as non-explicitly installed\n")); addlist(_(" --asexplicit install packages as explicitly installed\n")); + addlist(_(" --max-recv-speed <bits> limit download speed to bits per second (libcurl)\n")); addlist(_(" --ignore <pkg> ignore a package upgrade (can be used more than once)\n")); addlist(_(" --ignoregroup <grp>\n" " ignore a group upgrade (can be used more than once)\n")); @@ -510,6 +511,10 @@ static int parsearg_trans(int opt) check_optarg(); config->print_format = strdup(optarg); break; + case OP_MAXRECVSPEED: + check_optarg(); + config->max_recv_speed = (unsigned int)atoi(optarg); + break; default: return 1; } return 0; @@ -645,6 +650,7 @@ static int parseargs(int argc, char *argv[]) {"asexplicit", no_argument, 0, OP_ASEXPLICIT}, {"arch", required_argument, 0, OP_ARCH}, {"print-format", required_argument, 0, OP_PRINTFORMAT}, + {"max-recv-speed", required_argument, 0, OP_MAXRECVSPEED}, {"gpgdir", required_argument, 0, OP_GPGDIR}, {"dbonly", no_argument, 0, OP_DBONLY}, {"color", required_argument, 0, OP_COLOR}, -- 1.8.4
On 7 September 2013 22:36, Dave Reisner <d@falconindy.com> wrote:
On Sat, Sep 07, 2013 at 10:29:31PM +0100, Sami Kerola wrote:
My dad was watching internet connected television, while I decided to check fixes for Arch. There were new kernel, some libreoffice stuff, and so on making the download to be more than few megabytes. When I downloaded the packages television sound track was affected annoying way. Obviosly my download was affecting a stream that had greater service quality expectancy. This patch will allow an user to polite and throttle down bandwidth usage in benefit for others.
Signed-off-by: Sami Kerola <kerolasa@iki.fi> ---
I've been extremely hesistant to accept patches that tune the downloader -- it really ought to be an invisible thing that "just works". If you really need this functionality, I'd suggest using XferCommand in /etc/pacman.conf.
Because the libcurl already has everything needed it is just a matter of using what is provided. Therefore I do not see this as big change, but then again XferCommand is an alternative which allows to do the same without changes to pacman code.
doc/pacman.8.txt | 5 +++++ lib/libalpm/alpm.h | 5 +++++ lib/libalpm/dload.c | 4 ++++ lib/libalpm/handle.c | 13 +++++++++++++ lib/libalpm/handle.h | 1 + src/pacman/conf.c | 1 + src/pacman/conf.h | 2 ++ src/pacman/pacman.c | 6 ++++++ 8 files changed, 37 insertions(+)
diff --git a/doc/pacman.8.txt b/doc/pacman.8.txt index 574995c..da8fa1b 100644 --- a/doc/pacman.8.txt +++ b/doc/pacman.8.txt @@ -362,6 +362,11 @@ Sync Options[[SO]] packages that are no longer installed; use two to remove all files from the cache. In both cases, you will have a yes or no option to remove packages and/or unused downloaded databases. + +*\--max-recv-speed* <bits>:: + Define maximum transfer rate in bits per seconds. This option is + useful when network bandwidth is scarce resource, and other users + of the network has greater service expecations.
One might argue that QoS on a network device is a far better mechanism for dictating this rather than hoping that clients tune it on their own.
I agree, it kernel QoS probably more correct, but not quick & easy way to do limit transfer rate. Something like /sbin/tc class add dev eth1 parent 1:0 classid 1:10 htb rate 512kbps ceil 640kbps prio 0 seems to do the trick, if I am reading instructions correctly[1]. But that will outbound http, not only pacman traffic. Making the traffic shape line smart enough limit only targets from mirrorlist file feels a bit clumsy. The other use for --max-recv-speed I can think of is when an user chucks update to cron, and it does not matter if download takes a moment longer. From system admin point of view it makes sense to do that, if you are managing more than handful of installations, and you want to avoid them consuming (a) bandwidth that was meant for client use, and (b) killing server providing the packages as numerous servers run updates concurrently. But since it feels pacman will not support native rate limiting, perhaps the thing to do is to add the necessary XferCommand line to wiki[2]. [1] http://www.cyberciti.biz/faq/linux-traffic-shaping-using-tc-to-control-http-... [2] https://wiki.archlinux.org/index.php/Pacman_Tips -- Sami Kerola http://www.iki.fi/kerolasa/
participants (2)
-
Dave Reisner
-
Sami Kerola