[pacman-dev] bug: pacman giving up too fast downloading
Well, I have lots of errors while downloading packages with pacman, and almost never while downloading files from Shiretoko Firefox. I have tried many mirrors, that's the same for each. Oh... I use a 56k modem (Intel 536EP Data Fax modem). But even while downloading in Firefox, it sometimes give me the feeling it has stopped downloading for about 40 secs... while pacman seems to give up after about 20 secs. I did not tried yet to download with wget or curl (does pacman use one of them?). Is there a constant I could change in the code, and rebuild it?... I did not download the code of pacman yet.
On Tue, 2010-02-16 at 09:25 -0500, Paul Dufresne wrote:
Well, I have lots of errors while downloading packages with pacman, and almost never while downloading files from Shiretoko Firefox.
I have tried many mirrors, that's the same for each.
Oh... I use a 56k modem (Intel 536EP Data Fax modem).
But even while downloading in Firefox, it sometimes give me the feeling it has stopped downloading for about 40 secs... while pacman seems to give up after about 20 secs.
I did not tried yet to download with wget or curl (does pacman use one of them?).
Is there a constant I could change in the code, and rebuild it?... I did not download the code of pacman yet.
I am not a pacman developer, but I believe you can search for 'fetchTimeout' in pacman-3.3.3/lib/libalpm/dload.c and change it from 10 to whatever you want. Alternatively, you could use XferCommand in /etc/pacman.conf with wget, curl, etc. Read the man page for details. Hope this helps Jonathan
Thank you Jonathan for your answer. For now, uncommenting the xfercommand with curl almost totally fix the issue (well, once I decided to stop it because I was waiting for about 2 mins with no result, much harder to stop that way, have to use kill command rather than ^C), but after that it have gone well, I was able to download about 12 files of about 500k each, where I had to go about 3 times to get a 350k files before. It is not uncommon for the speed reported by curl to go to 0 for 20 seconds, then go as high as 25kB/sec for a short while, then going down to the normal 4kB/s speed. (I know, for most people, 4kB is way too slow, but that's what I expect). So, I suppose this is the out-of-order effect of TCP going on... that is, while it seems to be receiving nothing for the user-land, kernel is receiving packets, but transmit them only to userland when the next packet is received. I guess the 10 sec Timeout would not be so short, if it was really the kernel not receiving the packets for that much time, and not the user-space file not growing. That said, I would have to recompile pacman with a modified timeout to be sure it does really help... but I did not compile much if any packages yet outside the AUR ones. Not sure if it is as simple as makepkg. 2010/2/16, Jonathan Conder <j@skurvy.no-ip.org>:
On Tue, 2010-02-16 at 09:25 -0500, Paul Dufresne wrote:
Well, I have lots of errors while downloading packages with pacman, and almost never while downloading files from Shiretoko Firefox.
I have tried many mirrors, that's the same for each.
Oh... I use a 56k modem (Intel 536EP Data Fax modem).
But even while downloading in Firefox, it sometimes give me the feeling it has stopped downloading for about 40 secs... while pacman seems to give up after about 20 secs.
I did not tried yet to download with wget or curl (does pacman use one of them?).
Is there a constant I could change in the code, and rebuild it?... I did not download the code of pacman yet.
I am not a pacman developer, but I believe you can search for 'fetchTimeout' in pacman-3.3.3/lib/libalpm/dload.c and change it from 10 to whatever you want. Alternatively, you could use XferCommand in /etc/pacman.conf with wget, curl, etc. Read the man page for details.
Hope this helps Jonathan
On Tue, Feb 16, 2010 at 3:25 PM, Paul Dufresne <dufresnep@gmail.com> wrote:
Well, I have lots of errors while downloading packages with pacman, and almost never while downloading files from Shiretoko Firefox.
I have tried many mirrors, that's the same for each.
Oh... I use a 56k modem (Intel 536EP Data Fax modem).
But even while downloading in Firefox, it sometimes give me the feeling it has stopped downloading for about 40 secs... while pacman seems to give up after about 20 secs.
I did not tried yet to download with wget or curl (does pacman use one of them?).
Is there a constant I could change in the code, and rebuild it?... I did not download the code of pacman yet.
Using arch with a 56k modem is called masochism :) Seriously, I would never do that, I would just get a dvd of a linux distribution that already contains a lot of software. If that doesn't make any sense.. well, curl and wget are indeed known to behave more nicely than libfetch with crap connections.
On Tue, Feb 16, 2010 at 3:25 PM, Paul Dufresne <dufresnep@gmail.com> wrote:
Well, I have lots of errors while downloading packages with pacman, and almost never while downloading files from Shiretoko Firefox.
I have tried many mirrors, that's the same for each.
Oh... I use a 56k modem (Intel 536EP Data Fax modem).
But even while downloading in Firefox, it sometimes give me the feeling it has stopped downloading for about 40 secs... while pacman seems to give up after about 20 secs.
I did not tried yet to download with wget or curl (does pacman use one of them?).
Is there a constant I could change in the code, and rebuild it?... I did not download the code of pacman yet.
Using arch with a 56k modem is called masochism :)
Seriously, I would never do that, I would just get a dvd of a linux distribution that already contains a lot of software.
If that doesn't make any sense.. well, curl and wget are indeed known to behave more nicely than libfetch with crap connections.
I used Arch without internet for a while. :-) (-Sp) Xavier, I am happy to hear that someone uses a slow connection, at least we have a beta-tester for timeout-related patches. Maybe this is a good occasion to test fetchTimeout. ;-) Bye
I have tested pacman recompiled with a 40 sec fetchTimeout rather than 10. But first I began by recommenting out the XferCommand in /etc/pacman.conf, first rebuild pacman without modifications with abs to get sure the errors was there, then tried to make the modification then use abs, but realized I was changing nothing, so I rechange fetchTimeout in pacman-3.3.3/lib/libalpm/dload.c to 40 then use ./configure prefix=/usr/local, make, #make install, and then used #/usr/local/bin/pacman -Syu I have left it all the night (about 8 hours) and only had one error getting gimp (maybe the package was really not there anymore). That said, I may boost it to 50 secs rather than 40, because a while, I was counting manually about by 1 by 1 second when it seems stopped, and once got up to 33. I hope you will boost the fetchTimeout value for others people using a slow connection like me. Thank you again Jonathan for the hint! 2010/2/16, Nagy Gabor <ngaba@bibl.u-szeged.hu>:
On Tue, Feb 16, 2010 at 3:25 PM, Paul Dufresne <dufresnep@gmail.com> wrote:
Well, I have lots of errors while downloading packages with pacman, and almost never while downloading files from Shiretoko Firefox.
I have tried many mirrors, that's the same for each.
Oh... I use a 56k modem (Intel 536EP Data Fax modem).
But even while downloading in Firefox, it sometimes give me the feeling it has stopped downloading for about 40 secs... while pacman seems to give up after about 20 secs.
I did not tried yet to download with wget or curl (does pacman use one of them?).
Is there a constant I could change in the code, and rebuild it?... I did not download the code of pacman yet.
Using arch with a 56k modem is called masochism :)
Seriously, I would never do that, I would just get a dvd of a linux distribution that already contains a lot of software.
If that doesn't make any sense.. well, curl and wget are indeed known to behave more nicely than libfetch with crap connections.
I used Arch without internet for a while. :-) (-Sp)
Xavier, I am happy to hear that someone uses a slow connection, at least we have a beta-tester for timeout-related patches. Maybe this is a good occasion to test fetchTimeout. ;-)
Bye
On Wed, Feb 17, 2010 at 6:29 AM, Paul Dufresne <dufresnep@gmail.com> wrote:
I have tested pacman recompiled with a 40 sec fetchTimeout rather than 10.
But first I began by recommenting out the XferCommand in /etc/pacman.conf, first rebuild pacman without modifications with abs to get sure the errors was there, then tried to make the modification then use abs, but realized I was changing nothing, so I rechange fetchTimeout in pacman-3.3.3/lib/libalpm/dload.c to 40 then use ./configure prefix=/usr/local, make, #make install, and then used #/usr/local/bin/pacman -Syu
I have left it all the night (about 8 hours) and only had one error getting gimp (maybe the package was really not there anymore).
That said, I may boost it to 50 secs rather than 40, because a while, I was counting manually about by 1 by 1 second when it seems stopped, and once got up to 33.
I hope you will boost the fetchTimeout value for others people using a slow connection like me.
Thank you again Jonathan for the hint!
I did this a while back, which is included in 3.3.1: http://projects.archlinux.org/pacman.git/commit/?id=14a93b2e436ed19c814653c3... It was done in some part to try and address http://bugs.archlinux.org/task/15369, so I don't want to raise this back above 10 seconds for everyone, but then again it isn't all that configurable right now. -Dan
On Wed, Feb 17, 2010 at 7:21 AM, Dan McGee <dpmcgee@gmail.com> wrote:
On Wed, Feb 17, 2010 at 6:29 AM, Paul Dufresne <dufresnep@gmail.com> wrote:
I have tested pacman recompiled with a 40 sec fetchTimeout rather than 10.
But first I began by recommenting out the XferCommand in /etc/pacman.conf, first rebuild pacman without modifications with abs to get sure the errors was there, then tried to make the modification then use abs, but realized I was changing nothing, so I rechange fetchTimeout in pacman-3.3.3/lib/libalpm/dload.c to 40 then use ./configure prefix=/usr/local, make, #make install, and then used #/usr/local/bin/pacman -Syu
I have left it all the night (about 8 hours) and only had one error getting gimp (maybe the package was really not there anymore).
That said, I may boost it to 50 secs rather than 40, because a while, I was counting manually about by 1 by 1 second when it seems stopped, and once got up to 33.
I hope you will boost the fetchTimeout value for others people using a slow connection like me.
Thank you again Jonathan for the hint!
I did this a while back, which is included in 3.3.1: http://projects.archlinux.org/pacman.git/commit/?id=14a93b2e436ed19c814653c3...
You also removed the "TODO make this a config option" :) That'd address this issue nicely, just making the fetchTimeout variable via pacman.conf
On 17 February 2010 07:07, Xavier Chantry <chantry.xavier@gmail.com> wrote:
Using arch with a 56k modem is called masochism :)
Seriously, I would never do that, I would just get a dvd of a linux distribution that already contains a lot of software.
If that doesn't make any sense.. well, curl and wget are indeed known to behave more nicely than libfetch with crap connections.
Heh, I've been on crappy connections back and forth for almost a year. And more so during travel right now. But not as crappy as 4KB/s =p -- GPG/PGP ID: B42DDCAD
participants (7)
-
Aaron Griffin
-
Dan McGee
-
Jonathan Conder
-
Nagy Gabor
-
Paul Dufresne
-
Ray Rashif
-
Xavier Chantry