[arch-general] What happened to Powerpill?

Marek Otahal

26 Mar 2011 26 Mar '11

11:28 a.m.

Hi, anyone knows what happened to Xyne's powerpill [1] pacman wrapper? I started using it recently for it's aria2c multithreaded downloads but a couple of days it disappeared from community/aur and there's no trace. Thanks, marek [1] https://wiki.archlinux.org/index.php/Powerpill -- Marek Otahal :o)

Attachments:

signature.asc (application/pgp-signature — 836 bytes)

Show replies by date

Jelle van der Waa

26 Mar 26 Mar

11:32 a.m.

On Sat, 2011-03-26 at 11:28 +0100, Marek Otahal wrote:

...

Hi, anyone knows what happened to Xyne's powerpill [1] pacman wrapper? I started using it recently for it's aria2c multithreaded downloads but a couple of days it disappeared from community/aur and there's no trace. Thanks, marek

[1] https://wiki.archlinux.org/index.php/Powerpill

RIP -- Jelle van der Waa

cantabile

11:56 a.m.

Le 26/03/2011 12:28, Marek Otahal a écrit :

...

Hi, anyone knows what happened to Xyne's powerpill [1] pacman wrapper? I started using it recently for it's aria2c multithreaded downloads but a couple of days it disappeared from community/aur and there's no trace. Thanks, marek

[1] https://wiki.archlinux.org/index.php/Powerpill

This: https://bbs.archlinux.org/viewtopic.php?id=115660 -- cantabile "Jayne is a girl's name." -- River

Cédric Girard

28 Mar 28 Mar

10:08 a.m.

On Sat, Mar 26, 2011 at 11:28 AM, Marek Otahal wrote:

...

I started using it recently for it's aria2c multithreaded downloads

Hi, You may continue to use aria2c with various techniques explained here: https://wiki.archlinux.org/index.php/Improve_Pacman_Performance#Using_aria2 Regards, -- Cédric Girard

Oon-Ee Ng

10:13 a.m.

2011/3/28 Cédric Girard :

...

On Sat, Mar 26, 2011 at 11:28 AM, Marek Otahal wrote:

...
I started using it recently for it's aria2c multithreaded downloads

Hi, You may continue to use aria2c with various techniques explained here: https://wiki.archlinux.org/index.php/Improve_Pacman_Performance#Using_aria2

Regards, -- Cédric Girard

Unfortunately those techniques don't give the biggest speedup - simultaneous download of multiple files. Does help, though =)

Cédric Girard

10:31 a.m.

On Mon, Mar 28, 2011 at 10:13 AM, Oon-Ee Ng wrote:

...

2011/3/28 Cédric Girard :

...
On Sat, Mar 26, 2011 at 11:28 AM, Marek Otahal wrote:

...
I started using it recently for it's aria2c multithreaded downloads

Hi, You may continue to use aria2c with various techniques explained here:

https://wiki.archlinux.org/index.php/Improve_Pacman_Performance#Using_aria2

...
Regards, -- Cédric Girard

Unfortunately those techniques don't give the biggest speedup - simultaneous download of multiple files. Does help, though =)

Downloading one file from several mirrors should be equivalent to downloading multiple files from one mirror (or even better). Or am I wrong? -- Cédric Girard

Oon-Ee Ng

11:07 a.m.

2011/3/28 Cédric Girard :

...

On Mon, Mar 28, 2011 at 10:13 AM, Oon-Ee Ng wrote:

...
2011/3/28 Cédric Girard :

...
On Sat, Mar 26, 2011 at 11:28 AM, Marek Otahal wrote:

...
I started using it recently for it's aria2c multithreaded downloads

Hi, You may continue to use aria2c with various techniques explained here:

https://wiki.archlinux.org/index.php/Improve_Pacman_Performance#Using_aria2

...
Regards, -- Cédric Girard

Unfortunately those techniques don't give the biggest speedup - simultaneous download of multiple files. Does help, though =)

Downloading one file from several mirrors should be equivalent to downloading multiple files from one mirror (or even better). Or am I wrong?

-- Cédric Girard

If you have 10 files to download, powerpill allows for 1 file from mirror A, another from mirror B, and chunks of that large 68MB file from mirrors C, D, and E at the same time. With the other solutions, you'd still wait for file 1 to finish downloading before downloading file 2.

Cédric Girard

11:43 a.m.

On Mon, Mar 28, 2011 at 11:07 AM, Oon-Ee Ng wrote:

...

If you have 10 files to download, powerpill allows for 1 file from mirror A, another from mirror B, and chunks of that large 68MB file from mirrors C, D, and E at the same time.

With the other solutions, you'd still wait for file 1 to finish downloading before downloading file 2.

I understand this as being more flexible. But bandwith-wise, I do not see why the Powerpill solution is more efficient. Or maybe this come useful only when downloading small files where file content transfert itself is negligible compared to connection opening and other protocol handling... -- Cédric Girard

Nicolas Bigaouette

2:49 p.m.

I guess it would make sense if your own bandwidth is bigger then the mirror's... 2011/3/28 Cédric Girard

...

On Mon, Mar 28, 2011 at 11:07 AM, Oon-Ee Ng wrote:

...
If you have 10 files to download, powerpill allows for 1 file from mirror A, another from mirror B, and chunks of that large 68MB file from mirrors C, D, and E at the same time.

With the other solutions, you'd still wait for file 1 to finish downloading before downloading file 2.

I understand this as being more flexible. But bandwith-wise, I do not see why the Powerpill solution is more efficient. Or maybe this come useful only when downloading small files where file content transfert itself is negligible compared to connection opening and other protocol handling...

-- Cédric Girard

Corrado Primier

4 p.m.

2011/3/28 Nicolas Bigaouette :

...

I guess it would make sense if your own bandwidth is bigger then the mirror's...

It happens with my ISP that the single connection bandwidth looks to be capped. I have 8Mbit but a single transfer rarely exceeds 2Mbit, unless it is very late night or Sunday morning :) Powerpill allowed me to fully use my bandwidth. I recognize it's a bad traffic shaping problem (and powerpill had its shortcomings too), but it solved it for me. Corrado

Madhurya Kakati

5 Apr 5 Apr

4:46 p.m.

Please, someone restart powerpill development. It was awesome. I had to remove it in order to upgrade to pacman 3.5 but now I can't find it anywhere. On 3/28/11, Corrado Primier wrote:

...

2011/3/28 Nicolas Bigaouette :

...
I guess it would make sense if your own bandwidth is bigger then the mirror's...

It happens with my ISP that the single connection bandwidth looks to be capped. I have 8Mbit but a single transfer rarely exceeds 2Mbit, unless it is very late night or Sunday morning :) Powerpill allowed me to fully use my bandwidth. I recognize it's a bad traffic shaping problem (and powerpill had its shortcomings too), but it solved it for me.

Corrado

-- Sent from my mobile device

Jelle van der Waa

10:54 p.m.

On Tue, 2011-04-05 at 20:16 +0530, Madhurya Kakati wrote:

...

Please, someone restart powerpill development. It was awesome. I had to remove it in order to upgrade to pacman 3.5 but now I can't find it anywhere.

On 3/28/11, Corrado Primier wrote:

...
2011/3/28 Nicolas Bigaouette :

...
I guess it would make sense if your own bandwidth is bigger then the mirror's...

It happens with my ISP that the single connection bandwidth looks to be capped. I have 8Mbit but a single transfer rarely exceeds 2Mbit, unless it is very late night or Sunday morning :) Powerpill allowed me to fully use my bandwidth. I recognize it's a bad traffic shaping problem (and powerpill had its shortcomings too), but it solved it for me.

Corrado

The powerpill source is probably still available, so start hacking ;) -- Jelle van der Waa

Oon-Ee Ng

6 Apr 6 Apr

12:37 a.m.

On Wed, Apr 6, 2011 at 4:54 AM, Jelle van der Waa wrote:

...

On Tue, 2011-04-05 at 20:16 +0530, Madhurya Kakati wrote:

...
Please, someone restart powerpill development. It was awesome. I had to remove it in order to upgrade to pacman 3.5 but now I can't find it anywhere.

On 3/28/11, Corrado Primier wrote:

...
2011/3/28 Nicolas Bigaouette :

...
I guess it would make sense if your own bandwidth is bigger then the mirror's...

It happens with my ISP that the single connection bandwidth looks to be capped. I have 8Mbit but a single transfer rarely exceeds 2Mbit, unless it is very late night or Sunday morning :) Powerpill allowed me to fully use my bandwidth. I recognize it's a bad traffic shaping problem (and powerpill had its shortcomings too), but it solved it for me.

Corrado

The powerpill source is probably still available, so start hacking ;)

More to the point, see previous conversations on it both here and in the bbs, this has been covered multiple times.

Jeff Cook

7 Apr 7 Apr

8:25 a.m.

On Tue, Apr 5, 2011 at 8:46 AM, Madhurya Kakati wrote:

...

Please, someone restart powerpill development. It was awesome. I had to remove it in order to upgrade to pacman 3.5 but now I can't find it anywhere.

The developer's post on the forums indicates that he's been working on a far better replacement, but doesn't have the free time right now to complete it. Perhaps his priorities could be changed if some cash was on the table, perhaps not.

Brendan Long

4 Apr 4 Apr

6 a.m.

On 03/28/2011 03:43 AM, Cédric Girard wrote:

...

On Mon, Mar 28, 2011 at 11:07 AM, Oon-Ee Ng wrote:

...
If you have 10 files to download, powerpill allows for 1 file from mirror A, another from mirror B, and chunks of that large 68MB file from mirrors C, D, and E at the same time.

With the other solutions, you'd still wait for file 1 to finish downloading before downloading file 2.

I understand this as being more flexible. But bandwith-wise, I do not see why the Powerpill solution is more efficient. Or maybe this come useful only when downloading small files where file content transfert itself is negligible compared to connection opening and other protocol handling...

For me the benefit came in two parts: * When you download a bunch of small files, most of your "download time" is actually just starting connections. Powerpill starts a bunch of connections at once so you can actually use your bandwidth. * A lot of Arch's mirrors are really slow. Powerpill lets you not worry about the speed of individual mirrors, because it's only their combined speed that matters.

Isaac Dupree

6:35 a.m.

On 04/04/11 00:00, Brendan Long wrote:

...

On 03/28/2011 03:43 AM, Cédric Girard wrote:

...
On Mon, Mar 28, 2011 at 11:07 AM, Oon-Ee Ng wrote:

...
If you have 10 files to download, powerpill allows for 1 file from mirror A, another from mirror B, and chunks of that large 68MB file from mirrors C, D, and E at the same time.

With the other solutions, you'd still wait for file 1 to finish downloading before downloading file 2.

I understand this as being more flexible. But bandwith-wise, I do not see why the Powerpill solution is more efficient. Or maybe this come useful only when downloading small files where file content transfert itself is negligible compared to connection opening and other protocol handling...

For me the benefit came in two parts:

* When you download a bunch of small files, most of your "download time" is actually just starting connections. Powerpill starts a bunch of connections at once so you can actually use your bandwidth. * A lot of Arch's mirrors are really slow. Powerpill lets you not worry about the speed of individual mirrors, because it's only their combined speed that matters.

I found both of these are heavily mirror-dependent (and occasionally a given mirror changes to be worse or better than it used to be). It's dramatic enough that it feels like "some have problems, and some don't". I've got a good mirrorlist currently... check https://www.archlinux.de/?page=MirrorStatus , and I think there was a command-line tool... also the kernel.org one likely works well everywhere (?) so if overwhelmed by the options could check if that changes anything - http://mirrors.kernel.org/archlinux/$repo/os/$arch -Isaac

4773

Age (days ago)

4785

Last active (days ago)

List overview

Download

15 comments

11 participants

participants (11)

Brendan Long
cantabile
Corrado Primier
Cédric Girard
Isaac Dupree
Jeff Cook
Jelle van der Waa
Madhurya Kakati
Marek Otahal
Nicolas Bigaouette
Oon-Ee Ng