[arch-releng] Bittorrent with webseeds

newer
[arch-releng] [PATCH] Add a sleep...

Pierre Schmitz

23 Jan 2009 23 Jan '09

11:31 a.m.

Hi engineers, while creating a torrent with Ktorrent I noticed an option called webseeds. I have found the following site dealing with this feature: http://getright.com/seedtorrent.html In short: adding several webseed url the client will not only downlaod pieces from other pears but from ordinary http or ftp mirrors. This should increase the initial speed a lot and even if the user cannot use bittorrent due to firewall restriction, he still can download from several mirrors at the same time. I would like to try this out with our new iso; what do you think about it? Pierre -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de

Show replies by date

Gerhard Brauer

23 Jan 23 Jan

12:29 p.m.

Am Fri, 23 Jan 2009 11:31:08 +0100 schrieb Pierre Schmitz :

...

In short: adding several webseed url the client will not only downlaod pieces from other pears but from ordinary http or ftp mirrors. This should increase the initial speed a lot and even if the user cannot use bittorrent due to firewall restriction, he still can download from several mirrors at the same time.

I would like to try this out with our new iso; what do you think about it?

Do we have any statistic or experience how often people uses torrents instead downloading the iso/img from http/ftp mirrors directly? Maybe I'm a little too old for this technique, but make torrents not only sense when the download-source couldn't be stored completely on an server (music,warez,...). Or our FrosCon ISO, which we don't uploaded to our normal mirrors? But if i could download a file from a mirror (which fills my downlink bandwith), why using a torrent? On the other hand: if adding our normal mirrors to this technique speeds up the torrent download: let's try it...

...

Pierre

Gerhard

Dieter Plaetinck

6:10 p.m.

Gerhard Brauer wrote:

...

Am Fri, 23 Jan 2009 11:31:08 +0100 schrieb Pierre Schmitz :

...
In short: adding several webseed url the client will not only downlaod pieces from other pears but from ordinary http or ftp mirrors. This should increase the initial speed a lot and even if the user cannot use bittorrent due to firewall restriction, he still can download from several mirrors at the same time.

I would like to try this out with our new iso; what do you think about it?

I looked at this only briefly. If there is a commonly accepted, backwards-compatible (not breaking other clients) and open spec for this, then go for it. this looks cool.

...

Do we have any statistic or experience how often people uses torrents instead downloading the iso/img from http/ftp mirrors directly?

Maybe I'm a little too old for this technique, but make torrents not only sense when the download-source couldn't be stored completely on an server (music,warez,...). Or our FrosCon ISO, which we don't uploaded to our normal mirrors?

bittorrent is good for speed (if enough people are seeding the file) too, unless you can saturate your connection from a mirror.

...

But if i could download a file from a mirror (which fills my downlink bandwith), why using a torrent?

On the other hand: if adding our normal mirrors to this technique speeds up the torrent download: let's try it...

...
Pierre

Gerhard

Dieter

Aaron Griffin

6:46 p.m.

On Fri, Jan 23, 2009 at 4:31 AM, Pierre Schmitz wrote:

...

Hi engineers,

while creating a torrent with Ktorrent I noticed an option called webseeds. I have found the following site dealing with this feature: http://getright.com/seedtorrent.html

In short: adding several webseed url the client will not only downlaod pieces from other pears but from ordinary http or ftp mirrors. This should increase the initial speed a lot and even if the user cannot use bittorrent due to firewall restriction, he still can download from several mirrors at the same time.

I would like to try this out with our new iso; what do you think about it?

Yes, on one condition - if you do the generation and tracker uploads of the torrent files. I did it once and it was a big pain. :)

Pierre Schmitz

24 Jan 24 Jan

12:03 a.m.

Am Freitag 23 Januar 2009 18:46:49 schrieb Aaron Griffin:

...

Yes, on one condition - if you do the generation and tracker uploads of the torrent files. I did it once and it was a big pain. :)

I can do it (I did it several times before). But: I cannot do it before Monday evening and it's usefull to setup the torrents at least one day before the announcement. -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de

Grigorios Bouzakis

12:18 a.m.

On Sat, Jan 24, 2009 at 12:03:57AM +0100, Pierre Schmitz wrote:

...

Am Freitag 23 Januar 2009 18:46:49 schrieb Aaron Griffin:

...
Yes, on one condition - if you do the generation and tracker uploads of the torrent files. I did it once and it was a big pain. :)

I can do it (I did it several times before). But: I cannot do it before Monday evening and it's usefull to setup the torrents at least one day before the announcement.

Sorry for quoting the wrong part of the discussion, but does anyone have any knowledge regarding creation of torrent trackers? IMO it would be preferable to distribute ALL ISOs/IMGs even the alpha/betas using torrents with the least server load as possible. Many distributions have small torrent trackers set up. Maybe its time to have that in Arch too. Especially since theres an Official Release Team now, which plans on providing as much as possible frequent updates. FTR i had suggested the same thing around 2 years ago, but the discussion didnt lead anywhere. Maybe things are more mature now. Im pretty sure that its not so difficult to set up. Greg

Steffen Bönigk

10:28 a.m.

Grigorios Bouzakis schrieb:

...

On Sat, Jan 24, 2009 at 12:03:57AM +0100, Pierre Schmitz wrote:

...
Am Freitag 23 Januar 2009 18:46:49 schrieb Aaron Griffin:

...
Yes, on one condition - if you do the generation and tracker uploads of the torrent files. I did it once and it was a big pain. :) I can do it (I did it several times before). But: I cannot do it before Monday evening and it's usefull to setup the torrents at least one day before the announcement.

Sorry for quoting the wrong part of the discussion, but does anyone have any knowledge regarding creation of torrent trackers? IMO it would be preferable to distribute ALL ISOs/IMGs even the alpha/betas using torrents with the least server load as possible. Many distributions have small torrent trackers set up. Maybe its time to have that in Arch too. Especially since theres an Official Release Team now, which plans on providing as much as possible frequent updates. FTR i had suggested the same thing around 2 years ago, but the discussion didnt lead anywhere. Maybe things are more mature now. Im pretty sure that its not so difficult to set up.

Greg

Hello, to set up a torrent tracker you may want to try http://phpbttrkplus.sourceforge.net/ Steffen

Grigorios Bouzakis

5:02 p.m.

On Sat, Jan 24, 2009 at 10:28:46AM +0100, Steffen Bönigk wrote:

...

Grigorios Bouzakis schrieb:

...
Sorry for quoting the wrong part of the discussion, but does anyone have any knowledge regarding creation of torrent trackers? IMO it would be preferable to distribute ALL ISOs/IMGs even the alpha/betas using torrents with the least server load as possible. Many distributions have small torrent trackers set up. Maybe its time to have that in Arch too. Especially since theres an Official Release Team now, which plans on providing as much as possible frequent updates. FTR i had suggested the same thing around 2 years ago, but the discussion didnt lead anywhere. Maybe things are more mature now. Im pretty sure that its not so difficult to set up.

Greg

Hello, to set up a torrent tracker you may want to try http://phpbttrkplus.sourceforge.net/

Steffen

I was thinking more of something like http://www.whitsoftdev.com/opentracker/ . Statistics arent needed. Just a plain simple tracker. "OpenTracker is a simplistic, lightweight, standards-compliant BitTorrent peer tracker. It is written in PHP and requires only a single MySQL table to function. It does not create or use files on disk, nor is it a continuously executing process, making it ideal for virtual web hosts. Compared with other popular trackers, OpenTracker is limited. It provides no interface for searching or viewing files or statistics. Rather, it is purely a tracker, plain and simple, which in many cases is just what is wanted." Greg

Dieter Plaetinck

5:27 p.m.

On Sat, 24 Jan 2009 18:02:25 +0200 Grigorios Bouzakis wrote:

...

On Sat, Jan 24, 2009 at 10:28:46AM +0100, Steffen Bönigk wrote:

...
Grigorios Bouzakis schrieb:

...
Sorry for quoting the wrong part of the discussion, but does anyone have any knowledge regarding creation of torrent trackers? IMO it would be preferable to distribute ALL ISOs/IMGs even the alpha/betas using torrents with the least server load as possible. Many distributions have small torrent trackers set up. Maybe its time to have that in Arch too. Especially since theres an Official Release Team now, which plans on providing as much as possible frequent updates. FTR i had suggested the same thing around 2 years ago, but the discussion didnt lead anywhere. Maybe things are more mature now. Im pretty sure that its not so difficult to set up.

Greg

Hello, to set up a torrent tracker you may want to try http://phpbttrkplus.sourceforge.net/

Steffen

I was thinking more of something like http://www.whitsoftdev.com/opentracker/ . Statistics arent needed. Just a plain simple tracker.

Personally I like statistics :) amount of seeders, leechers, and if possible: amount of people who completed a download and/or (roughly) the amount of p2p traffic. linuxtracker also seems to show some statistics. assuming linuxtracker has a high uptime (it was just down for a while but that can be coincidence) I don't see what's wrong with using it. Pierre: does linuxtracker show the amount of people who completed a download and/or (roughly) the amount of p2p traffic.

...

"OpenTracker is a simplistic, lightweight, standards-compliant BitTorrent peer tracker. It is written in PHP and requires only a single MySQL table to function. It does not create or use files on disk, nor is it a continuously executing process, making it ideal for virtual web hosts. Compared with other popular trackers, OpenTracker is limited. It provides no interface for searching or viewing files or statistics. Rather, it is purely a tracker, plain and simple, which in many cases is just what is wanted."

Greg

"It tracks torrent swarms blindly without first requiring seeders to upload torrent files. In order to use the tracker, one only has to create a torrent with the tracker's announce URL and begin seeding that torrent. No preparation of the tracker is needed or possible. " Huh? does this mean that everyone could (ab)use our tracker, using it for torrents that have nothing to do with Arch? Dieter

Grigorios Bouzakis

6:05 p.m.

On Sat, Jan 24, 2009 at 05:27:34PM +0100, Dieter Plaetinck wrote:

...

On Sat, 24 Jan 2009 18:02:25 +0200 Grigorios Bouzakis wrote:

...
On Sat, Jan 24, 2009 at 10:28:46AM +0100, Steffen Bönigk wrote:

...
Grigorios Bouzakis schrieb:

...
Sorry for quoting the wrong part of the discussion, but does anyone have any knowledge regarding creation of torrent trackers? IMO it would be preferable to distribute ALL ISOs/IMGs even the alpha/betas using torrents with the least server load as possible. Many distributions have small torrent trackers set up. Maybe its time to have that in Arch too. Especially since theres an Official Release Team now, which plans on providing as much as possible frequent updates. FTR i had suggested the same thing around 2 years ago, but the discussion didnt lead anywhere. Maybe things are more mature now. Im pretty sure that its not so difficult to set up.

Greg

Hello, to set up a torrent tracker you may want to try http://phpbttrkplus.sourceforge.net/

Steffen

I was thinking more of something like http://www.whitsoftdev.com/opentracker/ . Statistics arent needed. Just a plain simple tracker.

Personally I like statistics :) amount of seeders, leechers, and if possible: amount of people who completed a download and/or (roughly) the amount of p2p traffic.

linuxtracker also seems to show some statistics. assuming linuxtracker has a high uptime (it was just down for a while but that can be coincidence) I don't see what's wrong with using it.

Pierre: does linuxtracker show the amount of people who completed a download and/or (roughly) the amount of p2p traffic.

What you describe sounds like a torrent site. That wasnt what i suggested. See for example how Slackware, Debian & Fedora distribute their torrents.

...

...
"OpenTracker is a simplistic, lightweight, standards-compliant BitTorrent peer tracker. It is written in PHP and requires only a single MySQL table to function. It does not create or use files on disk, nor is it a continuously executing process, making it ideal for virtual web hosts. Compared with other popular trackers, OpenTracker is limited. It provides no interface for searching or viewing files or statistics. Rather, it is purely a tracker, plain and simple, which in many cases is just what is wanted."

Greg

"It tracks torrent swarms blindly without first requiring seeders to upload torrent files. In order to use the tracker, one only has to create a torrent with the tracker's announce URL and begin seeding that torrent. No preparation of the tracker is needed or possible. "

Huh? does this mean that everyone could (ab)use our tracker, using it for torrents that have nothing to do with Arch?

Yeah, that sounds like it. If thats the case this behaviour is unwanted. The application suggestion was just an example. Maybe we could find out what the above mentioned distributions use. Greg

Dan McGee

6:09 p.m.

On Sat, Jan 24, 2009 at 10:27 AM, Dieter Plaetinck wrote:

...

On Sat, 24 Jan 2009 18:02:25 +0200 Grigorios Bouzakis wrote:

...
On Sat, Jan 24, 2009 at 10:28:46AM +0100, Steffen Bönigk wrote:

...
Grigorios Bouzakis schrieb:

...
Sorry for quoting the wrong part of the discussion, but does anyone have any knowledge regarding creation of torrent trackers? IMO it would be preferable to distribute ALL ISOs/IMGs even the alpha/betas using torrents with the least server load as possible. Many distributions have small torrent trackers set up. Maybe its time to have that in Arch too. Especially since theres an Official Release Team now, which plans on providing as much as possible frequent updates. FTR i had suggested the same thing around 2 years ago, but the discussion didnt lead anywhere. Maybe things are more mature now. Im pretty sure that its not so difficult to set up.

Greg

Hello, to set up a torrent tracker you may want to try http://phpbttrkplus.sourceforge.net/

Steffen

I was thinking more of something like http://www.whitsoftdev.com/opentracker/ . Statistics arent needed. Just a plain simple tracker.

Personally I like statistics :) amount of seeders, leechers, and if possible: amount of people who completed a download and/or (roughly) the amount of p2p traffic.

linuxtracker also seems to show some statistics. assuming linuxtracker has a high uptime (it was just down for a while but that can be coincidence) I don't see what's wrong with using it.

Pierre: does linuxtracker show the amount of people who completed a download and/or (roughly) the amount of p2p traffic.

...
"OpenTracker is a simplistic, lightweight, standards-compliant BitTorrent peer tracker. It is written in PHP and requires only a single MySQL table to function. It does not create or use files on disk, nor is it a continuously executing process, making it ideal for virtual web hosts. Compared with other popular trackers, OpenTracker is limited. It provides no interface for searching or viewing files or statistics. Rather, it is purely a tracker, plain and simple, which in many cases is just what is wanted."

Greg

"It tracks torrent swarms blindly without first requiring seeders to upload torrent files. In order to use the tracker, one only has to create a torrent with the tracker's announce URL and begin seeding that torrent. No preparation of the tracker is needed or possible. "

Huh? does this mean that everyone could (ab)use our tracker, using it for torrents that have nothing to do with Arch?

Lets take a quick step back here. We've discussed this in the past, so I'll try to bring that to the table. We need a tracker: 1) that only seeds torrents we tell it to, not anything else 2) is relatively lightweight 3) doesn't suck Gentoo uses PHPBTTracker+ 2.1. Debian and Ubuntu use BitTornado. I took a brief look at both of these products this morning and was not impressed- they seem old and a bit out of date, underdeveloped, and just don't do things "right" in my humble opinion. PHPBTTracker has some crazy caching idea where they try to reinvent the wheel and appear to fail miserably. BitTornado looks more usable, although it hasn't been touched in some time and the codebase is not so clean. -Dan

Dieter Plaetinck

7:40 p.m.

On Sat, 24 Jan 2009 11:09:24 -0600 Dan McGee wrote:

...

On Sat, Jan 24, 2009 at 10:27 AM, Dieter Plaetinck wrote:

...
On Sat, 24 Jan 2009 18:02:25 +0200 Grigorios Bouzakis wrote:

...
On Sat, Jan 24, 2009 at 10:28:46AM +0100, Steffen Bönigk wrote:

...
Grigorios Bouzakis schrieb:

...
Sorry for quoting the wrong part of the discussion, but does anyone have any knowledge regarding creation of torrent trackers? IMO it would be preferable to distribute ALL ISOs/IMGs even the alpha/betas using torrents with the least server load as possible. Many distributions have small torrent trackers set up. Maybe its time to have that in Arch too. Especially since theres an Official Release Team now, which plans on providing as much as possible frequent updates. FTR i had suggested the same thing around 2 years ago, but the discussion didnt lead anywhere. Maybe things are more mature now. Im pretty sure that its not so difficult to set up.

Greg

Hello, to set up a torrent tracker you may want to try http://phpbttrkplus.sourceforge.net/

Steffen

I was thinking more of something like http://www.whitsoftdev.com/opentracker/ . Statistics arent needed. Just a plain simple tracker.

Personally I like statistics :) amount of seeders, leechers, and if possible: amount of people who completed a download and/or (roughly) the amount of p2p traffic.

linuxtracker also seems to show some statistics. assuming linuxtracker has a high uptime (it was just down for a while but that can be coincidence) I don't see what's wrong with using it.

Pierre: does linuxtracker show the amount of people who completed a download and/or (roughly) the amount of p2p traffic.

...
"OpenTracker is a simplistic, lightweight, standards-compliant BitTorrent peer tracker. It is written in PHP and requires only a single MySQL table to function. It does not create or use files on disk, nor is it a continuously executing process, making it ideal for virtual web hosts. Compared with other popular trackers, OpenTracker is limited. It provides no interface for searching or viewing files or statistics. Rather, it is purely a tracker, plain and simple, which in many cases is just what is wanted."

Greg

"It tracks torrent swarms blindly without first requiring seeders to upload torrent files. In order to use the tracker, one only has to create a torrent with the tracker's announce URL and begin seeding that torrent. No preparation of the tracker is needed or possible. "

Huh? does this mean that everyone could (ab)use our tracker, using it for torrents that have nothing to do with Arch?

Lets take a quick step back here. We've discussed this in the past, so I'll try to bring that to the table.

We need a tracker: 1) that only seeds torrents we tell it to, not anything else 2) is relatively lightweight 3) doesn't suck

Gentoo uses PHPBTTracker+ 2.1. Debian and Ubuntu use BitTornado. I took a brief look at both of these products this morning and was not impressed- they seem old and a bit out of date, underdeveloped, and just don't do things "right" in my humble opinion. PHPBTTracker has some crazy caching idea where they try to reinvent the wheel and appear to fail miserably. BitTornado looks more usable, although it hasn't been touched in some time and the codebase is not so clean.

-Dan

So, is there anything wrong with linuxtracker.org ? It's not under our control but is that a problem? Dieter

Grigorios Bouzakis

7:44 p.m.

On Sat, Jan 24, 2009 at 8:40 PM, Dieter Plaetinck wrote:

...

On Sat, 24 Jan 2009 11:09:24 -0600 Dan McGee wrote:

...
We need a tracker: 1) that only seeds torrents we tell it to, not anything else 2) is relatively lightweight 3) doesn't suck

Gentoo uses PHPBTTracker+ 2.1. Debian and Ubuntu use BitTornado. I took a brief look at both of these products this morning and was not impressed- they seem old and a bit out of date, underdeveloped, and just don't do things "right" in my humble opinion. PHPBTTracker has some crazy caching idea where they try to reinvent the wheel and appear to fail miserably. BitTornado looks more usable, although it hasn't been touched in some time and the codebase is not so clean.

-Dan

So, is there anything wrong with linuxtracker.org ? It's not under our control but is that a problem?

Dieter

For example, & for obvious reasons, hosting the alphas on linuxtracker isnt an option. Downloading anything from archlinux.org is slow. -- Greg

Pierre Schmitz

2:11 p.m.

Am Samstag 24 Januar 2009 00:18:15 schrieb Grigorios Bouzakis:

...

On Sat, Jan 24, 2009 at 12:03:57AM +0100, Pierre Schmitz wrote:

...
Am Freitag 23 Januar 2009 18:46:49 schrieb Aaron Griffin:

...
Yes, on one condition - if you do the generation and tracker uploads of the torrent files. I did it once and it was a big pain. :)

I can do it (I did it several times before). But: I cannot do it before Monday evening and it's usefull to setup the torrents at least one day before the announcement.

Sorry for quoting the wrong part of the discussion, but does anyone have any knowledge regarding creation of torrent trackers? IMO it would be preferable to distribute ALL ISOs/IMGs even the alpha/betas using torrents with the least server load as possible. Many distributions have small torrent trackers set up. Maybe its time to have that in Arch too. Especially since theres an Official Release Team now, which plans on providing as much as possible frequent updates. FTR i had suggested the same thing around 2 years ago, but the discussion didnt lead anywhere. Maybe things are more mature now. Im pretty sure that its not so difficult to set up.

Greg

The point is that such a tracker could increase the load of our server an more important someone has to maintain and update it. So for the next ISO I'll upload it to linuxtracker.org. -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de

Dieter Plaetinck

3:17 p.m.

On Sat, 24 Jan 2009 14:11:36 +0100 Pierre Schmitz wrote:

...

Am Samstag 24 Januar 2009 00:18:15 schrieb Grigorios Bouzakis:

...
On Sat, Jan 24, 2009 at 12:03:57AM +0100, Pierre Schmitz wrote:

...
Am Freitag 23 Januar 2009 18:46:49 schrieb Aaron Griffin:

...
Yes, on one condition - if you do the generation and tracker uploads of the torrent files. I did it once and it was a big pain. :)

I can do it (I did it several times before). But: I cannot do it before Monday evening and it's usefull to setup the torrents at least one day before the announcement.

Sorry for quoting the wrong part of the discussion, but does anyone have any knowledge regarding creation of torrent trackers? IMO it would be preferable to distribute ALL ISOs/IMGs even the alpha/betas using torrents with the least server load as possible. Many distributions have small torrent trackers set up. Maybe its time to have that in Arch too. Especially since theres an Official Release Team now, which plans on providing as much as possible frequent updates. FTR i had suggested the same thing around 2 years ago, but the discussion didnt lead anywhere. Maybe things are more mature now. Im pretty sure that its not so difficult to set up.

Greg

The point is that such a tracker could increase the load of our server an more important someone has to maintain and update it. So for the next ISO I'll upload it to linuxtracker.org.

Linuxtracker.org is down right now :( Dieter

Aaron Griffin

12:28 a.m.

On Fri, Jan 23, 2009 at 5:03 PM, Pierre Schmitz wrote:

...

Am Freitag 23 Januar 2009 18:46:49 schrieb Aaron Griffin:

...
Yes, on one condition - if you do the generation and tracker uploads of the torrent files. I did it once and it was a big pain. :)

I can do it (I did it several times before). But: I cannot do it before Monday evening and it's usefull to setup the torrents at least one day before the announcement.

Yeah, we may be a tad off on our Friday target date. Gerhard, could you possibly build some of the ISOs? Is everything in order? I am adding dmraid to the package list right now. I won't be home until late tonight (now + 6 or 7 hours), and I can get to building then - I'm actually building the grub based ISOs now (yay ssh and screen).

Gerhard Brauer

11:20 a.m.

Am Fri, 23 Jan 2009 17:28:02 -0600 schrieb Aaron Griffin :

...

Gerhard, could you possibly build some of the ISOs? Is everything in order? I am adding dmraid to the package list right now.

I won't be home until late tonight (now + 6 or 7 hours), and I can get to building then - I'm actually building the grub based ISOs now (yay ssh and screen).

Yes, sure. I build today (Sat.) all i686 ISOs and upload them to http://users.archlinux.de/~gerbra/iso/ I try also building the i686 USB images (never tried this). SO you could pick up what you need. I start building the isolinux images when you say: i build grub ISOs. Gerhard

Gerhard Brauer

5:57 p.m.

Am Sat, 24 Jan 2009 11:20:03 +0100 schrieb Gerhard Brauer :

...

Am Fri, 23 Jan 2009 17:28:02 -0600 schrieb Aaron Griffin :

...
Gerhard, could you possibly build some of the ISOs? Is everything in order? I am adding dmraid to the package list right now.

I won't be home until late tonight (now + 6 or 7 hours), and I can get to building then - I'm actually building the grub based ISOs now (yay ssh and screen).

Yes, sure. I build today (Sat.) all i686 ISOs and upload them to http://users.archlinux.de/~gerbra/iso/ I try also building the i686 USB images (never tried this). SO you could pick up what you need. I start building the isolinux images when you say: i build grub ISOs.

OK, I've built the i686 ISOs and uploaded them. From my view this could be the final ISOs (dmraid is added, newest mkinitcpio is on board, they all boot...) I have not built the USB images (cause the stopper FS#12896), but I've uploaded a test image for USB which probably have fixed this problem. Myself could not test this. http://users.archlinux.de/~gerbra/iso/test-usb/archlinux-2009.01-1-ftp-i686.... http://users.archlinux.de/~gerbra/iso/test-usb/md5sum.txt I also put these links to the bugreport, so that reporting users could test them. Gerhard

Dieter Plaetinck

7:37 p.m.

On Sat, 24 Jan 2009 17:57:19 +0100 Gerhard Brauer wrote:

...

Am Sat, 24 Jan 2009 11:20:03 +0100 schrieb Gerhard Brauer :

...
Am Fri, 23 Jan 2009 17:28:02 -0600 schrieb Aaron Griffin :

...
Gerhard, could you possibly build some of the ISOs? Is everything in order? I am adding dmraid to the package list right now.

I won't be home until late tonight (now + 6 or 7 hours), and I can get to building then - I'm actually building the grub based ISOs now (yay ssh and screen).

Yes, sure. I build today (Sat.) all i686 ISOs and upload them to http://users.archlinux.de/~gerbra/iso/ I try also building the i686 USB images (never tried this). SO you could pick up what you need. I start building the isolinux images when you say: i build grub ISOs.

Can you check that /src/core/pkg exists on your core iso? when i built it (latest archiso from git, sudo make all-iso), the core iso does not contain an /src directory

...

OK, I've built the i686 ISOs and uploaded them. From my view this could be the final ISOs (dmraid is added, newest mkinitcpio is on board, they all boot...)

I have not built the USB images (cause the stopper FS#12896), but I've uploaded a test image for USB which probably have fixed this problem. Myself could not test this. http://users.archlinux.de/~gerbra/iso/test-usb/archlinux-2009.01-1-ftp-i686.... http://users.archlinux.de/~gerbra/iso/test-usb/md5sum.txt I also put these links to the bugreport, so that reporting users could test them.

Gerhard

Dieter

Dieter Plaetinck

8:52 p.m.

On Sat, 24 Jan 2009 17:57:19 +0100 Gerhard Brauer wrote:

...

Am Sat, 24 Jan 2009 11:20:03 +0100 schrieb Gerhard Brauer :

...
Am Fri, 23 Jan 2009 17:28:02 -0600 schrieb Aaron Griffin :

...
Gerhard, could you possibly build some of the ISOs? Is everything in order? I am adding dmraid to the package list right now.

I won't be home until late tonight (now + 6 or 7 hours), and I can get to building then - I'm actually building the grub based ISOs now (yay ssh and screen).

Yes, sure. I build today (Sat.) all i686 ISOs and upload them to http://users.archlinux.de/~gerbra/iso/ I try also building the i686 USB images (never tried this). SO you could pick up what you need. I start building the isolinux images when you say: i build grub ISOs.

OK, I've built the i686 ISOs and uploaded them. From my view this could be the final ISOs (dmraid is added, newest mkinitcpio is on board, they all boot...)

Hey uhm.. I've just discovered a quite big bug in aif: it used the live pacman at some point instead of the one for the target system without doing pacman -Sy first, while pacman -Sy is done (only) for the target system. Two options: 1) we don't rebuild and keep the isos like they are. aif will break during installation in all cases, unless the user intervenes manually and does a "pacman -Sy" in a terminal somewhere between configuring the network and selecting the packages. (note that you need a working internet connection during installation) 2) i've fixed the code, retagged it (2009.01.24) and retested. it works now. if ok for you guys: Aaron, can you update your PKGBUILD/the package and can we rebuild the isos? i can even build the iso's and upload them if you want (though I'll need an ftp account somewhere then) Pierre said that he can't fix the torrents until monday evening anyway so I guess we can pick option 2. There's a quite peculiar reason this issue pops up so late btw: I've always been working and testing with an install cd on which I installed git and fakeroot myself (pacman -Sy git fakeroot) so I had a synced database. now is the first time I am testing with an installcd that doesn't have a synced database. :o Dieter

...

I have not built the USB images (cause the stopper FS#12896), but I've uploaded a test image for USB which probably have fixed this problem. Myself could not test this. http://users.archlinux.de/~gerbra/iso/test-usb/archlinux-2009.01-1-ftp-i686.... http://users.archlinux.de/~gerbra/iso/test-usb/md5sum.txt I also put these links to the bugreport, so that reporting users could test them.

Gerhard

Dieter Plaetinck

8:54 p.m.

On Sat, 24 Jan 2009 20:52:46 +0100 Dieter Plaetinck wrote:

...

On Sat, 24 Jan 2009 17:57:19 +0100 Gerhard Brauer wrote:

...
Am Sat, 24 Jan 2009 11:20:03 +0100 schrieb Gerhard Brauer :

...
Am Fri, 23 Jan 2009 17:28:02 -0600 schrieb Aaron Griffin :

...
Gerhard, could you possibly build some of the ISOs? Is everything in order? I am adding dmraid to the package list right now.

I won't be home until late tonight (now + 6 or 7 hours), and I can get to building then - I'm actually building the grub based ISOs now (yay ssh and screen).

Yes, sure. I build today (Sat.) all i686 ISOs and upload them to http://users.archlinux.de/~gerbra/iso/ I try also building the i686 USB images (never tried this). SO you could pick up what you need. I start building the isolinux images when you say: i build grub ISOs.

OK, I've built the i686 ISOs and uploaded them. From my view this could be the final ISOs (dmraid is added, newest mkinitcpio is on board, they all boot...)

Hey uhm.. I've just discovered a quite big bug in aif: it used the live pacman at some point instead of the one for the target system without doing pacman -Sy first, while pacman -Sy is done (only) for the target system.

Two options: 1) we don't rebuild and keep the isos like they are. aif will break during installation in all cases, unless the user intervenes manually and does a "pacman -Sy" in a terminal somewhere between configuring the network and selecting the packages. (note that you need a working internet connection during installation) 2) i've fixed the code, retagged it (2009.01.24) and retested. it works now. if ok for you guys: Aaron, can you update your PKGBUILD/the package and can we rebuild the isos? i can even build the iso's and upload them if you want (though I'll need an ftp account somewhere then)

Pierre said that he can't fix the torrents until monday evening anyway so I guess we can pick option 2.

There's a quite peculiar reason this issue pops up so late btw: I've always been working and testing with an install cd on which I installed git and fakeroot myself (pacman -Sy git fakeroot) so I had a synced database. now is the first time I am testing with an installcd that doesn't have a synced database. :o

Dieter

Oh btw, the diff is here: http://github.com/Dieterbe/aif/commit/d9ffd2d3beccc4d768bdcaf7634a7de3f50518...

...

...
I have not built the USB images (cause the stopper FS#12896), but I've uploaded a test image for USB which probably have fixed this problem. Myself could not test this. http://users.archlinux.de/~gerbra/iso/test-usb/archlinux-2009.01-1-ftp-i686.... http://users.archlinux.de/~gerbra/iso/test-usb/md5sum.txt I also put these links to the bugreport, so that reporting users could test them.

Gerhard

Dan McGee

8:58 p.m.

On Sat, Jan 24, 2009 at 1:52 PM, Dieter Plaetinck wrote:

...

On Sat, 24 Jan 2009 17:57:19 +0100 Gerhard Brauer wrote:

...
Am Sat, 24 Jan 2009 11:20:03 +0100 schrieb Gerhard Brauer :

...
Am Fri, 23 Jan 2009 17:28:02 -0600 schrieb Aaron Griffin :

...
Gerhard, could you possibly build some of the ISOs? Is everything in order? I am adding dmraid to the package list right now.

I won't be home until late tonight (now + 6 or 7 hours), and I can get to building then - I'm actually building the grub based ISOs now (yay ssh and screen).

Yes, sure. I build today (Sat.) all i686 ISOs and upload them to http://users.archlinux.de/~gerbra/iso/ I try also building the i686 USB images (never tried this). SO you could pick up what you need. I start building the isolinux images when you say: i build grub ISOs.

OK, I've built the i686 ISOs and uploaded them. From my view this could be the final ISOs (dmraid is added, newest mkinitcpio is on board, they all boot...)

Hey uhm.. I've just discovered a quite big bug in aif: it used the live pacman at some point instead of the one for the target system without doing pacman -Sy first, while pacman -Sy is done (only) for the target system.

Two options: 1) we don't rebuild and keep the isos like they are. aif will break during installation in all cases, unless the user intervenes manually and does a "pacman -Sy" in a terminal somewhere between configuring the network and selecting the packages. (note that you need a working internet connection during installation) 2) i've fixed the code, retagged it (2009.01.24) and retested. it works now. if ok for you guys: Aaron, can you update your PKGBUILD/the package and can we rebuild the isos? i can even build the iso's and upload them if you want (though I'll need an ftp account somewhere then)

Pierre said that he can't fix the torrents until monday evening anyway so I guess we can pick option 2.

There's a quite peculiar reason this issue pops up so late btw: I've always been working and testing with an install cd on which I installed git and fakeroot myself (pacman -Sy git fakeroot) so I had a synced database. now is the first time I am testing with an installcd that doesn't have a synced database. :o

I didn't think these guys were final, are they? We go from alpha to release? This is a rather annoying issue that I really think should get addressed before pushing final ISOs, as it is a regression: http://bugs.archlinux.org/task/12936?project=6 -Dan

Dieter Plaetinck

10:08 p.m.

On Sat, 24 Jan 2009 13:58:05 -0600 Dan McGee wrote:

...

On Sat, Jan 24, 2009 at 1:52 PM, Dieter Plaetinck wrote:

...
On Sat, 24 Jan 2009 17:57:19 +0100 Gerhard Brauer wrote:

...
Am Sat, 24 Jan 2009 11:20:03 +0100 schrieb Gerhard Brauer :

...
Am Fri, 23 Jan 2009 17:28:02 -0600 schrieb Aaron Griffin :

...
Gerhard, could you possibly build some of the ISOs? Is everything in order? I am adding dmraid to the package list right now.

I won't be home until late tonight (now + 6 or 7 hours), and I can get to building then - I'm actually building the grub based ISOs now (yay ssh and screen).

Yes, sure. I build today (Sat.) all i686 ISOs and upload them to http://users.archlinux.de/~gerbra/iso/ I try also building the i686 USB images (never tried this). SO you could pick up what you need. I start building the isolinux images when you say: i build grub ISOs.

OK, I've built the i686 ISOs and uploaded them. From my view this could be the final ISOs (dmraid is added, newest mkinitcpio is on board, they all boot...)

Hey uhm.. I've just discovered a quite big bug in aif: it used the live pacman at some point instead of the one for the target system without doing pacman -Sy first, while pacman -Sy is done (only) for the target system.

Two options: 1) we don't rebuild and keep the isos like they are. aif will break during installation in all cases, unless the user intervenes manually and does a "pacman -Sy" in a terminal somewhere between configuring the network and selecting the packages. (note that you need a working internet connection during installation) 2) i've fixed the code, retagged it (2009.01.24) and retested. it works now. if ok for you guys: Aaron, can you update your PKGBUILD/the package and can we rebuild the isos? i can even build the iso's and upload them if you want (though I'll need an ftp account somewhere then)

Pierre said that he can't fix the torrents until monday evening anyway so I guess we can pick option 2.

There's a quite peculiar reason this issue pops up so late btw: I've always been working and testing with an install cd on which I installed git and fakeroot myself (pacman -Sy git fakeroot) so I had a synced database. now is the first time I am testing with an installcd that doesn't have a synced database. :o

I didn't think these guys were final, are they? We go from alpha to release?

heh.. I thought these iso's were beta :) Another reason why clearer naming could be useful ( http://www.archlinux.org/pipermail/arch-releng/2009-January/000046.html )

...

This is a rather annoying issue that I really think should get addressed before pushing final ISOs, as it is a regression: http://bugs.archlinux.org/task/12936?project=6

I hope that we can really use the "due in version" feature of flyspray, then we can use the roadmap feature to keep track of what needs to be done. http://bugs.archlinux.org/roadmap/proj6 Regarding your issue, I'm afraid I don't know much about that stuff. But I'm sure others understand what's going on.

...

-Dan

Dieter

Dieter Plaetinck

11:29 a.m.

Aaron Griffin wrote:

...

On Fri, Jan 23, 2009 at 5:03 PM, Pierre Schmitz wrote:

...
Am Freitag 23 Januar 2009 18:46:49 schrieb Aaron Griffin:

...
Yes, on one condition - if you do the generation and tracker uploads of the torrent files. I did it once and it was a big pain. :)

I can do it (I did it several times before). But: I cannot do it before Monday evening and it's usefull to setup the torrents at least one day before the announcement.

Yeah, we may be a tad off on our Friday target date.

Gerhard, could you possibly build some of the ISOs? Is everything in order? I am adding dmraid to the package list right now.

Aaron/Gerhard, can you share with us exactly how you build the iso's? So everyone can do it... (I would like to document this on the wiki too, I couldn't find a tutorial/wiki page/manpage) What I have: * git clone git://projects.archlinux.org/archiso.git * cd archiso/archiso * sudo make install (all of this to be replaced with 'pacman -Sy archiso' when a new archiso release is done, which is planned after 2009-01 release) cd ../configs/install-iso sudo make all-iso (or ftp-iso, or core-iso) questions: 1) any specific requirements on /etc/pacman.conf? (related to settings/available repos?) 2) how do you build iso's with only core or with core+testing available? by just (un)commenting testing from /etc/pacman.conf? 3) assuming that the relevant repositories are frozen, if I build an iso on my system, and someone else builds the iso on his system, will the iso's match perfectly? (eg same md5sums) I think it would be useful in the future we build locally first and test it, and then instead of uploading to gerolde, we can just rebuild it there?

...

I won't be home until late tonight (now + 6 or 7 hours), and I can get to building then - I'm actually building the grub based ISOs now (yay ssh and screen).

5609

Age (days ago)

5610

Last active (days ago)

List overview

Download

23 comments

7 participants

participants (7)

Aaron Griffin
Dan McGee
Dieter Plaetinck
Gerhard Brauer
Grigorios Bouzakis
Pierre Schmitz
Steffen Bönigk