[arch-releng] How to create a torrent for the ISO
Hi engineers, thanks to a new version of mktorrent it is now easy to create webseed enabled torrents. I have updated my wrapper script to use this instead of my evil ktorrent hack. This script can be found at http://git.archlinux.de/archlinux-torrent-utils/plain/mktorrent-archlinux You will need kmtorrent from http://mktorrent.sourceforge.net/; there is also a PKGBUILD. I would suggest to install this on sigurd and also the opentracker on it. This way the releng team can create those torrents right on the server and will also be able to add those to the trackers whitelist. Pierre -- Pierre Schmitz, http://users.archlinux.de/~pierre
Am Freitag, den 31.07.2009, 00:22 +0200 schrieb Pierre Schmitz:
I would suggest to install this on sigurd and also the opentracker on it. This way the releng team can create those torrents right on the server and will also be able to add those to the trackers whitelist.
+1. This could also help us to make some images (ex. the RC) more public without that they all get downloaded only from sigurd.... Aaron, do you agree?
Pierre
Gerhard
On Thu, Jul 30, 2009 at 5:22 PM, Pierre Schmitz<pierre@archlinux.de> wrote:
Hi engineers,
thanks to a new version of mktorrent it is now easy to create webseed enabled torrents. I have updated my wrapper script to use this instead of my evil ktorrent hack.
This script can be found at http://git.archlinux.de/archlinux-torrent-utils/plain/mktorrent-archlinux
You will need kmtorrent from http://mktorrent.sourceforge.net/; there is also a PKGBUILD.
I would suggest to install this on sigurd and also the opentracker on it. This way the releng team can create those torrents right on the server and will also be able to add those to the trackers whitelist.
Hey guys, I'd like to do this today. I'm currently building and installing opentracker (and libowfat). Pierre, do you happen to know much about the config? Do we get a web interface with this, or is it dumb?
Am Montag 03 August 2009 22:13:22 schrieb Aaron Griffin:
On Thu, Jul 30, 2009 at 5:22 PM, Pierre Schmitz<pierre@archlinux.de> wrote:
Hi engineers,
thanks to a new version of mktorrent it is now easy to create webseed enabled torrents. I have updated my wrapper script to use this instead of my evil ktorrent hack.
This script can be found at http://git.archlinux.de/archlinux-torrent-utils/plain/mktorrent-archlinux
You will need kmtorrent from http://mktorrent.sourceforge.net/; there is also a PKGBUILD.
I would suggest to install this on sigurd and also the opentracker on it. This way the releng team can create those torrents right on the server and will also be able to add those to the trackers whitelist.
Hey guys, I'd like to do this today.
I'm currently building and installing opentracker (and libowfat).
Pierre, do you happen to know much about the config? Do we get a web interface with this, or is it dumb?
Its a really KISS application. Here are some install instructions: After installing the package you might want to alter the rc script to use a dedicated user like opentracker isntead of nobody; this user does not need any rights nor does it need to write anything anywhere. Create a config file /etc/opentracker/config containing: tracker.rootdir /etc/opentracker access.whitelist whitelist access.stats 127.0.0.1 This virtually disables any stats page etc.. Maybe we want to enable it later. Then we need a whitelist file for all torrents we like to share in the same dir. I use this which includes the previous ones on those archbott images from tpowa. Everything after a whitespace will be ignored; so I used it for commenting for the recent entries: dc33a27b9ad0d97a1d2c536ae45561f507c25ad1 7434c995950fdf3c29bf90f46bc26defc4e9d2f6 5d49e09821539ce9753b5b8968633317b049fb74 5959847e44c24bd7b432db603e6a4b495a3ac192 b329016903efb29d7677f42bbc5f4a1f13f7a921 91f754453d87bf2956bd21dbc3bf4703aaaeb78c 72e23e8836e22f253dfd810b5138809b468e09f8 5ae22f7cef9b033f5e4830511c6f929ba672b9ec 8d8413b188224de03facf6c79258dcb2356cbcb5 8719f00b1927ef85cca616a292ab6e073183be05 97a5c0701ceee6598efe08ad8577513bdc341139 2009.04-1-archboot.iso 783668c0dd9bd9f73ec1e3fd2156727f33fec3d1 2009.05-1-archboot.iso 772AA65BDCF417F927D78E0A8CA27F9931D12214 2009.08-1-archboot.iso I recommend to add an extra dns-entry like tracker.archlinux.org for it. The default port should be 6969. So if this thing is running I just could change tracker.archlinux.de to the new IP and the old torrents will still work. -- Pierre Schmitz, http://users.archlinux.de/~pierre
On Mon, Aug 3, 2009 at 3:54 PM, Pierre Schmitz<pierre@archlinux.de> wrote:
Its a really KISS application. Here are some install instructions:
After installing the package you might want to alter the rc script to use a dedicated user like opentracker isntead of nobody; this user does not need any rights nor does it need to write anything anywhere.
Create a config file /etc/opentracker/config containing:
tracker.rootdir /etc/opentracker access.whitelist whitelist access.stats 127.0.0.1
I get the following with these config options. Similar error when using the full path: Warning: Can't open accesslist file: whitelist (but will try to create it later, if necessary and possible). PWD: /etc/opentracker
Then we need a whitelist file for all torrents we like to share in the same dir. I use this which includes the previous ones on those archbott images from tpowa. Everything after a whitespace will be ignored; so I used it for commenting for the recent entries:
So sha1sum file >> whitelist is all that is needed?
I recommend to add an extra dns-entry like tracker.archlinux.org for it. The default port should be 6969. So if this thing is running I just could change tracker.archlinux.de to the new IP and the old torrents will still work.
tracker.archlinux.org added, 6969 opened in iptables
Am Montag 03 August 2009 23:05:22 schrieb Aaron Griffin:
I get the following with these config options. Similar error when using the full path:
Warning: Can't open accesslist file: whitelist (but will try to create it later, if necessary and possible). PWD: /etc/opentracker
Maybe the opentracker user cannot read those files? How do you start it?
Then we need a whitelist file for all torrents we like to share in the same dir. I use this which includes the previous ones on those archbott images from tpowa. Everything after a whitespace will be ignored; so I used it for commenting for the recent entries:
So sha1sum file >> whitelist is all that is needed?
Its not that easy. You need some torrent tools to get the info hash or just upload it at some site like http://torrenteditor.com/ and copy the hash fro mthere (remvoe the white spaces) -- Pierre Schmitz, http://users.archlinux.de/~pierre
On Mon, Aug 3, 2009 at 4:31 PM, Pierre Schmitz<pierre@archlinux.de> wrote:
Am Montag 03 August 2009 23:05:22 schrieb Aaron Griffin:
I get the following with these config options. Similar error when using the full path:
Warning: Can't open accesslist file: whitelist (but will try to create it later, if necessary and possible). PWD: /etc/opentracker
Maybe the opentracker user cannot read those files? How do you start it?
$ ls -l /etc/opentracker/ total 4 -rw-r--r-- 1 root root 3432 2009-08-03 17:07 config -rw-r--r-- 1 opentracker users 0 2009-08-03 17:05 whitelist $ grep su /etc/rc.d/opentracker [ -z "$PID" ] && su opentracker -s /bin/sh -c "/usr/bin/opentracker -f /etc/opentracker/config&"
Then we need a whitelist file for all torrents we like to share in the same dir. I use this which includes the previous ones on those archbott images from tpowa. Everything after a whitespace will be ignored; so I used it for commenting for the recent entries:
So sha1sum file >> whitelist is all that is needed?
Its not that easy. You need some torrent tools to get the info hash or just upload it at some site like http://torrenteditor.com/ and copy the hash fro mthere (remvoe the white spaces)
Sounds good. What tools output this hash info?
Am Montag 03 August 2009 23:36:05 schrieb Aaron Griffin:
$ ls -l /etc/opentracker/ total 4 -rw-r--r-- 1 root root 3432 2009-08-03 17:07 config -rw-r--r-- 1 opentracker users 0 2009-08-03 17:05 whitelist
% ls -l /etc/opentracker/ insgesamt 8 -rw-r--r-- 1 root root 84 18. Jul 01:24 config -rw-r--r-- 1 root root 656 31. Jul 00:03 whitelist
$ grep su /etc/rc.d/opentracker [ -z "$PID" ] && su opentracker -s /bin/sh -c "/usr/bin/opentracker -f /etc/opentracker/config&"
% grep su /etc/rc.d/opentracker [ -z "$PID" ] && su opentracker -s /bin/sh -c "/usr/bin/opentracker -f /etc/opentracker/config&" OK, I don't really know then. Did you compile yourself or use the package from http://users.archlinux.de/~pierre/packages/x86_64/opentracker-20090722-1- x86_64.pkg.tar.gz
Then we need a whitelist file for all torrents we like to share in the same dir. I use this which includes the previous ones on those archbott images from tpowa. Everything after a whitespace will be ignored; so I used it for commenting for the recent entries:
So sha1sum file >> whitelist is all that is needed?
Its not that easy. You need some torrent tools to get the info hash or just upload it at some site like http://torrenteditor.com/ and copy the hash fro mthere (remvoe the white spaces)
Sounds good. What tools output this hash info?
If you have found a super simple command line tool which does just this wihout heavy deps, let me know. In general its just reading the hash in binary format from the torrent file. -- Pierre Schmitz, http://users.archlinux.de/~pierre
On Mon, Aug 3, 2009 at 4:46 PM, Pierre Schmitz<pierre@archlinux.de> wrote:
Am Montag 03 August 2009 23:36:05 schrieb Aaron Griffin:
$ ls -l /etc/opentracker/ total 4 -rw-r--r-- 1 root root 3432 2009-08-03 17:07 config -rw-r--r-- 1 opentracker users 0 2009-08-03 17:05 whitelist
% ls -l /etc/opentracker/ insgesamt 8 -rw-r--r-- 1 root root 84 18. Jul 01:24 config -rw-r--r-- 1 root root 656 31. Jul 00:03 whitelist
$ grep su /etc/rc.d/opentracker [ -z "$PID" ] && su opentracker -s /bin/sh -c "/usr/bin/opentracker -f /etc/opentracker/config&"
% grep su /etc/rc.d/opentracker [ -z "$PID" ] && su opentracker -s /bin/sh -c "/usr/bin/opentracker -f /etc/opentracker/config&"
OK, I don't really know then. Did you compile yourself or use the package from http://users.archlinux.de/~pierre/packages/x86_64/opentracker-20090722-1- x86_64.pkg.tar.gz
I used your PKGBUILD from the AUR
If you have found a super simple command line tool which does just this wihout heavy deps, let me know. In general its just reading the hash in binary format from the torrent file.
I might be able to throw something cheesy together to get the hash, that doesn't understand the torrent format. I'll see later
Am Montag, den 03.08.2009, 16:49 -0500 schrieb Aaron Griffin:
I might be able to throw something cheesy together to get the hash, that doesn't understand the torrent format. I'll see later
The RCs are now moved to http://build.archlinux.org/isos/ along with md5sums and sha1sums. Should we offer this as a public download location for the announcements on bbs and arch-general or wait until you finished the torrent-tracker and then only provide torrent files? I assume we don't will have many downloads, but probably this could burst a transfer limit or similar on sigurd... Gerhard
On Mon, Aug 3, 2009 at 5:11 PM, Gerhard Brauer<gerbra@archlinux.de> wrote:
Am Montag, den 03.08.2009, 16:49 -0500 schrieb Aaron Griffin:
I might be able to throw something cheesy together to get the hash, that doesn't understand the torrent format. I'll see later
The RCs are now moved to http://build.archlinux.org/isos/ along with md5sums and sha1sums. Should we offer this as a public download location for the announcements on bbs and arch-general or wait until you finished the torrent-tracker and then only provide torrent files? I assume we don't will have many downloads, but probably this could burst a transfer limit or similar on sigurd...
OK, generated the torrent files, and dumped the hashes to the opentracker whitelist (upon restart, I got no warnings - that may have been due to an empty file originally). Pierre, I used transmission-cli to do this: for x in *.torrent; do echo -n "$x "; transmissioncli -i $x 2>/dev/null | grep "hash:"; done Is there an easy way to test if the tracker is working?
Am Montag, den 03.08.2009, 17:37 -0500 schrieb Aaron Griffin:
Is there an easy way to test if the tracker is working?
I put one torrent file in transmission. It's not downloading at all. I se all mirrors as webseeds, but it seems that build.archlinux .org is not also a seeder. As a tracker tracker.archlinux.org anwers fine with 200/OK. Gerhard
Am Dienstag 04 August 2009 00:37:51 schrieb Aaron Griffin:
OK, generated the torrent files, and dumped the hashes to the opentracker whitelist (upon restart, I got no warnings - that may have been due to an empty file originally).
Good point. Could you add my list though? So I can redirect my DNS entry later?
Pierre, I used transmission-cli to do this: for x in *.torrent; do echo -n "$x "; transmissioncli -i $x 2>/dev/null | grep "hash:"; done
Is there an easy way to test if the tracker is working?
Well, we could open the torrent file with a client. Is it available for download anywhere? -- Pierre Schmitz, http://users.archlinux.de/~pierre
On Mon, Aug 3, 2009 at 5:52 PM, Pierre Schmitz<pierre@archlinux.de> wrote:
Am Dienstag 04 August 2009 00:37:51 schrieb Aaron Griffin:
OK, generated the torrent files, and dumped the hashes to the opentracker whitelist (upon restart, I got no warnings - that may have been due to an empty file originally).
Good point. Could you add my list though? So I can redirect my DNS entry later?
Done On Mon, Aug 3, 2009 at 5:50 PM, Gerhard Brauer<gerbra@archlinux.de> wrote:
Am Montag, den 03.08.2009, 17:37 -0500 schrieb Aaron Griffin:
Is there an easy way to test if the tracker is working?
I put one torrent file in transmission. It's not downloading at all. I se all mirrors as webseeds, but it seems that build.archlinux .org is not also a seeder. As a tracker tracker.archlinux.org anwers fine with 200/OK.
Added build.archlinux.org as a webseed and regenerated them all. Please download a new .torrent and try again
Am Montag, den 03.08.2009, 18:01 -0500 schrieb Aaron Griffin:
Added build.archlinux.org as a webseed and regenerated them all. Please download a new .torrent and try again
Seems not to work. I have no downloads currently on 2 different clients. I also put my already downloaded finished files in my local download, but i see also not upload activity... All the other webseeds have a tailing slash(/) on their download locations, maybe it's this? Or we have to many other webseeds in with no files to download from.... Gerhard
Am Dienstag 04 August 2009 01:16:57 schrieb Gerhard Brauer:
Seems not to work. I have no downloads currently on 2 different clients. I also put my already downloaded finished files in my local download, but i see also not upload activity...
All the other webseeds have a tailing slash(/) on their download locations, maybe it's this? Or we have to many other webseeds in with no files to download from....
Right the / needs to be there. Opentracker does either not run or is blocked by a firewall. -- Pierre Schmitz, http://users.archlinux.de/~pierre
On Mon, Aug 3, 2009 at 4:44 PM, Pierre Schmitz<pierre@archlinux.de> wrote:
Am Dienstag 04 August 2009 01:16:57 schrieb Gerhard Brauer:
Seems not to work. I have no downloads currently on 2 different clients. I also put my already downloaded finished files in my local download, but i see also not upload activity...
All the other webseeds have a tailing slash(/) on their download locations, maybe it's this? Or we have to many other webseeds in with no files to download from....
Right the / needs to be there. Opentracker does either not run or is blocked by a firewall.
Ok, I can fix this... what do you mean it doesn't run? I thought Gerhard confirmed it was connecting. I opened the port via iptables...
Am Dienstag 04 August 2009 04:28:14 schrieb Aaron Griffin:
On Mon, Aug 3, 2009 at 4:44 PM, Pierre Schmitz<pierre@archlinux.de> wrote:
Am Dienstag 04 August 2009 01:16:57 schrieb Gerhard Brauer:
Seems not to work. I have no downloads currently on 2 different clients. I also put my already downloaded finished files in my local download, but i see also not upload activity...
All the other webseeds have a tailing slash(/) on their download locations, maybe it's this? Or we have to many other webseeds in with no files to download from....
Right the / needs to be there. Opentracker does either not run or is blocked by a firewall.
Ok, I can fix this... what do you mean it doesn't run? I thought Gerhard confirmed it was connecting. I opened the port via iptables...
OK, the tracker is working now. -- Pierre Schmitz, http://users.archlinux.de/~pierre
On Thu, Jul 30, 2009 at 5:22 PM, Pierre Schmitz<pierre@archlinux.de> wrote:
Hi engineers,
thanks to a new version of mktorrent it is now easy to create webseed enabled torrents. I have updated my wrapper script to use this instead of my evil ktorrent hack.
This script can be found at http://git.archlinux.de/archlinux-torrent-utils/plain/mktorrent-archlinux
You will need kmtorrent from http://mktorrent.sourceforge.net/; there is also a PKGBUILD.
I would suggest to install this on sigurd and also the opentracker on it. This way the releng team can create those torrents right on the server and will also be able to add those to the trackers whitelist.
Hmmm note that this is NOT the same mktorrent that is in the AUR :roll:
participants (3)
-
Aaron Griffin
-
Gerhard Brauer
-
Pierre Schmitz