[arch-dev-public] FTP (re)organization
While discussing with Dale about where to put the new isos, the issue of the ftp organization came up: Right now, we have subdirectories, 0.1, ..., 0.8 and current is a symlink to 0.8. This is bad for several reasons. It suggests that we have a "release", which we don't. And everytime we release a new version, we move 0.X to 0.Y, and re-set the current symlink, that means the mirrors will re-pull all the current packages (which they already have, but download anyway). I suggest doing this for cleanup: - rm current; mv 0.8 current (I know this will cause the same problem as described above, but at least it will be the last time) - rm release - remove all isos from current/iso/* and replace them with the new ones (this means we won't keep the old isos, but do we need them?) - ln -s current/iso iso-download so that people can find the isos more easily on the mirror. Any objections/suggestions?
On 5/14/07, Thomas Bächler <thomas@archlinux.org> wrote:
While discussing with Dale about where to put the new isos, the issue of the ftp organization came up:
Right now, we have subdirectories, 0.1, ..., 0.8 and current is a symlink to 0.8. This is bad for several reasons. It suggests that we have a "release", which we don't. And everytime we release a new version, we move 0.X to 0.Y, and re-set the current symlink, that means the mirrors will re-pull all the current packages (which they already have, but download anyway). I suggest doing this for cleanup:
- rm current; mv 0.8 current (I know this will cause the same problem as described above, but at least it will be the last time) - rm release - remove all isos from current/iso/* and replace them with the new ones (this means we won't keep the old isos, but do we need them?) - ln -s current/iso iso-download so that people can find the isos more easily on the mirror.
Any objections/suggestions?
This will also speed up pacman. The way it works via ftp is as follows: GET foo-1.0-1.pkg.tar.gz # ok now lets get bar PWD 0.8/os/i686 # oh that's not correct CWD .. CWD .. CWD .. CWD current CWD os CWD i686 GET bar-1.0-1.pkg.tar.gz # now baz PWD 0.8/os/current # not correct ...... etc etc etc
2007/5/14, Thomas Bächler <thomas@archlinux.org>:
While discussing with Dale about where to put the new isos, the issue of the ftp organization came up:
Right now, we have subdirectories, 0.1, ..., 0.8 and current is a symlink to 0.8. This is bad for several reasons. It suggests that we have a "release", which we don't. And everytime we release a new version, we move 0.X to 0.Y, and re-set the current symlink, that means the mirrors will re-pull all the current packages (which they already have, but download anyway). I suggest doing this for cleanup:
- rm current; mv 0.8 current (I know this will cause the same problem as described above, but at least it will be the last time) - rm release - remove all isos from current/iso/* and replace them with the new ones (this means we won't keep the old isos, but do we need them?) - ln -s current/iso iso-download so that people can find the isos more easily on the mirror.
Afrer discussing the same issue on ICQ... HUGE YES! additionally: - remove 0.7.2 - remove images (we don't support floppy installation anymore, do we?) - remove other (any sense to have it on mirrors?) with this additional removals we should get: /iso-download /current/os/{i686,x86_64} /current/iso -> /iso-download /extrat/... /unstable/... /testing/... /community/... this will make rsyncing with rsync.archlinux.org::ftp/ to get nice clean tree. -- Roman Kyrylych (Роман Кирилич)
Am Montag, 14. Mai 2007 20:29:59 schrieb Roman Kyrylych:
Afrer discussing the same issue on ICQ... HUGE YES! additionally: - remove 0.7.2 - remove images (we don't support floppy installation anymore, do we?) - remove other (any sense to have it on mirrors?) with this additional removals we should get: /iso-download /current/os/{i686,x86_64} /current/iso -> /iso-download /extrat/... /unstable/... /testing/... /community/... this will make rsyncing with rsync.archlinux.org::ftp/ to get nice clean tree.
I agree. Remove those 0.X folders. Even if all mirrors have to resync again; this would be an one-time-issue. ;-) -- Pierre Schmitz Clemens-August-Straße 76 53115 Bonn Telefon 0228 9716608 Mobil 0160 95269831 Jabber pierre@jabber.archlinux.de WWW http://www.archlinux.de
Am Montag, 14. Mai 2007 20:29:59 schrieb Roman Kyrylych:
Afrer discussing the same issue on ICQ... HUGE YES! additionally: - remove 0.7.2 - remove images (we don't support floppy installation anymore, do we?) - remove other (any sense to have it on mirrors?) with this additional removals we should get: /iso-download /current/os/{i686,x86_64} /current/iso -> /iso-download /extrat/... /unstable/... /testing/... /community/... this will make rsyncing with rsync.archlinux.org::ftp/ to get nice clean tree.
I agree. Remove those 0.X folders. Even if all mirrors have to resync again; this would be an one-time-issue. ;-)
This would help clean up the rsync config a bit too. Less directories to ignore. :)
Am Mon, 14 May 2007 20:43:40 +0200 schrieb Pierre Schmitz <pierre@archlinux.de>:
I agree. Remove those 0.X folders. Even if all mirrors have to resync again; this would be an one-time-issue. ;-)
i guess not. think off the upcoming current/extra -> core merge! AndyRTR
On 5/14/07, Andreas Radke <a.radke@arcor.de> wrote:
Am Mon, 14 May 2007 20:43:40 +0200 schrieb Pierre Schmitz <pierre@archlinux.de>:
I agree. Remove those 0.X folders. Even if all mirrors have to resync again; this would be an one-time-issue. ;-)
i guess not. think off the upcoming current/extra -> core merge!
I think it's better to do that in a slow progression, but yes, if we do make a rename to "core" that will have the same issue. It'll be less packages though.
2007/5/14, Aaron Griffin <aaronmgriffin@gmail.com>:
On 5/14/07, Andreas Radke <a.radke@arcor.de> wrote:
Am Mon, 14 May 2007 20:43:40 +0200 schrieb Pierre Schmitz <pierre@archlinux.de>:
I agree. Remove those 0.X folders. Even if all mirrors have to resync again; this would be an one-time-issue. ;-)
i guess not. think off the upcoming current/extra -> core merge!
I think it's better to do that in a slow progression, but yes, if we do make a rename to "core" that will have the same issue. It'll be less packages though.
Yes, that will be another change, but by that time we should have much lighter current anyway (as a result of repo cleanup). -- Roman Kyrylych (Роман Кирилич)
On 5/14/07, Roman Kyrylych <roman.kyrylych@gmail.com> wrote:
2007/5/14, Thomas Bächler <thomas@archlinux.org>:
While discussing with Dale about where to put the new isos, the issue of the ftp organization came up:
Right now, we have subdirectories, 0.1, ..., 0.8 and current is a symlink to 0.8. This is bad for several reasons. It suggests that we have a "release", which we don't. And everytime we release a new version, we move 0.X to 0.Y, and re-set the current symlink, that means the mirrors will re-pull all the current packages (which they already have, but download anyway). I suggest doing this for cleanup:
- rm current; mv 0.8 current (I know this will cause the same problem as described above, but at least it will be the last time) - rm release - remove all isos from current/iso/* and replace them with the new ones (this means we won't keep the old isos, but do we need them?) - ln -s current/iso iso-download so that people can find the isos more easily on the mirror.
Afrer discussing the same issue on ICQ... HUGE YES! additionally: - remove 0.7.2 - remove images (we don't support floppy installation anymore, do we?) - remove other (any sense to have it on mirrors?) with this additional removals we should get: /iso-download /current/os/{i686,x86_64} /current/iso -> /iso-download /extrat/... /unstable/... /testing/... /community/... this will make rsyncing with rsync.archlinux.org::ftp/ to get nice clean tree.
Remove other from rsync only- not from FTP completely. Otherwise +1 on this idea. -Dan
Everything else seems fine.
- remove all isos from current/iso/* and replace them with the new ones (this means we won't keep the old isos, but do we need them?)
What if a cd doesn't work for some people? Or we were to rush a release? There have been historic instances where people have wanted slightly older isos. It would be a shame to lock out people entirely because they didn't download isos on the right day. I'm not talking about isos that are more than one or two releases old though. Jason
On 5/14/07, Jason Chu <jason@archlinux.org> wrote:
Everything else seems fine.
- remove all isos from current/iso/* and replace them with the new ones (this means we won't keep the old isos, but do we need them?)
What if a cd doesn't work for some people? Or we were to rush a release? There have been historic instances where people have wanted slightly older isos. It would be a shame to lock out people entirely because they didn't download isos on the right day.
On Mon, May 14, 2007 at 04:58:54PM -0500, Aaron Griffin wrote:
On 5/14/07, Jason Chu <jason@archlinux.org> wrote:
Everything else seems fine.
- remove all isos from current/iso/* and replace them with the new ones (this means we won't keep the old isos, but do we need them?)
What if a cd doesn't work for some people? Or we were to rush a release? There have been historic instances where people have wanted slightly older isos. It would be a shame to lock out people entirely because they didn't download isos on the right day.
Hehehe, ok. As long as that's going to be the official arch iso archive. Jason
On 5/14/07, Jason Chu <jason@archlinux.org> wrote:
On Mon, May 14, 2007 at 04:58:54PM -0500, Aaron Griffin wrote:
On 5/14/07, Jason Chu <jason@archlinux.org> wrote:
Everything else seems fine.
- remove all isos from current/iso/* and replace them with the new ones (this means we won't keep the old isos, but do we need them?)
What if a cd doesn't work for some people? Or we were to rush a release? There have been historic instances where people have wanted slightly older isos. It would be a shame to lock out people entirely because they didn't download isos on the right day.
Hehehe, ok. As long as that's going to be the official arch iso archive.
Official or not, stonecrest has replaced all the "how to downgrade" instructions with this. Instead of searching for an old mirror, here's one that never deletes anything, might as well make use of my dreamhost bandwidth and disk space
Jason Chu <jason@archlinux.org> wrote: Everything else seems fine.
- remove all isos from current/iso/* and replace them with the new ones (this means we won't keep the old isos, but do we need them?)
What if a cd doesn't work for some people? Or we were to rush a release? There have been historic instances where people have wanted slightly older isos. It would be a shame to lock out people entirely because they didn't download isos on the right day.
I'm not talking about isos that are more than one or two releases old though.
If the directories are restructured a bit, I don't see why there couldn't be old isos too...(maybe a rev or two back). /isos/200705/i686/200705-base.iso /isos/200705/i686/200705-ftp.iso /isos/200705/i686/200705-full.iso /isos/200705/x86_64/200705-base.iso /isos/200705/x86_64/200705-ftp.iso /isos/200705/x86_64/200705-full.iso /isos/200701/i686/200701-base.iso /isos/200701/i686/200701-ftp.iso /isos/200701/i686/200701-full.iso /isos/200701/x86_64/200701-base.iso /isos/200701/x86_64/200701-ftp.iso /isos/200701/x86_64/200701-full.iso /core/ /extra/ /blah/
eliott schrieb:
If the directories are restructured a bit, I don't see why there couldn't be old isos too...(maybe a rev or two back).
/isos/200705/i686/200705-base.iso /isos/200705/i686/200705-ftp.iso /isos/200705/i686/200705-full.iso /isos/200705/x86_64/200705-base.iso /isos/200705/x86_64/200705-ftp.iso /isos/200705/x86_64/200705-full.iso /isos/200701/i686/200701-base.iso /isos/200701/i686/200701-ftp.iso /isos/200701/i686/200701-full.iso /isos/200701/x86_64/200701-base.iso /isos/200701/x86_64/200701-ftp.iso /isos/200701/x86_64/200701-full.iso /core/ /extra/ /blah/
Right now, any ftp restructuring mustn't require the mirrors to reconfigure. Many mirrors don't sync the full ftp, but current/ extra/ and community/ only (they are a minority, but some of them do).
eliott schrieb: Right now, any ftp restructuring mustn't require the mirrors to reconfigure. Many mirrors don't sync the full ftp, but current/ extra/ and community/ only (they are a minority, but some of them do).
I was under the impression that most sites mirrored "ftp" rsync target. Pretty sure the "community" target was only recently added... In the event that I am correct, a top level iso directory would still be mirrrored just fine. Splitting them out would be a great benefit to sites that *do* only want to mirror current, extra, and community. As it stands right now, they get the isos too with the current target..so they have to exclude that dir manually in their rsync pull (which isn't really a big deal). It just seems to me that it would be cleaner to have iso's living in their own place.
Me again. No matter what we decide, I'd like the isos to be on the ftp today, so that the mirrors can start syncing.
On 5/15/07, Thomas Bächler <thomas@archlinux.org> wrote:
Me again. No matter what we decide, I'd like the isos to be on the ftp today, so that the mirrors can start syncing.
You should have write access - I set it up with ftp-arch permissions
Aaron Griffin schrieb:
On 5/15/07, Thomas Bächler <thomas@archlinux.org> wrote:
Me again. No matter what we decide, I'd like the isos to be on the ftp today, so that the mirrors can start syncing.
You should have write access - I set it up with ftp-arch permissions
I know, but I didn't proceed due to the objections made by Dale (I suggested simply putting them in current/os/* and deleting the old ones), thus this thread. But instead of getting a consensus on a short-term solution, there is only pointless discussion :) I hoped you enablers would say something definite. Short-term, I am in favor of deleting the old isos, putting the new ones instead of them, long-term make current a directory, instead of a symlink to 0.8.
On Tue, 2007-05-15 at 17:58 +0200, Thomas Bächler wrote:
Aaron Griffin schrieb:
On 5/15/07, Thomas Bächler <thomas@archlinux.org> wrote:
Me again. No matter what we decide, I'd like the isos to be on the ftp today, so that the mirrors can start syncing.
You should have write access - I set it up with ftp-arch permissions
I know, but I didn't proceed due to the objections made by Dale (I suggested simply putting them in current/os/* and deleting the old ones), thus this thread.
But instead of getting a consensus on a short-term solution, there is only pointless discussion :)
I hoped you enablers would say something definite.
Short-term, I am in favor of deleting the old isos, putting the new ones instead of them, long-term make current a directory, instead of a symlink to 0.8.
Why not do dated subfolders/isos like Eliott suggested? We can then keep the old isos too, just not in the same place they're kept now. Dale
On 5/15/07, Thomas Bächler <thomas@archlinux.org> wrote:
Aaron Griffin schrieb:
On 5/15/07, Thomas Bächler <thomas@archlinux.org> wrote:
Me again. No matter what we decide, I'd like the isos to be on the ftp today, so that the mirrors can start syncing.
You should have write access - I set it up with ftp-arch permissions
I know, but I didn't proceed due to the objections made by Dale (I suggested simply putting them in current/os/* and deleting the old ones), thus this thread.
But instead of getting a consensus on a short-term solution, there is only pointless discussion :)
I hoped you enablers would say something definite.
Short-term, I am in favor of deleting the old isos, putting the new ones instead of them, long-term make current a directory, instead of a symlink to 0.8.
Maybe I didn't pay enough attention to this thread, but I didn't know you wanted to delete the old isos. Interim solution: just put the new isos out there. If people have problems with removing the ISOs, then we can worry about that later - the important thing, at this moment in time, is putting the new isos out there. So here's the definite: don't delete things just yet, push the new ISOs out so we can get this release moving, and we can worry about deletion / moving dirs after the release.
Aaron Griffin schrieb:
Interim solution: just put the new isos out there. If people have problems with removing the ISOs, then we can worry about that later - the important thing, at this moment in time, is putting the new isos out there.
So here's the definite: don't delete things just yet, push the new ISOs out so we can get this release moving, and we can worry about deletion / moving dirs after the release.
Put them where? In the directory where the ISOs are right now, there is an "addons" subdir. The new addons will be incompatible with the old isos. Also I don't want people to be confused as to which ISOs to use, therefore, two sets of ISOs in the same subdirectory are bad. Should we create /installer/2007.05 or /iso/2007.05 and mention it in the announcement?
On 5/15/07, Thomas Bächler <thomas@archlinux.org> wrote:
Aaron Griffin schrieb:
Interim solution: just put the new isos out there. If people have problems with removing the ISOs, then we can worry about that later - the important thing, at this moment in time, is putting the new isos out there.
So here's the definite: don't delete things just yet, push the new ISOs out so we can get this release moving, and we can worry about deletion / moving dirs after the release.
Put them where? In the directory where the ISOs are right now, there is an "addons" subdir. The new addons will be incompatible with the old isos.
Also I don't want people to be confused as to which ISOs to use, therefore, two sets of ISOs in the same subdirectory are bad.
Ah, I see the issue now.
Should we create /installer/2007.05 or /iso/2007.05 and mention it in the announcement?
I'd say just make a new iso/2007.05/ dir and do everything in there.
Aaron Griffin schrieb:
Ah, I see the issue now.
Should we create /installer/2007.05 or /iso/2007.05 and mention it in the announcement?
I'd say just make a new iso/2007.05/ dir and do everything in there.
Isos are in /iso/2007.05/{i686,x86_64}. As soon as ftp.archlinux.org has synced, could one of you (with proper permissions): - announce it (changelog is in the iso dir, please don't forget to mention that newest isos are in /iso/2007.05 on the ftp) - change the torrent download links on the download page (we can link them to ftp.archlinux.org) - mention on the download page that isos are in /iso on the ftp mirror - maybe add an /iso rsyncd target (most mirrors sync /ftp, but not all of them) All seeders, please change your torrents to the new ones.
participants (10)
-
Aaron Griffin
-
Andreas Radke
-
Dale Blount
-
Dan McGee
-
eliott
-
eliott＠cactuswax.net
-
Jason Chu
-
Pierre Schmitz
-
Roman Kyrylych
-
Thomas Bächler