[pacman-dev] rankmirrors script
Here is a alternative to the current sortmirrrors script. It was designed to address this bug: http://bugs.archlinux.org/task/2952. It does not touch your current mirror lists, it simply reads them and outputs timing information for each, and outputs a Server format that can be used in the mirrors list. I've never done any python before, and this is heavily based off the script found in the forums but slightly refined to do different and slightly more intuitive output. (http://bbs.archlinux.org/viewtopic.php?t=19278&sid=475ab18a5d9b31952459fd9912d20d19) Dan __________________________________ #! /usr/bin/python # # rankmirrors : read a list of mirrors from a file and rank them by speed # # Original Idea copyright (c) 2006 R.G. <chesercat> # Modified 2006 by Dan McGee <dpmcgee@gmail.com> # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, # USA. # import sys, time, socket, urllib2 from optparse import OptionParser socket.setdefaulttimeout(10) def createOptParser(): parser = OptionParser() parser.add_option("-n", dest="num", default=0, help="number of servers to print, 0 for all") parser.add_option("-v", "--verbose", action="store_true", dest="verbose", default=False, help="be verbose in output") return parser def timeCmd(cmd): before = time.time(); try: cmd(); except KeyboardInterrupt, ki: raise ki except socket.timeout, ioe: return 'timeout' except Exception, e: return 'unreachable' return time.time() - before; def talkToServer(serverUrl): opener = urllib2.build_opener() tmp = opener.open(serverUrl).read() def getFuncToTime(serverUrl): return lambda : talkToServer(serverUrl) def cmpPairBySecond(p1, p2): if p1[1] == p2[1]: return 0 if p1[1] < p2[1]: return -1 return 1 if __name__ == "__main__": parser = createOptParser() (options, args) = parser.parse_args() if len(args) != 1: parser.print_help() sys.exit(0) fl = open(args[0], 'r') serverToTime = {} print 'Querying servers, this may take some time...' for ln in fl.readlines(): splitted = ln.split('=') if splitted[0].strip() != 'Server': continue serverUrl = splitted[1].strip() if serverUrl[-1] == '\n': serverUrl = serverUrl[0:-1] if options.verbose: print serverUrl, '...', else: print ' * ', sys.stdout.flush() serverToTime[serverUrl] = timeCmd(getFuncToTime(serverUrl)) if options.verbose: print serverToTime[serverUrl] items = serverToTime.items() items.sort(cmpPairBySecond) numToShow = int(options.num) if numToShow == 0: numToShow = len(items) if len(items) > 0: print print ' Servers sorted by time:' print for i in items[0:numToShow]: print i[1], ':', i[0] if len(items) > 0: print print ' Server list format:' print for i in items[0:numToShow]: print 'Server =', i[0]
On 12/8/06, Dan McGee <dpmcgee@gmail.com> wrote:
Here is a alternative to the current sortmirrrors script. It was designed to address this bug: http://bugs.archlinux.org/task/2952. It does not touch your current mirror lists, it simply reads them and outputs timing information for each, and outputs a Server format that can be used in the mirrors list.
Great - I'm testing it out right now, and will most likely include it in pacman cvs. One note however: the usage doesn't mention the need for a pacman.d file. I'll let you fix that one however you like 8) Thanks!
On 12/8/06, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
On 12/8/06, Dan McGee <dpmcgee@gmail.com> wrote:
Here is a alternative to the current sortmirrrors script. It was designed to address this bug: http://bugs.archlinux.org/task/2952. It does not touch your current mirror lists, it simply reads them and outputs timing information for each, and outputs a Server format that can be used in the mirrors list.
Great - I'm testing it out right now, and will most likely include it in pacman cvs. One note however: the usage doesn't mention the need for a pacman.d file. I'll let you fix that one however you like 8)
Thanks!
With my limited knowledge of python (about 2 hours now), I'll see what I can do. Let me know if you have any more suggestions for additions. By the way, I also asked the original for permission to include it and it is no problem, I put the copyright line as he requested at the top.
On 12/8/06, Dan McGee <dpmcgee@gmail.com> wrote:
With my limited knowledge of python (about 2 hours now), I'll see what I can do. Let me know if you have any more suggestions for additions.
I just mean something like this: $ diff -u rankmirrors rankmirrors.2 --- rankmirrors 2006-12-08 13:56:54.000000000 -0600 +++ rankmirrors.2 2006-12-08 14:09:19.000000000 -0600 @@ -26,7 +26,7 @@ socket.setdefaulttimeout(10) def createOptParser(): - parser = OptionParser() + parser = OptionParser(usage="usage: %prog [options] <server list file>") parser.add_option("-n", dest="num", default=0, help="number of servers to print, 0 for all") parser.add_option("-v", "--verbose", action="store_true", dest="verbose", As for other suggestions, the only thing I can think of is an option to output in "file format" - that is, everything prefixed with '#' except the server lines, so one could just > it to a file.
Version 2 adds the file format option, as well as adds some description and usage information. I also fixed all the indenting to make it more consistent and added a vim modeline at the end. Let me know what you think. _______________ #! /usr/bin/python # # rankmirrors : read a list of mirrors from a file and rank them by speed # # Original Idea copyright (c) 2006 R.G. <chesercat> # Modified 2006 by Dan McGee <dpmcgee@gmail.com> # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, # USA. # import sys, datetime, time, socket, urllib2 from optparse import OptionParser def createOptParser(): usage = "usage: %prog [options] MIRRORFILE" description = "Ranks pacman mirrors by their connection and opening speed. Pacman mirror files are located in /etc/pacman.d/." parser = OptionParser(usage=usage,description=description) parser.add_option("-f", "--formatted", action="store_true", dest = "formatted", default=False, help="output in mirror file format") parser.add_option("-n", dest="num", default=0, help="number of servers to output, 0 for all") parser.add_option("-v", "--verbose", action="store_true", dest="verbose", default=False, help="be verbose in output") return parser def timeCmd(cmd): before = time.time(); try: cmd(); except KeyboardInterrupt, ki: raise ki except socket.timeout, ioe: return 'timeout' except Exception, e: return 'unreachable' return time.time() - before; def talkToServer(serverUrl): opener = urllib2.build_opener() tmp = opener.open(serverUrl).read() def getFuncToTime(serverUrl): return lambda : talkToServer(serverUrl) def cmpPairBySecond(p1, p2): if p1[1] == p2[1]: return 0 if p1[1] < p2[1]: return -1 return 1 if __name__ == "__main__": parser = createOptParser() (options, args) = parser.parse_args() if len(args) != 1: parser.print_help(sys.stderr) sys.exit(0) # allows connections to time out if they take too long socket.setdefaulttimeout(10) fl = open(args[0], 'r') serverToTime = {} if options.formatted: print "# Server list generated by rankmirrors on", datetime.date.today() else: print 'Querying servers, this may take some time...' for ln in fl.readlines(): splitted = ln.split('=') if splitted[0].strip() != 'Server': if options.formatted: print ln, continue serverUrl = splitted[1].strip() if serverUrl[-1] == '\n': serverUrl = serverUrl[0:-1] if options.verbose and options.formatted: print '#',serverUrl,'...', elif options.verbose: print serverUrl,'...', elif not options.formatted: print ' * ', sys.stdout.flush() serverToTime[serverUrl] = timeCmd(getFuncToTime(serverUrl)) if options.verbose: print serverToTime[serverUrl] items = serverToTime.items() items.sort(cmpPairBySecond) numToShow = int(options.num) if numToShow == 0: numToShow = len(items) if len(items) > 0: if options.formatted: for i in items[0:numToShow]: print 'Server =', i[0] else: print print ' Servers sorted by time:' for i in items[0:numToShow]: print i[1], ':', i[0] # vim: set ts=4 sw=4 sta et sts ai:
On 12/8/06, Dan McGee <dpmcgee@gmail.com> wrote:
Version 2 adds the file format option, as well as adds some description and usage information. I also fixed all the indenting to make it more consistent and added a vim modeline at the end. Let me know what you think.
Added to cvs. Thanks alot.
kris|~$ pacrankmirrors File "/home/kris/bin/pacrankmirrors", line 28 description = "Ranks pacman mirrors by their connection and ^ SyntaxError: EOL while scanning single-quoted string i fixed it though by making the first 28 -> lines into one-lines but maybe someone else experiences this. -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
That line with the description has to be on one line. This patch should fix that, a problem in a parser.add_option, and the Gmail inserted "Show quoted text". Dan --- pacman-lib.orig/scripts/rankmirrors 2006-12-08 17:07:34.000000000 -0500 +++ pacman-lib/scripts/rankmirrors 2006-12-08 17:35:09.000000000 -0500 @@ -25,16 +25,14 @@ def createOptParser(): usage = "usage: %prog [options] MIRRORFILE" - description = "Ranks pacman mirrors by their connection and -opening speed. Pacman mirror files are located in /etc/pacman.d/." + description = "Ranks pacman mirrors by their connection and opening speed. Pacman mirror files are located in /etc/pacman.d/." parser = OptionParser(usage=usage,description=description) parser.add_option("-f", "--formatted", action="store_true", - dest = "formatted", default=False, help="output in mirror -file format") + dest = "formatted", default=False, + help="output in mirror file format") parser.add_option("-n", dest="num", default=0, help="number of servers to output, 0 for all") parser.add_option("-v", "--verbose", action="store_true", dest="verbose", -- Show quoted text - default=False, help="be verbose in output") return parser
On 12/8/06, Dan McGee <dpmcgee@gmail.com> wrote:
That line with the description has to be on one line. This patch should fix that, a problem in a parser.add_option, and the Gmail inserted "Show quoted text".
Yeah, I apologize for that - I'm at work, so including this script was kinda done in the background 8) All good now.
participants (3)
-
Aaron Griffin
-
Dan McGee
-
kfs1＠online.no