[arch-general] Archweb package update script

21 Jul 2010

      On Wed, Jul 21, 2010 at 5:45 AM, Netanel Shine <netanelshine@gmail.com> wrote:
...
...
...
Hey all,
My name is Netanel Shine, and a few days ago i began the process of
building
an israeli community of archlinux. I used a git clone to set up the main
site. (like some other great projects file that archlinux git have to
offer
like the wiki and the forums)
Everything went smoothly and just works great, except for one problem -
...
package rss feeds.
The site address is: http://archlinux.org.il - under heavy development.
The problem: As you all can see, the left rss box with the latest updates
dont work and the list is empty. - because the system comes with an
integrated "packages" part (you can see it here:
http://archlinux.org.il/packages - not gonna use it), so the rss list
updates from here (i guess)
After discussions with some great people (in the irc) we concluded that
Date: Sun, 18 Jul 2010 21:36:42 -0500
From: Dan McGee <dpmcgee@gmail.com>
Subject: Re: [arch-general] RSS feed - missing feature|problem
On Mon, Jul 12, 2010 at 3:59 AM, Netanel Shine <netanelshine@gmail.com>
wrote:
the
the
...
rss box pull`s the data from the 'models.Package' object - that connected
to
archlinux mainsite DB (Package.objects=PackageManager()), so if i dont
have
any access for the database - thats not gonna work.
All sorts of ideas were came up like bulding a script that will do the
job
or, a an iframe that will copy the box from the mainsite (but the problem
it
will not be in hebrew) and an idea to replace the table that has for
update
in pkg_updates. (with logic that pulls from the feed) - but thats not
easy
because the logic is a wrapped up in django MVC.
So dear friends, i turn to you for help - and I`m sure I would find it.
Check out the reporead command baked into manage.py- you can very
easily populate the package DB yourself.
I have a small script I can send you for this, but I don't have it
handy at the moment. Let me know if that would be useful to you.
-Dan
First of all, I sorry for seeing this message just now.
Thanks Dan, I appreciate your comment, and i would like to have the script
you're talking about , please send him to: netanelshine at gmail dot com ;)
Something like the following:

$ cat projects/archweb/fetch_update_package_db.sh
#!/bin/bash -e

# run at nice 3. it can churn quite a bit of cpu after all.
renice +3 -p $$ > /dev/null

# setup paths
DLURL="ftp://ftp.archlinux.org"
SPATH="/path/to/archweb"
SCRIPT="${SPATH}/scripts/reporead.py"
TMPPATH="/tmp/updaterepos"
ARCHES="i686 x86_64"
REPOS="core extra testing community community-testing"

startdir="$(pwd)"
[ ! -d ${TMPPATH} ] && mkdir ${TMPPATH}

for arch in ${ARCHES}; do
    [ ! -d ${TMPPATH}/${arch} ] && mkdir ${TMPPATH}/${arch}
    for repo in ${REPOS}; do
        #[ ! -d ${TMPPATH}/${repo} ] && mkdir ${TMPPATH}/${repo}
        #cd ${TMPPATH}/${repo}
        echo "Updating ${repo}-${arch}"
        cd $TMPPATH/${arch}
        wget -N ${DLURL}/${repo}/os/${arch}/${repo}.db.tar.gz
        #${SCRIPT} ${arch} ${TMPPATH}/${arch}/${repo}.db.tar.gz
        cd "$startdir"
        ./manage.py reporead ${arch} ${TMPPATH}/${arch}/${repo}.db.tar.gz
        echo ""
    done
done