[aur-dev] [PATCH] Add newly submitted packages functionality with json interface

older
[aur-dev] Remove LocationID and...

Laszlo Papp

5 Oct 2009 5 Oct '09

10:02 p.m.

From: Laszlo Papp Add a json interface for the newly submitted packages output, after a query operation, and it will be easier to handle it from a frontend for example. Signed-off-by: Laszlo Papp --- web/html/rpc.php | 1 + web/lib/aurjson.class.php | 32 +++++++++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 1 deletions(-) diff --git a/web/html/rpc.php b/web/html/rpc.php index 240cad1..52e5603 100644 --- a/web/html/rpc.php +++ b/web/html/rpc.php @@ -19,6 +19,7 @@ if ( $_SERVER['REQUEST_METHOD'] == 'GET' ) { echo '<li>search</li>'; echo '<li>info</li>'; echo '<li>msearch</li>'; + echo '<li>newpackages</li>'; echo '</ul> '; echo 'Each method requires the following HTTP GET syntax: '; echo ' type=methodname&arg=data '; diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index 251203a..884e327 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -18,7 +18,7 @@ include_once("aur.inc"); **/ class AurJSON { private $dbh = false; - private $exposed_methods = array('search','info','msearch'); + private $exposed_methods = array('search','info','msearch','newpackages'); private $fields = array('Packages.ID','Name','Version','CategoryID', 'Description', 'LocationID', 'URL','URLPath','License','NumVotes', 'OutOfDate'); @@ -116,6 +116,36 @@ class AurJSON { } /** + * Returns the latest packages from the database. + * @param $quantity_integer how much packages should return. + * @return mixed Returns an array with the latests quantity_integer packages. + **/ + private function newpackages($quantity_integer) { + echo $keywrd_string; + if ($quantity_integer <= 0) { + return $this->json_error('Quantity should be positive'); + } + + $query = "SELECT " . implode(',', $this->fields) . + " FROM Packages ORDER BY SubmittedTS DESC LIMIT 0," .$quantity_integer; + + $result = db_query($query, $this->dbh); + + if ( $result && (mysql_num_rows($result) > 0) ) { + $search_data = array(); + while ( $row = mysql_fetch_assoc($result) ) { + array_push($search_data, $row); + } + + mysql_free_result($result); + return $this->json_results('newpackages', $search_data); + } + else { + return $this->json_error('No results found'); + } + } + + /** * Returns the info on a specific package. * @param $pqdata The ID or name of the package. Package Query Data. * @return mixed Returns an array of value data containing the package data -- 1.6.4.4

Show replies by date

elij

5 Oct 5 Oct

11:40 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

I guess I don't see the need for this. If you want to see 'new packages', just use the rss feed. Dumping this in the rcp api seems... wrong to me. On Mon, Oct 5, 2009 at 3:02 PM, Laszlo Papp wrote:

...

From: Laszlo Papp

Add a json interface for the newly submitted packages output, after a query operation, and it will be easier to handle it from a frontend for example.

Signed-off-by: Laszlo Papp --- web/html/rpc.php | 1 + web/lib/aurjson.class.php | 32 +++++++++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 1 deletions(-)

diff --git a/web/html/rpc.php b/web/html/rpc.php index 240cad1..52e5603 100644 --- a/web/html/rpc.php +++ b/web/html/rpc.php @@ -19,6 +19,7 @@ if ( $_SERVER['REQUEST_METHOD'] == 'GET' ) { echo '<li>search</li>'; echo '<li>info</li>'; echo '<li>msearch</li>'; + echo '<li>newpackages</li>'; echo '</ul> '; echo 'Each method requires the following HTTP GET syntax: '; echo ' type=methodname&arg=data '; diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index 251203a..884e327 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -18,7 +18,7 @@ include_once("aur.inc"); **/ class AurJSON { private $dbh = false; - private $exposed_methods = array('search','info','msearch'); + private $exposed_methods = array('search','info','msearch','newpackages'); private $fields = array('Packages.ID','Name','Version','CategoryID', 'Description', 'LocationID', 'URL','URLPath','License','NumVotes', 'OutOfDate'); @@ -116,6 +116,36 @@ class AurJSON { }

/** + * Returns the latest packages from the database. + * @param $quantity_integer how much packages should return. + * @return mixed Returns an array with the latests quantity_integer packages. + **/ + private function newpackages($quantity_integer) { + echo $keywrd_string; + if ($quantity_integer <= 0) { + return $this->json_error('Quantity should be positive'); + } + + $query = "SELECT " . implode(',', $this->fields) . + " FROM Packages ORDER BY SubmittedTS DESC LIMIT 0," .$quantity_integer; + + $result = db_query($query, $this->dbh); + + if ( $result && (mysql_num_rows($result) > 0) ) { + $search_data = array(); + while ( $row = mysql_fetch_assoc($result) ) { + array_push($search_data, $row); + } + + mysql_free_result($result); + return $this->json_results('newpackages', $search_data); + } + else { + return $this->json_error('No results found'); + } + } + + /** * Returns the info on a specific package. * @param $pqdata The ID or name of the package. Package Query Data. * @return mixed Returns an array of value data containing the package data -- 1.6.4.4

Laszlo Papp

6 Oct 6 Oct

6:52 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Tue, Oct 6, 2009 at 1:40 AM, elij wrote:

...

I guess I don't see the need for this. If you want to see 'new packages', just use the rss feed. Dumping this in the rcp api seems... wrong to me.

Hello Elliot! Thank you the feedback! My opinion in this matter is that if I'd like to create a frontend program for AUR, especially console based e.g., or to create another API/backend for AUR, then the json interface/output would be more portable than parsing html/xml pages to get an option for a command line frontend to get the newly submitted/updated packages. Rss feed and this option are different purposes in fact. With this option from command line you could get anytime the newly updated/submitted packages, but with rss you see them continously. The first facility is really console based, but the second is webpage based, I think it's different or maybe I'm wrong. Summary, to do as many things in json interface as we can, it's a good idea in my opinion now, mainly for porting and easier parsing purposes. Best Regards, Laszlo Papp

Aaron Griffin

7:03 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Tue, Oct 6, 2009 at 1:52 PM, Laszlo Papp wrote:

...

On Tue, Oct 6, 2009 at 1:40 AM, elij wrote:

...
I guess I don't see the need for this. If you want to see 'new packages', just use the rss feed. Dumping this in the rcp api seems... wrong to me.

Hello Elliot!

Thank you the feedback! My opinion in this matter is that if I'd like to create a frontend program for AUR, especially console based e.g., or to create another API/backend for AUR, then the json interface/output would be more portable than parsing html/xml pages to get an option for a command line frontend to get the newly submitted/updated packages.

Rss feed and this option are different purposes in fact. With this option from command line you could get anytime the newly updated/submitted packages, but with rss you see them continously. The first facility is really console based, but the second is webpage based, I think it's different or maybe I'm wrong.

You could do the exact same thing with an RSS feed... I don't understand how this data being in RSS makes it so that you cannot fetch the results whenever you want. RSS isn't made of magic. $ cat aurpkgs.py #!/usr/bin/env python import feedparser pkgs=feedparser.parse("http://aur.archlinux.org/rss2.php") print pkgs.feed.title for i in pkgs.entries: print i.title $ python aurpkgs.py AUR Newest Packages munipack rawtran fitspng sloppy lib32-jack-audio-connection-kit lib32-libsndfile lib32-flac perl-proc-pid-file libcss-svn libwapcaplet-svn vpipe aircrack-ng-cuda perl-compress-zlib generatorrunner haskell-vty-ui lua-md5 haskell-bindings-libcddb geda-gaf haskell-sample-frame gdatacopier-svn

Laszlo Papp

7:12 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Tue, Oct 6, 2009 at 9:03 PM, Aaron Griffin wrote:

...

...
On Tue, Oct 6, 2009 at 1:40 AM, elij wrote:

...
I guess I don't see the need for this. If you want to see 'new packages', just use the rss feed. Dumping this in the rcp api seems... wrong to me.

Hello Elliot!

Thank you the feedback! My opinion in this matter is that if I'd like to create a frontend program for AUR, especially console based e.g., or to create another API/backend for AUR, then the json interface/output would be more portable than parsing html/xml pages to get an option for a command line frontend to get the newly submitted/updated packages.

Rss feed and this option are different purposes in fact. With this option from command line you could get anytime the newly updated/submitted packages, but with rss you see them continously. The first facility is really console based, but the second is webpage based, I

On Tue, Oct 6, 2009 at 1:52 PM, Laszlo Papp wrote: think

...
it's different or maybe I'm wrong.

You could do the exact same thing with an RSS feed... I don't understand how this data being in RSS makes it so that you cannot fetch the results whenever you want. RSS isn't made of magic.

$ cat aurpkgs.py #!/usr/bin/env python

import feedparser

pkgs=feedparser.parse("http://aur.archlinux.org/rss2.php")

print pkgs.feed.title

for i in pkgs.entries: print i.title

$ python aurpkgs.py AUR Newest Packages munipack rawtran fitspng sloppy lib32-jack-audio-connection-kit lib32-libsndfile lib32-flac perl-proc-pid-file libcss-svn libwapcaplet-svn vpipe aircrack-ng-cuda perl-compress-zlib generatorrunner haskell-vty-ui lua-md5 haskell-bindings-libcddb geda-gaf haskell-sample-frame gdatacopier-svn

It seems good in fact, I don't know yet it can be implemented in C, I will look after it later, thanks. It can't be solved with rss if a new API/AUR backend will be born. In that case a pure C mysql api handling, or json interface would be better choice in my opinion. I mean for a new backend, API without depending on any webpage application. Best Regards, Laszlo Papp

Loui Chang

8 Oct 8 Oct

10:54 a.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Tue 06 Oct 2009 14:03 -0500, Aaron Griffin wrote:

...

On Tue, Oct 6, 2009 at 1:52 PM, Laszlo Papp wrote:

...
On Tue, Oct 6, 2009 at 1:40 AM, elij wrote:

...
I guess I don't see the need for this. If you want to see 'new packages', just use the rss feed. Dumping this in the rcp api seems... wrong to me.

Thank you the feedback! My opinion in this matter is that if I'd like to create a frontend program for AUR, especially console based e.g., or to create another API/backend for AUR, then the json interface/output would be more portable than parsing html/xml pages to get an option for a command line frontend to get the newly submitted/updated packages.

Rss feed and this option are different purposes in fact. With this option from command line you could get anytime the newly updated/submitted packages, but with rss you see them continously. The first facility is really console based, but the second is webpage based, I think it's different or maybe I'm wrong.

You could do the exact same thing with an RSS feed... I don't understand how this data being in RSS makes it so that you cannot fetch the results whenever you want. RSS isn't made of magic.

I wasn't sure if this was a good idea, but then I wondered why we're fragmenting the data into different interfaces (RSS, JSON, web) rather than unifying everything under one interface. So after my initial apprehension this enhancement makes sense to me, but I'd like to see it do caching like the RSS does.

elij

10:15 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu, Oct 8, 2009 at 3:54 AM, Loui Chang wrote:

...

On Tue 06 Oct 2009 14:03 -0500, Aaron Griffin wrote:

...
On Tue, Oct 6, 2009 at 1:52 PM, Laszlo Papp wrote:

...
On Tue, Oct 6, 2009 at 1:40 AM, elij wrote:

...
I guess I don't see the need for this. If you want to see 'new packages', just use the rss feed. Dumping this in the rcp api seems... wrong to me.

Thank you the feedback! My opinion in this matter is that if I'd like to create a frontend program for AUR, especially console based e.g., or to create another API/backend for AUR, then the json interface/output would be more portable than parsing html/xml pages to get an option for a command line frontend to get the newly submitted/updated packages.

Rss feed and this option are different purposes in fact. With this option from command line you could get anytime the newly updated/submitted packages, but with rss you see them continously. The first facility is really console based, but the second is webpage based, I think it's different or maybe I'm wrong.

You could do the exact same thing with an RSS feed... I don't understand how this data being in RSS makes it so that you cannot fetch the results whenever you want. RSS isn't made of magic.

I wasn't sure if this was a good idea, but then I wondered why we're fragmenting the data into different interfaces (RSS, JSON, web) rather than unifying everything under one interface.

So after my initial apprehension this enhancement makes sense to me, but I'd like to see it do caching like the RSS does.

If you are bound and determined to do it, then memcache would be sufficient for caching it (so it can kind of cache like the RSS does). Not sure if memcached is running on the aur server yet, but I am sure someone could slap it on there without difficulty if it isn't.

elij

10:16 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu, Oct 8, 2009 at 3:15 PM, elij wrote:

...

On Thu, Oct 8, 2009 at 3:54 AM, Loui Chang wrote:

...
On Tue 06 Oct 2009 14:03 -0500, Aaron Griffin wrote:

...
On Tue, Oct 6, 2009 at 1:52 PM, Laszlo Papp wrote:

...
On Tue, Oct 6, 2009 at 1:40 AM, elij wrote:

...
I guess I don't see the need for this. If you want to see 'new packages', just use the rss feed. Dumping this in the rcp api seems... wrong to me.

Thank you the feedback! My opinion in this matter is that if I'd like to create a frontend program for AUR, especially console based e.g., or to create another API/backend for AUR, then the json interface/output would be more portable than parsing html/xml pages to get an option for a command line frontend to get the newly submitted/updated packages.

Rss feed and this option are different purposes in fact. With this option from command line you could get anytime the newly updated/submitted packages, but with rss you see them continously. The first facility is really console based, but the second is webpage based, I think it's different or maybe I'm wrong.

You could do the exact same thing with an RSS feed... I don't understand how this data being in RSS makes it so that you cannot fetch the results whenever you want. RSS isn't made of magic.

I wasn't sure if this was a good idea, but then I wondered why we're fragmenting the data into different interfaces (RSS, JSON, web) rather than unifying everything under one interface.

So after my initial apprehension this enhancement makes sense to me, but I'd like to see it do caching like the RSS does.

If you are bound and determined to do it, then memcache would be sufficient for caching it (so it can kind of cache like the RSS does). Not sure if memcached is running on the aur server yet, but I am sure someone could slap it on there without difficulty if it isn't.

fyi. I still think it is a bad idea. Just trying to point out where the duct tape is laying. :P

Randy Morris

10:30 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu, Oct 08, 2009 at 03:16:26PM -0700, elij wrote:

...

On Thu, Oct 8, 2009 at 3:15 PM, elij wrote:

...
On Thu, Oct 8, 2009 at 3:54 AM, Loui Chang wrote:

...
On Tue 06 Oct 2009 14:03 -0500, Aaron Griffin wrote:

...
On Tue, Oct 6, 2009 at 1:52 PM, Laszlo Papp wrote:

...
On Tue, Oct 6, 2009 at 1:40 AM, elij wrote:

...
I guess I don't see the need for this. If you want to see 'new packages', just use the rss feed. Dumping this in the rcp api seems... wrong to me.

Thank you the feedback! My opinion in this matter is that if I'd like to create a frontend program for AUR, especially console based e.g., or to create another API/backend for AUR, then the json interface/output would be more portable than parsing html/xml pages to get an option for a command line frontend to get the newly submitted/updated packages.

Rss feed and this option are different purposes in fact. With this option from command line you could get anytime the newly updated/submitted packages, but with rss you see them continously. The first facility is really console based, but the second is webpage based, I think it's different or maybe I'm wrong.

You could do the exact same thing with an RSS feed... I don't understand how this data being in RSS makes it so that you cannot fetch the results whenever you want. RSS isn't made of magic.

I wasn't sure if this was a good idea, but then I wondered why we're fragmenting the data into different interfaces (RSS, JSON, web) rather than unifying everything under one interface.

So after my initial apprehension this enhancement makes sense to me, but I'd like to see it do caching like the RSS does.

If you are bound and determined to do it, then memcache would be sufficient for caching it (so it can kind of cache like the RSS does). Not sure if memcached is running on the aur server yet, but I am sure someone could slap it on there without difficulty if it isn't.

fyi. I still think it is a bad idea. Just trying to point out where the duct tape is laying. :P

FWIW, I agree with cactus here. Moving the recent updates off of RSS would make the behavior of the AUR different from the main Arch site in this regard. The RPC interface just doesn't seem to be the right place for this.

Aaron Griffin

10:36 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu, Oct 8, 2009 at 5:30 PM, Randy Morris wrote:

...

On Thu, Oct 08, 2009 at 03:16:26PM -0700, elij wrote:

...
On Thu, Oct 8, 2009 at 3:15 PM, elij wrote:

...
On Thu, Oct 8, 2009 at 3:54 AM, Loui Chang wrote:

...
On Tue 06 Oct 2009 14:03 -0500, Aaron Griffin wrote:

...
On Tue, Oct 6, 2009 at 1:52 PM, Laszlo Papp wrote:

...
On Tue, Oct 6, 2009 at 1:40 AM, elij wrote:

> I guess I don't see the need for this. > If you want to see 'new packages', just use the rss feed. > Dumping this in the rcp api seems... wrong to me.

Thank you the feedback! My opinion in this matter is that if I'd like to create a frontend program for AUR, especially console based e.g., or to create another API/backend for AUR, then the json interface/output would be more portable than parsing html/xml pages to get an option for a command line frontend to get the newly submitted/updated packages.

Rss feed and this option are different purposes in fact. With this option from command line you could get anytime the newly updated/submitted packages, but with rss you see them continously. The first facility is really console based, but the second is webpage based, I think it's different or maybe I'm wrong.

You could do the exact same thing with an RSS feed... I don't understand how this data being in RSS makes it so that you cannot fetch the results whenever you want. RSS isn't made of magic.

I wasn't sure if this was a good idea, but then I wondered why we're fragmenting the data into different interfaces (RSS, JSON, web) rather than unifying everything under one interface.

So after my initial apprehension this enhancement makes sense to me, but I'd like to see it do caching like the RSS does.

If you are bound and determined to do it, then memcache would be sufficient for caching it (so it can kind of cache like the RSS does). Not sure if memcached is running on the aur server yet, but I am sure someone could slap it on there without difficulty if it isn't.

fyi. I still think it is a bad idea. Just trying to point out where the duct tape is laying. :P

FWIW, I agree with cactus here. Moving the recent updates off of RSS would make the behavior of the AUR different from the main Arch site in this regard. The RPC interface just doesn't seem to be the right place for this.

Flickr actually has two APIs - a feed based one and a REST based "ajax" API. Both accept a format=foo parameter and json is allowed for both sets. * Is the AUR's rss feed generated per request? Or is it a static output file? * If it's generated, why not simply use the same "format=" thing here. Note that Flickr finds it totally acceptable and ideal to use feeds in addition to their API

elij

11:02 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu, Oct 8, 2009 at 3:36 PM, Aaron Griffin wrote:

...

Flickr actually has two APIs - a feed based one and a REST based "ajax" API. Both accept a format=foo parameter and json is allowed for both sets.

* Is the AUR's rss feed generated per request? Or is it a static output file? * If it's generated, why not simply use the same "format=" thing here.

Note that Flickr finds it totally acceptable and ideal to use feeds in addition to their API

As I recall, the feed is generated, then saved to a static file. This static file is then served up php script reads it to stdout if not expired, until such time as it expires. Then it is generated again. It appears to work that way as an artifact of the php class/import that is being used. There appears to be no option (without adding it yourself) to either use an alternate cache mechanism (memcached) or to return the feed in alternate formats (other than rss2.0). This is _mostly_ painting the shed though (or format war, tabs vs spaces, etc). Yet, for some reason this just doesn't 'smell' right to me though. I can be a bit conservative at times though. It just seems to me that, getting a list of the latest updates is .. wait for ... what feeds are for.

Aaron Griffin

11:03 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu, Oct 8, 2009 at 6:02 PM, elij wrote:

...

On Thu, Oct 8, 2009 at 3:36 PM, Aaron Griffin wrote:

...
Flickr actually has two APIs - a feed based one and a REST based "ajax" API. Both accept a format=foo parameter and json is allowed for both sets.

* Is the AUR's rss feed generated per request? Or is it a static output file? * If it's generated, why not simply use the same "format=" thing here.

Note that Flickr finds it totally acceptable and ideal to use feeds in addition to their API

As I recall, the feed is generated, then saved to a static file. This static file is then served up php script reads it to stdout if not expired, until such time as it expires. Then it is generated again.

It appears to work that way as an artifact of the php class/import that is being used. There appears to be no option (without adding it yourself) to either use an alternate cache mechanism (memcached) or to return the feed in alternate formats (other than rss2.0).

This is _mostly_ painting the shed though (or format war, tabs vs spaces, etc). Yet, for some reason this just doesn't 'smell' right to me though. I can be a bit conservative at times though.

It just seems to me that, getting a list of the latest updates is .. wait for ... what feeds are for.

Yes, exactly... but a "feed" doesn't always have to be RSS :)

elij

11:30 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

Well, if people can get memcached installed on the aur box, and it can be made a requirement for running the aur, then I would have no problem coding up a new rss class that returned rss2.0 _or_ json, based on a passed in parameter. This could wholly replace the existing rss2.php/rss.php mechanism. I think adding alternate export types (json) to the rss mechanism makes more sense than adding rss logic to the rpc interface. Thoughts? (ooh. I am top-posting now.) On Thu, Oct 8, 2009 at 4:03 PM, Aaron Griffin wrote:

...

On Thu, Oct 8, 2009 at 6:02 PM, elij wrote:

...
On Thu, Oct 8, 2009 at 3:36 PM, Aaron Griffin wrote:

...
Flickr actually has two APIs - a feed based one and a REST based "ajax" API. Both accept a format=foo parameter and json is allowed for both sets.

* Is the AUR's rss feed generated per request? Or is it a static output file? * If it's generated, why not simply use the same "format=" thing here.

Note that Flickr finds it totally acceptable and ideal to use feeds in addition to their API

As I recall, the feed is generated, then saved to a static file. This static file is then served up php script reads it to stdout if not expired, until such time as it expires. Then it is generated again.

It appears to work that way as an artifact of the php class/import that is being used. There appears to be no option (without adding it yourself) to either use an alternate cache mechanism (memcached) or to return the feed in alternate formats (other than rss2.0).

This is _mostly_ painting the shed though (or format war, tabs vs spaces, etc). Yet, for some reason this just doesn't 'smell' right to me though. I can be a bit conservative at times though.

It just seems to me that, getting a list of the latest updates is .. wait for ... what feeds are for.

Yes, exactly... but a "feed" doesn't always have to be RSS :)

Dan McGee

10 Oct 10 Oct

2:55 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu, Oct 8, 2009 at 6:30 PM, elij wrote:

...

Well, if people can get memcached installed on the aur box, and it can be made a requirement for running the aur, then I would have no problem coding up a new rss class that returned rss2.0 _or_ json, based on a passed in parameter. This could wholly replace the existing rss2.php/rss.php mechanism.

I don't think requiring memcached would be a bad idea; maybe make it just decoupled enough that people could still develop locally without it being necessary (fall back to no caching at all). I already added some caching changes [1] a while back that uses APC if it is available, but it would be trivial to just migrate everything to memcached and put in a generic caching layer/API. Now let me know when you want to move the DB off of MySQL, and we'll talk. :P -Dan [1] http://projects.archlinux.org/?p=aur.git;a=commitdiff;h=5d6f465170392861c043...

elij

8:04 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

I made a first run through refactoring the feed code, and throwing memcache in. I need to setup a php test machine to test and verify though... So it might take a little while for me to make that happen. That is, unless someone has an aur instance running that is willing to test my patches (and setup memcached and the requisite php module). ;) On Sat, Oct 10, 2009 at 7:55 AM, Dan McGee wrote:

...

On Thu, Oct 8, 2009 at 6:30 PM, elij wrote:

...
Well, if people can get memcached installed on the aur box, and it can be made a requirement for running the aur, then I would have no problem coding up a new rss class that returned rss2.0 _or_ json, based on a passed in parameter. This could wholly replace the existing rss2.php/rss.php mechanism.

I don't think requiring memcached would be a bad idea; maybe make it just decoupled enough that people could still develop locally without it being necessary (fall back to no caching at all). I already added some caching changes [1] a while back that uses APC if it is available, but it would be trivial to just migrate everything to memcached and put in a generic caching layer/API.

Now let me know when you want to move the DB off of MySQL, and we'll talk. :P

-Dan

[1] http://projects.archlinux.org/?p=aur.git;a=commitdiff;h=5d6f465170392861c043...

Loui Chang

11 Oct 11 Oct

4:14 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu 08 Oct 2009 16:30 -0700, elij wrote:

...

Well, if people can get memcached installed on the aur box, and it can be made a requirement for running the aur, then I would have no problem coding up a new rss class that returned rss2.0 _or_ json, based on a passed in parameter. This could wholly replace the existing rss2.php/rss.php mechanism.

I think adding alternate export types (json) to the rss mechanism makes more sense than adding rss logic to the rpc interface.

Thoughts?

Actually I think it makes more sense to keep the json stuff together. Functions for newest packages, or other interactions could be put into 'include files' if that makes things neater.

elij

10:28 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Sun, Oct 11, 2009 at 9:14 AM, Loui Chang wrote:

...

On Thu 08 Oct 2009 16:30 -0700, elij wrote:

...
Well, if people can get memcached installed on the aur box, and it can be made a requirement for running the aur, then I would have no problem coding up a new rss class that returned rss2.0 _or_ json, based on a passed in parameter. This could wholly replace the existing rss2.php/rss.php mechanism.

I think adding alternate export types (json) to the rss mechanism makes more sense than adding rss logic to the rpc interface.

Thoughts?

Actually I think it makes more sense to keep the json stuff together. Functions for newest packages, or other interactions could be put into 'include files' if that makes things neater.

Well...I disagree. I don't think 'package update' rss-style feed data belongs in the api interface. Tell you what. I will test and submit my patches to the list, and then you guys can decide if you want them or not. I was just trying to be helpful and actually improve things. Silly me. *shrug*

Loui Chang

8 Oct 8 Oct

11:35 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu 08 Oct 2009 18:03 -0500, Aaron Griffin wrote:

...

On Thu, Oct 8, 2009 at 6:02 PM, elij wrote:

...
On Thu, Oct 8, 2009 at 3:36 PM, Aaron Griffin wrote:

...
Flickr actually has two APIs - a feed based one and a REST based "ajax" API. Both accept a format=foo parameter and json is allowed for both sets.

* Is the AUR's rss feed generated per request? Or is it a static output file? * If it's generated, why not simply use the same "format=" thing here.

Note that Flickr finds it totally acceptable and ideal to use feeds in addition to their API

As I recall, the feed is generated, then saved to a static file. This static file is then served up php script reads it to stdout if not expired, until such time as it expires. Then it is generated again.

It appears to work that way as an artifact of the php class/import that is being used. There appears to be no option (without adding it yourself) to either use an alternate cache mechanism (memcached) or to return the feed in alternate formats (other than rss2.0).

This is _mostly_ painting the shed though (or format war, tabs vs spaces, etc). Yet, for some reason this just doesn't 'smell' right to me though. I can be a bit conservative at times though.

It just seems to me that, getting a list of the latest updates is .. wait for ... what feeds are for.

Yes, exactly... but a "feed" doesn't always have to be RSS :)

We could change the API to XML and then use RSS. But I'm not sure that we could represent all the data we'd like to when it comes to the RSS. Maybe we could add a custom XML namespace into the RSS feed. We'd need to declare one anyways for the API and to validate everything. I don't know. I'm a big noob when it comes to these things. ...

elij

11:44 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu, Oct 8, 2009 at 4:35 PM, Loui Chang wrote:

...

On Thu 08 Oct 2009 18:03 -0500, Aaron Griffin wrote:

...
On Thu, Oct 8, 2009 at 6:02 PM, elij wrote:

...
On Thu, Oct 8, 2009 at 3:36 PM, Aaron Griffin wrote:

...
Flickr actually has two APIs - a feed based one and a REST based "ajax" API. Both accept a format=foo parameter and json is allowed for both sets.

* Is the AUR's rss feed generated per request? Or is it a static output file? * If it's generated, why not simply use the same "format=" thing here.

Note that Flickr finds it totally acceptable and ideal to use feeds in addition to their API

As I recall, the feed is generated, then saved to a static file. This static file is then served up php script reads it to stdout if not expired, until such time as it expires. Then it is generated again.

It appears to work that way as an artifact of the php class/import that is being used. There appears to be no option (without adding it yourself) to either use an alternate cache mechanism (memcached) or to return the feed in alternate formats (other than rss2.0).

This is _mostly_ painting the shed though (or format war, tabs vs spaces, etc). Yet, for some reason this just doesn't 'smell' right to me though. I can be a bit conservative at times though.

It just seems to me that, getting a list of the latest updates is .. wait for ... what feeds are for.

Yes, exactly... but a "feed" doesn't always have to be RSS :)

We could change the API to XML and then use RSS. But I'm not sure that we could represent all the data we'd like to when it comes to the RSS. Maybe we could add a custom XML namespace into the RSS feed. We'd need to declare one anyways for the API and to validate everything. I don't know. I'm a big noob when it comes to these things. ...

Change the api to xml? Namespaces? wtf are you talking about? We have RSS in xml format. And we are talking about adding a json output format representing the same data. rss.php?fmt=json (json format) rss.php?fmt=rss2 (rss2.0 format) rss.php (default would be rss2.0) This is because Laszlo thinks (and quite possibly so), that some tool writers might want the 'latest data' that the feeds puke out, available in the same json format as the api utils. I personally don't think that belongs in the aurrpc interface. I think feeds should be in their own 'feed interface'. Am I confused? Did I miss something?

Aaron Griffin

9 Oct 9 Oct

2:48 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Thu, Oct 8, 2009 at 6:44 PM, elij wrote:

...

On Thu, Oct 8, 2009 at 4:35 PM, Loui Chang wrote:

...
On Thu 08 Oct 2009 18:03 -0500, Aaron Griffin wrote:

...
On Thu, Oct 8, 2009 at 6:02 PM, elij wrote:

...
On Thu, Oct 8, 2009 at 3:36 PM, Aaron Griffin wrote:

...
Flickr actually has two APIs - a feed based one and a REST based "ajax" API. Both accept a format=foo parameter and json is allowed for both sets.

* Is the AUR's rss feed generated per request? Or is it a static output file? * If it's generated, why not simply use the same "format=" thing here.

Note that Flickr finds it totally acceptable and ideal to use feeds in addition to their API

As I recall, the feed is generated, then saved to a static file. This static file is then served up php script reads it to stdout if not expired, until such time as it expires. Then it is generated again.

It appears to work that way as an artifact of the php class/import that is being used. There appears to be no option (without adding it yourself) to either use an alternate cache mechanism (memcached) or to return the feed in alternate formats (other than rss2.0).

This is _mostly_ painting the shed though (or format war, tabs vs spaces, etc). Yet, for some reason this just doesn't 'smell' right to me though. I can be a bit conservative at times though.

It just seems to me that, getting a list of the latest updates is .. wait for ... what feeds are for.

Yes, exactly... but a "feed" doesn't always have to be RSS :)

We could change the API to XML and then use RSS. But I'm not sure that we could represent all the data we'd like to when it comes to the RSS. Maybe we could add a custom XML namespace into the RSS feed. We'd need to declare one anyways for the API and to validate everything. I don't know. I'm a big noob when it comes to these things. ...

Change the api to xml? Namespaces? wtf are you talking about?

We have RSS in xml format. And we are talking about adding a json output format representing the same data. rss.php?fmt=json (json format) rss.php?fmt=rss2 (rss2.0 format) rss.php (default would be rss2.0)

Bikeshedding, but if the format isn't always RSS, it shouldn't be called rss.php - perhaps feed.php

elij

4:20 p.m.

New subject: [aur-dev] [PATCH] Add newly submitted packages functionality with json interface

On Fri, Oct 9, 2009 at 7:48 AM, Aaron Griffin wrote:

...

On Thu, Oct 8, 2009 at 6:44 PM, elij wrote:

...
On Thu, Oct 8, 2009 at 4:35 PM, Loui Chang wrote:

...
On Thu 08 Oct 2009 18:03 -0500, Aaron Griffin wrote:

...
On Thu, Oct 8, 2009 at 6:02 PM, elij wrote:

...
On Thu, Oct 8, 2009 at 3:36 PM, Aaron Griffin wrote:

...
Flickr actually has two APIs - a feed based one and a REST based "ajax" API. Both accept a format=foo parameter and json is allowed for both sets.

* Is the AUR's rss feed generated per request? Or is it a static output file? * If it's generated, why not simply use the same "format=" thing here.

Note that Flickr finds it totally acceptable and ideal to use feeds in addition to their API

As I recall, the feed is generated, then saved to a static file. This static file is then served up php script reads it to stdout if not expired, until such time as it expires. Then it is generated again.

It appears to work that way as an artifact of the php class/import that is being used. There appears to be no option (without adding it yourself) to either use an alternate cache mechanism (memcached) or to return the feed in alternate formats (other than rss2.0).

This is _mostly_ painting the shed though (or format war, tabs vs spaces, etc). Yet, for some reason this just doesn't 'smell' right to me though. I can be a bit conservative at times though.

It just seems to me that, getting a list of the latest updates is .. wait for ... what feeds are for.

Yes, exactly... but a "feed" doesn't always have to be RSS :)

We could change the API to XML and then use RSS. But I'm not sure that we could represent all the data we'd like to when it comes to the RSS. Maybe we could add a custom XML namespace into the RSS feed. We'd need to declare one anyways for the API and to validate everything. I don't know. I'm a big noob when it comes to these things. ...

Change the api to xml? Namespaces? wtf are you talking about?

We have RSS in xml format. And we are talking about adding a json output format representing the same data. rss.php?fmt=json (json format) rss.php?fmt=rss2 (rss2.0 format) rss.php (default would be rss2.0)

Bikeshedding, but if the format isn't always RSS, it shouldn't be called rss.php - perhaps feed.php

Agreed. Also, Loui informed me on irc that he was being sarcastic. I didn't catch it with the missing sarcasm tag. :/

5320

Age (days ago)

5326

Last active (days ago)

List overview

Download

20 comments

7 participants

participants (7)

Aaron Griffin
Dan McGee
elij
Laszlo Papp
Laszlo Papp
Loui Chang
Randy Morris