[aur-dev] Method to get all packages
Hey all! Loui Chang pointed me here since I prepared a patch to get all packages from AUR via the rpc-Interface... The reason why I need to get all packages is to build a list of available packages (which gets cached then...), the user can search in it then... (or just dig a bit in it...). This all is for the AUR-Intergration into Shaman2 (you might probably already know the 1.x version which is in community...)... Now the patch is attached, it's not tested at all and I don't have much knowledge of php/sql/json at all, so it's mostly a copy-over from the other methods... Lukas
On Sun, Jan 25, 2009 at 07:39:26PM +0100, Lukas Appelhans wrote:
The reason why I need to get all packages is to build a list of available packages (which gets cached then...), the user can search in it then... (or just dig a bit in it...). This all is for the AUR-Intergration into Shaman2 (you might probably already know the 1.x version which is in community...)...
Alright. That's not so bad, but we'd have to make sure this is aggressively cached on the server side as well to prevent abuse. Updating only every 12 to 24 hours.
On Mittwoch 28 Januar 2009 03:55:30 Loui Chang wrote:
On Sun, Jan 25, 2009 at 07:39:26PM +0100, Lukas Appelhans wrote:
The reason why I need to get all packages is to build a list of available packages (which gets cached then...), the user can search in it then... (or just dig a bit in it...). This all is for the AUR-Intergration into Shaman2 (you might probably already know the 1.x version which is in community...)...
Alright. That's not so bad, but we'd have to make sure this is aggressively cached on the server side as well to prevent abuse. Updating only every 12 to 24 hours. Ok :) I attached a patch, which makes update it every 24 Hours (first call after new day... ;))...
Untested though (as usual) ;) Comments? Lukas
On Thu, Jan 29, 2009 at 8:17 AM, Lukas Appelhans <l.appelhans@gmx.de> wrote:
On Mittwoch 28 Januar 2009 03:55:30 Loui Chang wrote:
On Sun, Jan 25, 2009 at 07:39:26PM +0100, Lukas Appelhans wrote:
The reason why I need to get all packages is to build a list of available packages (which gets cached then...), the user can search in it then... (or just dig a bit in it...). This all is for the AUR-Intergration into Shaman2 (you might probably already know the 1.x version which is in community...)...
Alright. That's not so bad, but we'd have to make sure this is aggressively cached on the server side as well to prevent abuse. Updating only every 12 to 24 hours. Ok :) I attached a patch, which makes update it every 24 Hours (first call after new day... ;))...
Untested though (as usual) ;)
Comments?
Your caching method isn't going to work- PHP isn't going to store your state between requests. Loui and Lukas- I would recommend looking at APC for this, the same way I modified the stats page. Of course, if we are going to use it outside of that page, we should probably look to make it a universally usable interface. -Dan
On Thu, Jan 29, 2009 at 03:17:54PM +0100, Lukas Appelhans wrote:
Ok :) I attached a patch, which makes update it every 24 Hours (first call after new day... ;))...
Untested though (as usual) ;)
Comments?
Please submit patches formatted with `git format-patch`. Please test all your patches before submission. I've added a bit more about submitting patches in HACKING.
On Donnerstag 29 Januar 2009 16:25:50 Loui Chang wrote:
On Thu, Jan 29, 2009 at 03:17:54PM +0100, Lukas Appelhans wrote:
Ok :) I attached a patch, which makes update it every 24 Hours (first call after new day... ;))...
Untested though (as usual) ;)
Comments?
Please submit patches formatted with `git format-patch`. Please test all your patches before submission. I've added a bit more about submitting patches in HACKING. Sorry for not testing, but I have no Apache nor anything Web-related running on my pc and don't want to run it ;) Good that a friend gave me remote access to his server, so I can test the stuff from now ;)
@Dan: Ok, thanks, I will check that stuff... Lukas PS: New patch might take a bit, I'm way busy with some other programming things ;)
On Fri, Jan 30, 2009 at 02:02:39AM +0100, Lukas Appelhans wrote:
On Donnerstag 29 Januar 2009 16:25:50 Loui Chang wrote:
On Thu, Jan 29, 2009 at 03:17:54PM +0100, Lukas Appelhans wrote:
Ok :) I attached a patch, which makes update it every 24 Hours (first call after new day... ;))...
Untested though (as usual) ;)
Comments?
Please submit patches formatted with `git format-patch`. Please test all your patches before submission. I've added a bit more about submitting patches in HACKING. Sorry for not testing, but I have no Apache nor anything Web-related running on my pc and don't want to run it ;) Good that a friend gave me remote access to his server, so I can test the stuff from now ;)
@Dan: Ok, thanks, I will check that stuff...
Lukas
PS: New patch might take a bit, I'm way busy with some other programming things ;)
I was chatting and brainstorming with Callan about this. We were thinking that it would be really cool to have all information in a pacman-like database that people could download a tarball of once a day. Anyways we still really have to see how heavy even this json method is going to be. Removing comments, user info, and some other unneeded things a MySQL DB dump from Dec weighs in at 2.5M. Uncompressed it's 9M. Just throwing out some ideas that could be considered here.
participants (3)
-
Dan McGee
-
Loui Chang
-
Lukas Appelhans