[aur-general] AUR RPC search results custom fields
Hi, I just subscribed to the AUR mailing list. I developed tupac, a script that does fast aur searches. I got really glad when i found that there is an RPC for AUR, but all my happiness vanished when i saw that the search results show no information, and that i have to add another query for each result if i want to show aditional information to the search results. From my house, this adds a 550 ms query to each result. I gave a try to arson to proove it. Adding the possibility of specifying the fields you want to show in the search results would reduce rpc query time and would also reduce server load. Getting custom fields would be 1 connection vs N+1 (N=number results, 1 the search) connections that you have to do now for a pacman-like search. And in what time regards, as an example, for 30 results now I need (30+1)*550ms = around 17 seconds. I also see that this would be more intense work for the database, so it would be reasonable to add a result limit to the search, if you use custom fields, in order to reduce subqueries. And, about usability, imagine that http://aur.archlinux.org/packages.php took 13 seconds to load. What do you think, did I sound convincing enough? :P
Francesc Ortiz wrote:
Hi,
I just subscribed to the AUR mailing list. I developed tupac, a script that does fast aur searches. I got really glad when i found that there is an RPC for AUR, but all my happiness vanished when i saw that the search results show no information, and that i have to add another query for each result if i want to show aditional information to the search results. From my house, this adds a 550 ms query to each result. I gave a try to arson to proove it.
Adding the possibility of specifying the fields you want to show in the search results would reduce rpc query time and would also reduce server load. Getting custom fields would be 1 connection vs N+1 (N=number results, 1 the search) connections that you have to do now for a pacman-like search. And in what time regards, as an example, for 30 results now I need (30+1)*550ms = around 17 seconds.
I also see that this would be more intense work for the database, so it would be reasonable to add a result limit to the search, if you use custom fields, in order to reduce subqueries.
And, about usability, imagine that http://aur.archlinux.org/packages.php took 13 seconds to load.
What do you think, did I sound convincing enough? :P
Welcome. You probably want to try the aur-dev mailing list for this as that is where the people who develop the AUR hang out. Your idea sounds reasonable but I have no idea if it is implementable. Of course, if you know how those sort of things work, you can alway help out improving the AUR! Cheers, Allan
On Wed, 7 May 2008 08:35:34 +0200 "Francesc Ortiz" <francescortiz@gmail.com> wrote:
Hi,
Adding the possibility of specifying the fields you want to show in the search results would reduce rpc query time and would also reduce server load. Getting custom fields would be 1 connection vs N+1 (N=number results, 1 the search) connections that you have to do now for a pacman-like search. And in what time regards, as an example, for 30 results now I need (30+1)*550ms = around 17 seconds.
I also see that this would be more intense work for the database, so it would be reasonable to add a result limit to the search, if you use custom fields, in order to reduce subqueries.
I totally agree with you. The RPC interface should really be at least as powerful as the web search, but no one has really had the motivation to write a patch for that yet heh. If you're interested, you can check out the AUR source code at http://projects.archlinux.org/git/?p=aur.git write a patch and submit it to aur-dev@archlinux.org. Cheers.
Hey, I created a patch for custom fields in searches, but i don't where to refer to, and i can't test it. I have uploaded it to http://lapacaloca.com/arch/aur-rpc-custom-fields.patch What should I do now? Thank you On 5/7/08, Loui <louipc.ist@gmail.com> wrote:
On Wed, 7 May 2008 08:35:34 +0200 "Francesc Ortiz" <francescortiz@gmail.com> wrote:
Hi,
Adding the possibility of specifying the fields you want to show in the search results would reduce rpc query time and would also reduce server load. Getting custom fields would be 1 connection vs N+1 (N=number results, 1 the search) connections that you have to do now for a pacman-like search. And in what time regards, as an example, for 30 results now I need (30+1)*550ms = around 17 seconds.
I also see that this would be more intense work for the database, so it would be reasonable to add a result limit to the search, if you use custom fields, in order to reduce subqueries.
I totally agree with you. The RPC interface should really be at least as powerful as the web search, but no one has really had the motivation to write a patch for that yet heh. If you're interested, you can check out the AUR source code at http://projects.archlinux.org/git/?p=aur.git write a patch and submit it to aur-dev@archlinux.org.
Cheers.
participants (3)
-
Allan McRae
-
Francesc Ortiz
-
Loui