[aur-dev] Wildcards in search queries
I just recently noticed that the wildcards have been disabled in aur-1.9.1 - which is a pity, since it has removed some fairly useful functionality. I found bug FS#26527 that prompted the change, and read through the links - but I couldn't completely understand the rationale for it. It looks like there may be a problem with search queries that consist of only '%' wildcards, as that could match everything in the database. But wouldn't it be fairly easy to eliminate pathological cases like that? And aren't there already checks in place to limit the number of results returned? For me, the most useful feature of wildcards is that they can be used to *reduce* the number of results returned - which is somewhat ironic, if I have interpreted the rationale for this change correctly ;-)
On Thu, Oct 04, 2012 at 06:59:49PM +0100, kachelaqa wrote:
I just recently noticed that the wildcards have been disabled in aur-1.9.1 - which is a pity, since it has removed some fairly useful functionality.
I found bug FS#26527 that prompted the change, and read through the links - but I couldn't completely understand the rationale for it.
It looks like there may be a problem with search queries that consist of only '%' wildcards, as that could match everything in the database.
But wouldn't it be fairly easy to eliminate pathological cases like that? And aren't there already checks in place to limit the number of results returned?
For me, the most useful feature of wildcards is that they can be used to *reduce* the number of results returned - which is somewhat ironic, if I have interpreted the rationale for this change correctly ;-)
I never even knew this feature existed. <shameless bump> If you want something that's actually good at searching the AUR, I suggest using cower. It hackishly implements regex and boolean AND'd results. I've been very happy with it. </shameless bump>
On 04/10/12 19:06, Dave Reisner wrote:
On Thu, Oct 04, 2012 at 06:59:49PM +0100, kachelaqa wrote:
I just recently noticed that the wildcards have been disabled in aur-1.9.1 - which is a pity, since it has removed some fairly useful functionality.
I never even knew this feature existed.
<shameless bump> If you want something that's actually good at searching the AUR, I suggest using cower. It hackishly implements regex and boolean AND'd results. I've been very happy with it. </shameless bump>
There are at least two utilities i know of that currently use this feature - package-query (which is used by yaourt), and pkgbrowser. The main reason for my asking about this, is that I sometimes use the former two utilities - and I am the author of the latter one :)
On Thu, Oct 4, 2012 at 1:59 PM, kachelaqa <kachelaqa@gmail.com> wrote:
I just recently noticed that the wildcards have been disabled in aur-1.9.1 - which is a pity, since it has removed some fairly useful functionality.
I found bug FS#26527 that prompted the change, and read through the links - but I couldn't completely understand the rationale for it.
It looks like there may be a problem with search queries that consist of only '%' wildcards, as that could match everything in the database.
But wouldn't it be fairly easy to eliminate pathological cases like that? And aren't there already checks in place to limit the number of results returned?
For me, the most useful feature of wildcards is that they can be used to *reduce* the number of results returned - which is somewhat ironic, if I have interpreted the rationale for this change correctly ;-)
It's good practice to escape '%' and '_' wildcards. Without those wildcards escaped it is more difficult for a user to search for those characters literally. There is also a potential for a simple denial of service attack if a malicious visitor created a search string using wildcards that had the sole purpose of consuming server resources. I apologize that the change created problems in your AUR helper, but I don't think it is something that should be undone. It has been in place for over 7 months now and many users didn't even know it was previously possible. Any functionality that depended on it was depending on a bug. You may want to look at how some other AUR helpers implement AUR searching if you would like ideas for your own AUR helper. Regards, Jason
On 04/10/12 22:24, canyonknight@gmail.com wrote:
It's good practice to escape '%' and '_' wildcards. Without those wildcards escaped it is more difficult for a user to search for those characters literally.
They can be escaped with '\', but it's true that the underscores can cause a problem if there is only one method of searching.
There is also a potential for a simple denial of service attack if a malicious visitor created a search string using wildcards that had the sole purpose of consuming server resources.
But isn't is just search strings like '%%%' that are a problem (which could be easily to screened out)?
I apologize that the change created problems in your AUR helper, but I don't think it is something that should be undone. It has been in place for over 7 months now and many users didn't even know it was previously possible. Any functionality that depended on it was depending on a bug.
No apology needed :) You haven't really caused much of a problem - it's just a loss of some functionality that I was hoping could be avoided.
You may want to look at how some other AUR helpers implement AUR searching if you would like ideas for your own AUR helper.
The main purpose of my program (which has been around for 18-months or so), is to integrate searching the user's local pacman databases with the AUR. The wildcards were handy, because I could translate them into a simple glob-style syntax that would work equally well in both contexts. So I mainly wanted to confirm that there is no chance of wildcards being brought back before I removed the functionality from my own program. Thanks for the feedback.
participants (3)
-
canyonknight＠gmail.com
-
Dave Reisner
-
kachelaqa