[aur-dev] Making the AUR package list more useful

[Dave Reisner]

On Apr 29, 2016 02:50, "Lukas Fleischer" <lfleischer at archlinux.org> wrote:
>
> Recently, there were a couple of feature requests to make the AUR
> package search more powerful. While I do not plan on adding more
> patterns or regular expressions to the RPC interface itself, my idea is
> that more tools should be using the package name list. However, there
> seem to be two issues with that:

What are your reasons for not wanting to extend the RPC interface?
Splitting the API in this way is cumbersome for clients. We now are
supposed to download and maintain data which is essentially an index (which
ought to be internal to the server to allow strong consistency). You're
also asking clients of a read-only interface to become stateful, which
isn't something that really interests me.

> 1. The list is outdated. Right now, it is updated every two hours. I
>    do not think there is a good reason for those long intervals.
>    Reducing it to, say, ten minutes should be totally fine. Or maybe
>    even trigger list generation whenever a package is created or deleted
>    (which is clearly a lot more work, though). Thoughts?

Or, change the storage for the name list such that updates can be fast.
Turns out, you already have such a thing, you'd just need an index on the
Packages and PackageBases tables.

> 2. Transferring the whole package name list is inefficient. Even if we
>    use gzip compression here, the whole list is several hundreds of
>    kilobytes large. We need to retransfer the full list, even if only a
>    single package is added. Maybe we can do better than pacman here. My
>    idea is to add zsync support to the lists such that only relevant
>    parts are downloaded (for those who do not know: zsync is like rsync
>    but it works via HTTP as well and does not require any special
>    software on the server side). I did not experiment with how much
>    bandwidth we can actually save using this yet. Maybe the block size
>    needs to be adjusted. Are there any opinions or other suggestions on
>    this topic?

I'm not understanding why any of this is considered a good direction for
the API. What are the reasons for wanting the whole list of package names
on the client side? Are there use cases other than search?

> Regards,
> Lukas