Currently AUR web interface is able to resolve the dependencies in case if the dependendent package is provided by another package (see python2-catkin-pkg in the dependencies list): https://aur.archlinux.org/packages/ros-indigo-catkin/ However AUR RPC is not allowing querying by Provides field but only by `name`, `name-desc` and `maintainer` fields: https://aur.archlinux.org/rpc.php The discussion about that problem (https://github.com/ actionless/pikaur/issues/53) resulted in at least 3 possible ways to resolve that issue: - adding name-desc-provides filter - adding provides filter - when searching by name match not only Name field but also Provides field Please let me know if you're OK with one of the following approaches or propose another one. And if you fine with the idea let me know if you can implement such filter or I should send a patch. Also related ticket: https://bugs.archlinux.org/task/49090 All best, Yauheni Kirylau
On 03/07/2018 09:24 PM, Actionless Loveless wrote:
Currently AUR web interface is able to resolve the dependencies in case if the dependendent package is provided by another package (see python2-catkin-pkg in the dependencies list): https://aur.archlinux.org/packages/ros-indigo-catkin/
However AUR RPC is not allowing querying by Provides field but only by `name`, `name-desc` and `maintainer` fields: https://aur.archlinux.org/rpc.php
The discussion about that problem (https://github.com/ actionless/pikaur/issues/53) resulted in at least 3 possible ways to resolve that issue: - adding name-desc-provides filter - adding provides filter - when searching by name match not only Name field but also Provides field
Please let me know if you're OK with one of the following approaches or propose another one. And if you fine with the idea let me know if you can implement such filter or I should send a patch. Also related ticket: https://bugs.archlinux.org/task/49090
All best, Yauheni Kirylau
I think I would prefer extending `name` and `name-desc` as we always want to know any package that can be the searched-for name. And it is not like anyone was assuming only one result would be returned... this was never an exact name search to begin with. P.S. The email subject should probably refer to the topic matter, not the email of the person who suggested you post here. :) -- Eli Schwartz Bug Wrangler and Trusted User
Would you like me to send a patch for that? RE your PS: i think subject got auto-edited by mail list tool because i've put him into CC On Thu, Mar 8, 2018 at 3:44 AM, Eli Schwartz <eschwartz@archlinux.org> wrote:
On 03/07/2018 09:24 PM, Actionless Loveless wrote:
Currently AUR web interface is able to resolve the dependencies in case if the dependendent package is provided by another package (see python2-catkin-pkg in the dependencies list): https://aur.archlinux.org/packages/ros-indigo-catkin/
However AUR RPC is not allowing querying by Provides field but only by `name`, `name-desc` and `maintainer` fields: https://aur.archlinux.org/rpc.php
The discussion about that problem (https://github.com/ actionless/pikaur/issues/53) resulted in at least 3 possible ways to resolve that issue: - adding name-desc-provides filter - adding provides filter - when searching by name match not only Name field but also Provides field
Please let me know if you're OK with one of the following approaches or propose another one. And if you fine with the idea let me know if you can implement such filter or I should send a patch. Also related ticket: https://bugs.archlinux.org/task/49090
All best, Yauheni Kirylau
I think I would prefer extending `name` and `name-desc` as we always want to know any package that can be the searched-for name. And it is not like anyone was assuming only one result would be returned... this was never an exact name search to begin with.
P.S. The email subject should probably refer to the topic matter, not the email of the person who suggested you post here. :)
-- Eli Schwartz Bug Wrangler and Trusted User
Btw do you have docker or some other other kind of container prepopulated with AUR db so i can test the changes before submitting? On Thu, Mar 8, 2018 at 3:49 AM, Actionless Loveless < actionless.loveless@gmail.com> wrote:
Would you like me to send a patch for that?
RE your PS: i think subject got auto-edited by mail list tool because i've put him into CC
On Thu, Mar 8, 2018 at 3:44 AM, Eli Schwartz <eschwartz@archlinux.org> wrote:
On 03/07/2018 09:24 PM, Actionless Loveless wrote:
Currently AUR web interface is able to resolve the dependencies in case if the dependendent package is provided by another package (see python2-catkin-pkg in the dependencies list): https://aur.archlinux.org/packages/ros-indigo-catkin/
However AUR RPC is not allowing querying by Provides field but only by `name`, `name-desc` and `maintainer` fields: https://aur.archlinux.org/rpc.php
The discussion about that problem (https://github.com/ actionless/pikaur/issues/53) resulted in at least 3 possible ways to resolve that issue: - adding name-desc-provides filter - adding provides filter - when searching by name match not only Name field but also Provides field
Please let me know if you're OK with one of the following approaches or propose another one. And if you fine with the idea let me know if you can implement such filter or I should send a patch. Also related ticket: https://bugs.archlinux.org/task/49090
All best, Yauheni Kirylau
I think I would prefer extending `name` and `name-desc` as we always want to know any package that can be the searched-for name. And it is not like anyone was assuming only one result would be returned... this was never an exact name search to begin with.
P.S. The email subject should probably refer to the topic matter, not the email of the person who suggested you post here. :)
-- Eli Schwartz Bug Wrangler and Trusted User
On 03/07/2018 09:52 PM, Actionless Loveless wrote:
Btw do you have docker or some other other kind of container prepopulated with AUR db so i can test the changes before submitting? No, but there are instructions in the "TESTING" file.
-- Eli Schwartz Bug Wrangler and Trusted User
Done:
From 6f7e9c081706d1ef9996f6b3138a4bdf11a64675 Mon Sep 17 00:00:00 2001 From: actionless <actionless.loveless@gmail.com> Date: Thu, 8 Mar 2018 22:25:27 +0100 Subject: [PATCH] feat(rpc): return "providers" packages when querying by `name` or `name-desc`
--- doc/rpc.txt | 4 ++-- web/lib/aurjson.class.php | 16 +++++++++++++++- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/doc/rpc.txt b/doc/rpc.txt index f353ff0..83cdae3 100644 --- a/doc/rpc.txt +++ b/doc/rpc.txt @@ -8,8 +8,8 @@ Package searches can be performed by issuing HTTP GET requests of the form +/rpc/?v=5&type=search&by=_field_&arg=_keywords_+ where _keywords_ is the search argument and _field_ is one of the following values: -* `name` (search by package name only) -* `name-desc` (search by package name and description) +* `name` (search by package name or packages which provide that name) +* `name-desc` (the same as `name` and search by description) * `maintainer` (search by package maintainer) The _by_ parameter can be skipped and defaults to `name-desc`. diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index 9eeaafd..6e580a9 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -392,12 +392,26 @@ class AurJSON { if (strlen($keyword_string) < 2) { return $this->json_error('Query arg too small.'); } + + //packages which provide the package we are looking for: + $providers = pkg_providers(addcslashes($keyword_string, '%_')); + $provided_names = array(); + foreach ($providers as $provider) { + if ($provider[0] != 0) { // if package is not from repo + $name = $this->dbh->quote($provider[1]); + array_push($provided_names, $name); + } + } + $provided_query = "(" . join(", ", $provided_names) . ")"; + $keyword_string = $this->dbh->quote("%" . addcslashes($keyword_string, '%_') . "%"); if ($search_by === 'name') { - $where_condition = "(Packages.Name LIKE $keyword_string)"; + $where_condition = "(Packages.Name LIKE $keyword_string OR "; + $where_condition .= "Packages.Name IN $provided_query )"; } else if ($search_by === 'name-desc') { $where_condition = "(Packages.Name LIKE $keyword_string OR "; + $where_condition .= "Packages.Name IN $provided_query OR "; $where_condition .= "Description LIKE $keyword_string)"; } } else if ($search_by === 'maintainer') { -- 2.16.2
On 03/08/2018 04:27 PM, Actionless Loveless wrote:
Done:
Your patch does not appear to be git formatted, please resubmit it using for example git-send-email(1) -- Eli Schwartz Bug Wrangler and Trusted User
From: actionless <actionless.loveless@gmail.com> To: aur-dev@archlinux.org Cc: actionless <actionless.loveless@gmail.com> Subject: [PATCH] feat(rpc): return "providers" packages when querying by `name` or `name-desc` Date: Thu, 8 Mar 2018 22:50:13 +0100 Message-Id: <20180308215013.8626-1-actionless.loveless@gmail.com> X-Mailer: git-send-email 2.16.2 --- doc/rpc.txt | 4 ++-- web/lib/aurjson.class.php | 16 +++++++++++++++- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/doc/rpc.txt b/doc/rpc.txt index f353ff0..83cdae3 100644 --- a/doc/rpc.txt +++ b/doc/rpc.txt @@ -8,8 +8,8 @@ Package searches can be performed by issuing HTTP GET requests of the form +/rpc/?v=5&type=search&by=_field_&arg=_keywords_+ where _keywords_ is the search argument and _field_ is one of the following values: -* `name` (search by package name only) -* `name-desc` (search by package name and description) +* `name` (search by package name or packages which provide that name) +* `name-desc` (the same as `name` and search by description) * `maintainer` (search by package maintainer) The _by_ parameter can be skipped and defaults to `name-desc`. diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index 9eeaafd..6e580a9 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -392,12 +392,26 @@ class AurJSON { if (strlen($keyword_string) < 2) { return $this->json_error('Query arg too small.'); } + + //packages which provide the package we are looking for: + $providers = pkg_providers(addcslashes($keyword_string, '%_')); + $provided_names = array(); + foreach ($providers as $provider) { + if ($provider[0] != 0) { // if package is not from repo + $name = $this->dbh->quote($provider[1]); + array_push($provided_names, $name); + } + } + $provided_query = "(" . join(", ", $provided_names) . ")"; + $keyword_string = $this->dbh->quote("%" . addcslashes($keyword_string, '%_') . "%"); if ($search_by === 'name') { - $where_condition = "(Packages.Name LIKE $keyword_string)"; + $where_condition = "(Packages.Name LIKE $keyword_string OR "; + $where_condition .= "Packages.Name IN $provided_query )"; } else if ($search_by === 'name-desc') { $where_condition = "(Packages.Name LIKE $keyword_string OR "; + $where_condition .= "Packages.Name IN $provided_query OR "; $where_condition .= "Description LIKE $keyword_string)"; } } else if ($search_by === 'maintainer') { -- 2.16.2
On 03/08/2018 04:53 PM, Actionless Loveless wrote: This is still not a git formatted patch, it is a git-formatted patch that is then pasted into like Gmail's web interface or something. Look at the raw patch: https://paste.xinu.at/Qx2ys2gs/ Note the junk at the bottom where it includes the html version of the patch. Note the plaintext copy of the email which 1) translated all tab characters into spaces, and 2) occasionally adds extra linebreaks when it decides the line is too long. It is "possible" to convince some email clients to not completely mangle patches. (Gmail' web interface is not one of these.) But if you've never done it before... don't try. Just use git-send-email(1) which will ask for the email to/from addresses and can be given the in-reply-to message id, and will *guarantee* that the patch is not corrupted. -- Eli Schwartz Bug Wrangler and Trusted User
https://transfer.sh/L3H4Z/0001-feat-rpc-return-providers-packages-when-query... On Thu, Mar 8, 2018 at 11:04 PM, Eli Schwartz <eschwartz@archlinux.org> wrote:
On 03/08/2018 04:53 PM, Actionless Loveless wrote:
This is still not a git formatted patch, it is a git-formatted patch that is then pasted into like Gmail's web interface or something.
Look at the raw patch: https://paste.xinu.at/Qx2ys2gs/
Note the junk at the bottom where it includes the html version of the patch. Note the plaintext copy of the email which 1) translated all tab characters into spaces, and 2) occasionally adds extra linebreaks when it decides the line is too long.
It is "possible" to convince some email clients to not completely mangle patches. (Gmail' web interface is not one of these.) But if you've never done it before... don't try. Just use git-send-email(1) which will ask for the email to/from addresses and can be given the in-reply-to message id, and will *guarantee* that the patch is not corrupted.
-- Eli Schwartz Bug Wrangler and Trusted User
I'm pretty new to this mailing list. But I'm been thinking the same thing. Talked about it in a previous mail. https://lists.archlinux.org/pipermail/aur-general/2018-March/034001.html I think it's a great idea. When searching by name, i think the rpc should at least also search by the "replaces" attribute too. Provides would be a separate query but it could also be the same. Cheers, Mikael tors 8 mars 2018 kl. 03:25 skrev Actionless Loveless < actionless.loveless@gmail.com>:
Currently AUR web interface is able to resolve the dependencies in case if the dependendent package is provided by another package (see python2-catkin-pkg in the dependencies list): https://aur.archlinux.org/packages/ros-indigo-catkin/
However AUR RPC is not allowing querying by Provides field but only by `name`, `name-desc` and `maintainer` fields: https://aur.archlinux.org/rpc.php
The discussion about that problem (https://github.com/ actionless/pikaur/issues/53) resulted in at least 3 possible ways to resolve that issue: - adding name-desc-provides filter - adding provides filter - when searching by name match not only Name field but also Provides field
Please let me know if you're OK with one of the following approaches or propose another one. And if you fine with the idea let me know if you can implement such filter or I should send a patch. Also related ticket: https://bugs.archlinux.org/task/49090
All best, Yauheni Kirylau
On 03/08/2018 12:24 AM, Mikael Blomstrand wrote:
I'm pretty new to this mailing list. But I'm been thinking the same thing. Talked about it in a previous mail. https://lists.archlinux.org/pipermail/aur-general/2018-March/034001.html
I think it's a great idea. When searching by name, i think the rpc should at least also search by the "replaces" attribute too. Provides would be a separate query but it could also be the same.
That would have very little utility, as a package which replaces another package cannot satisfy a build dependency on it, whereas a provides is literally meant to ensure that that package is considered to be the same as the package it provides. It is semantically wrong, as that is simply not what a "replaces" keyword even means.... Its only conceivable use would be to enable AUR packages to automatically sync and replace existing packages, which would simply be incorrect. No thanks, let's stick to adding provides as an alias for the package they provide. -- Eli Schwartz Bug Wrangler and Trusted User
tors 8 mars 2018 kl. 13:39 skrev Eli Schwartz <eschwartz@archlinux.org>:
On 03/08/2018 12:24 AM, Mikael Blomstrand
When searching by name, i think the rpc should at least also search by the "replaces" attribute too. Provides would be a separate query but it could also be the same.
That would have very little utility, as a package which replaces another package cannot satisfy a build dependency on it, whereas a provides is literally meant to ensure that that package is considered to be the same as the package it provides.
I missed this before replying before since the thread switched name again. OK, maybe I've misunderstood the purpose of replaces when reading the man page. It's been my impression that if package A replaces B, then A must be the "new" B, and everyone using B should switch to A. I just assumed that this also means A should be able to be used in place of B. I had an idea that it would be nice to have merged packages "replaced" by the new base. This might be completely stupid, but it was my thought. It is semantically wrong, as that is simply not what a "replaces"
keyword even means...
Would you mind explaining what it means or what correct use would be if I've misunderstood the man pages? :) Its only conceivable use would be to enable AUR packages to
automatically sync and replace existing packages, which would simply be incorrect.
Yes, having packages in AUR arbitrarily replace packages is obviously a bad idea. It's not really what I had in mind either. My thought was more that a AUR helper that is smart enough could see that an installed package has been removed from the AUR, and if there is a package in the AUR replacing that package, then it could ask the user if they want to look at the PKGBUILD for that instead. This is relevant if a user installs package foo-bar which provides foo. When searching for "foo-bar", "foo" won't show up, because it doesn't provide "foo-bar". It could however replace "foo-bar". Is this still unsound? Regards, Mikael
On 03/08/2018 05:20 PM, Mikael Blomstrand wrote:
I missed this before replying before since the thread switched name again.
OK, maybe I've misunderstood the purpose of replaces when reading the man page. It's been my impression that if package A replaces B, then A must be the "new" B, and everyone using B should switch to A. I just assumed that this also means A should be able to be used in place of B.
I had an idea that it would be nice to have merged packages "replaced" by the new base.
This might be completely stupid, but it was my thought.
It is semantically wrong, as that is simply not what a "replaces" keyword even means...
Would you mind explaining what it means or what correct use would be if I've misunderstood the man pages? :)
If you are searching the AUR for packages that "provide" foo, adding packages that "replace" it will not help you resolve dependencies. It would help in the case where one package is deleted from the AUR and renamed to something else, the same way it helps in the official repositories when a package is deleted and renamed to something else. However, in that case pacman tells you about the replacement and asks if you would like to uninstall foo and install bar instead. Therefore, if it would be an available search at all, then rather than adding replaces to name and adding provides to a separate keyword, we should do the exact opposite and add provides to name and name-desc (which this thread is about), and add replaces to a separate keyword. Then AUR helpers could theoretically offer to replace packages with completely different "successor" packages, by searching all foreign installed pkgnames and seeing if there is a replacement available. This would be completely different from the "search for packages for dependency resolution" bit.
Its only conceivable use would be to enable AUR packages to automatically sync and replace existing packages, which would simply be incorrect.
Yes, having packages in AUR arbitrarily replace packages is obviously a bad idea. It's not really what I had in mind either. My thought was more that a AUR helper that is smart enough could see that an installed package has been removed from the AUR, and if there is a package in the AUR replacing that package, then it could ask the user if they want to look at the PKGBUILD for that instead.
This is relevant if a user installs package foo-bar which provides foo. When searching for "foo-bar", "foo" won't show up, because it doesn't provide "foo-bar". It could however replace "foo-bar".
Is this still unsound?
Well, as I said you do not want it available as "name", if I try to install a given package then offering to install something totally different is wrong. Even pacman will allow you to install a pkgname that you specify regardless of whether there is a replaces available instead.... And generally speaking, the rate at which the replaces keyword is misused in AUR packages makes me think any implementation would simply not work out in practice. -- Eli Schwartz Bug Wrangler and Trusted User
participants (3)
-
Actionless Loveless
-
Eli Schwartz
-
Mikael Blomstrand