February 2019 - Aur-dev - lists.archlinux.org

[aur-dev][PATCH] Correctly handle package sources which do not validate as an url
by Eli Schwartz 28 Apr '19

28 Apr '19

php's parse_url does not handle proper rfc3986 URIs, specifically, it does not handle the case of an empty authority such as file:/// or local:/// and only handles the case of file by applying a special case for file itself. These URIs are deemed "malformed" and return false. When such URIs were used, we would end up always treating the package source as a filename (despite that this is incorrect, since plain files will be correctly handled by parse_url, we will correctly determine that there is no schema, and we will go to the source_file_uri). Instead, handle the case of a "malformed" URI by treating it as another example of a source with a schema, and linking it as-is. See https://lists.archlinux.org/pipermail/aur-general/2019-January/034782.html for details. Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- This fixes the case of local:///, but are there other cases where php would claim a malformed url where we would actually want to link to cgit? Should we just be dumb like makepkg and git/update.py, and check if it has the string literal '://'? Given the other two places where a source url might be handled don't even make a pretense of being proper rfc3986 parsers, this would at least mean we're highly consistent in our behavior. web/lib/pkgfuncs.inc.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/lib/pkgfuncs.inc.php b/web/lib/pkgfuncs.inc.php index ced1f8e..126b5c3 100644 --- a/web/lib/pkgfuncs.inc.php +++ b/web/lib/pkgfuncs.inc.php @@ -481,7 +481,7 @@ function pkg_source_link($url, $arch, $package) { $url = explode('::', $url); $parsed_url = parse_url($url[0]); - if (isset($parsed_url['scheme']) || isset($url[1])) { + if ($parsed_url === false || isset($parsed_url['scheme']) || isset($url[1])) { $link = '<a href="' . htmlspecialchars((isset($url[1]) ? $url[1] : $url[0]), ENT_QUOTES) . '">' . htmlspecialchars($url[0]) . '</a>'; } else { $file_url = sprintf(config_get('options', 'source_file_uri'), htmlspecialchars($url[0]), $package); -- 2.20.1

2 4

[PATCH] Fix German translation for DISCLAIMER line on the home page.
by Kevin Morris 25 Feb '19

25 Feb '19

Something was leftover (probably from a previous update?) Took out the extra line to sync up with how the other languages handle it. Signed-off-by: Kevin Morris <kevr.gtalk(a)gmail.com> --- po/de.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/po/de.po b/po/de.po index 00ab95c..ee5631d 100644 --- a/po/de.po +++ b/po/de.po @@ -233,7 +233,7 @@ msgstr "Manche Pakete könnten als Binär-Pakete in [community] bereitgestellt s #: html/home.php msgid "DISCLAIMER" -msgstr "DISCLAIMER: Offiziell nicht unterstützte PKGBUILDS werden von den Benutzern erstellt - durch den Download willigt man ein, diese auf eigene Gefahr zu benutzen." +msgstr "DISCLAIMER" #: html/home.php template/footer.php msgid "" -- 2.20.1

1 0

[PATCH] feat(rpc): return "providers" packages when querying by `name` or `name-desc`
by morganamilo 18 Feb '19

18 Feb '19

From: actionless <actionless.loveless(a)gmail.com> --- doc/rpc.txt | 4 ++-- web/lib/aurjson.class.php | 16 +++++++++++++++- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/doc/rpc.txt b/doc/rpc.txt index f353ff0..83cdae3 100644 --- a/doc/rpc.txt +++ b/doc/rpc.txt @@ -8,8 +8,8 @@ Package searches can be performed by issuing HTTP GET requests of the form +/rpc/?v=5&type=search&by=_field_&arg=_keywords_+ where _keywords_ is the search argument and _field_ is one of the following values: -* `name` (search by package name only) -* `name-desc` (search by package name and description) +* `name` (search by package name or packages which provide that name) +* `name-desc` (the same as `name` and search by description) * `maintainer` (search by package maintainer) The _by_ parameter can be skipped and defaults to `name-desc`. diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index 9eeaafd..6e580a9 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -392,12 +392,26 @@ class AurJSON { if (strlen($keyword_string) < 2) { return $this->json_error('Query arg too small.'); } + + //packages which provide the package we are looking for: + $providers = pkg_providers(addcslashes($keyword_string, '%_')); + $provided_names = array(); + foreach ($providers as $provider) { + if ($provider[0] != 0) { // if package is not from repo + $name = $this->dbh->quote($provider[1]); + array_push($provided_names, $name); + } + } + $provided_query = "(" . join(", ", $provided_names) . ")"; + $keyword_string = $this->dbh->quote("%" . addcslashes($keyword_string, '%_') . "%"); if ($search_by === 'name') { - $where_condition = "(Packages.Name LIKE $keyword_string)"; + $where_condition = "(Packages.Name LIKE $keyword_string OR "; + $where_condition .= "Packages.Name IN $provided_query )"; } else if ($search_by === 'name-desc') { $where_condition = "(Packages.Name LIKE $keyword_string OR "; + $where_condition .= "Packages.Name IN $provided_query OR "; $where_condition .= "Description LIKE $keyword_string)"; } } else if ($search_by === 'maintainer') { -- 2.16.2

6 25

Django-hosted aurweb Proposal
by Kevin Morris 18 Feb '19

18 Feb '19

To whom it may concern, The following is a proposal for a Django-hosted aurweb application. It is meant to be a drop-in replacement for aur.archlinux.org; effectively cloning its user interface and capabilities into a Python Django extension. Following https://patchwork.archlinux.org/patch/1000/, I would like to put together a new aurweb. The current revamped version of archweb runs inside of a django server as a django extension (or app). I would like to do this same thing for `aurweb`. The new django `aurweb` shall support all of the current v1-6 capabilities that aurweb provides, as well as the front-end user website located at https://aur.archlinux.org. It shall be an exact clone from the user's perspective. The major differences between maintaining a PHP vs Django server would be that the Django server would be: 1. Integrated with any Django server and easily removed. 2. Developed via Python, which has vastly more support and familiarity than PHP within the community. 3. Django models, which greatly reduce the complexity of SQL queries inside of `aurweb` backend. 4. Baked-in extendable templates with templatetags. 5. View decorators which enforce requirements of a specific route. No more manually handling $http_data user errors. 6. Everybody loves Python; there will most likely be many more code-base contributions in Python than in PHP. I would like to hear your thoughts on this. If approved, I would love to begin this project within the next few weeks. Thanks, Kevin

6 6

[PATCH] Introduce API v6
by Kevin Morris 12 Feb '19

12 Feb '19

Add new method of multiple by[] search in API v6. The 'provides' by field is also supported in the new multiple by[] method included with this patch. 'provides' may not be used in legacy (v5) singular by search. A helpful json error message is returned to the user to indicate so. This patch is a follow-up implementation to https://patchwork.archlinux.org/patch/479/ Signed-off-by: Kevin Morris <kevr.gtalk(a)gmail.com> --- doc/rpc.txt | 24 ++++++--- web/lib/aurjson.class.php | 105 ++++++++++++++++++++++++++++++++++++-- 2 files changed, 116 insertions(+), 13 deletions(-) diff --git a/doc/rpc.txt b/doc/rpc.txt index 3148ebe..486a49d 100644 --- a/doc/rpc.txt +++ b/doc/rpc.txt @@ -5,7 +5,7 @@ Package Search -------------- Package searches can be performed by issuing HTTP GET requests of the form -+/rpc/?v=5&type=search&by=_field_&arg=_keywords_+ where _keywords_ is the ++/rpc/?v=6&type=search&by[]=_field_&arg[]=_keywords_+ where _keywords_ is the search argument and _field_ is one of the following values: * `name` (search by package name only) @@ -15,31 +15,39 @@ search argument and _field_ is one of the following values: * `makedepends` (search for packages that makedepend on _keywords_) * `optdepends` (search for packages that optdepend on _keywords_) * `checkdepends` (search for packages that checkdepend on _keywords_) +* `provides` (search by package provides; v6 multiple by[] only) The _by_ parameter can be skipped and defaults to `name-desc`. If a maintainer search is performed and the search argument is left empty, a list of orphan packages is returned. +Note: Legacy v5 support is still enabled for singular _by_ searches. + Package Details --------------- Package information can be obtained by issuing HTTP GET requests of the form -+/rpc/?v=5&type=info&arg[]=_pkg1_&arg[]=_pkg2_&...+ where _pkg1_, _pkg2_, ... ++/rpc/?v=6&type=info&arg[]=_pkg1_&arg[]=_pkg2_&...+ where _pkg1_, _pkg2_, ... are the names of packages to retrieve package details for. Examples -------- `search`:: - `/rpc/?v=5&type=search&arg=foobar` + `/rpc/?v=6&type=search&arg=foobar` `search` by maintainer:: - `/rpc/?v=5&type=search&by=maintainer&arg=john` + `/rpc/?v=6&type=search&by[]=maintainer&arg=john` `search` packages that have _boost_ as `makedepends`:: - `/rpc/?v=5&type=search&by=makedepends&arg=boost` + `/rpc/?v=6&type=search&by[]=makedepends&arg=boost` `search` with callback:: - `/rpc/?v=5&type=search&arg=foobar&callback=jsonp1192244621103` + `/rpc/?v=6&type=search&arg=foobar&callback=jsonp1192244621103` +`search` by provides:: + `/rpc/?v=6&type=search&by[]=provides&arg[]=gcc` +`search` by provides and name:: + `/rpc/?v=6&type=search&by[]=name&by[]=provides&arg[]=gcc` `info`:: - `/rpc/?v=5&type=info&arg[]=foobar` + `/rpc/?v=6&type=info&arg[]=foobar` `info` with multiple packages:: - `/rpc/?v=5&type=info&arg[]=foo&arg[]=bar` + `/rpc/?v=6&type=info&arg[]=foo&arg[]=bar` + diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index c275d21..ab4eb19 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -80,7 +80,8 @@ class AurJSON { if (isset($http_data['v'])) { $this->version = intval($http_data['v']); } - if ($this->version < 1 || $this->version > 5) { + + if ($this->version < 1 || $this->version > 6) { return $this->json_error('Invalid version specified.'); } @@ -94,8 +95,14 @@ class AurJSON { if (isset($http_data['search_by']) && !isset($http_data['by'])) { $http_data['by'] = $http_data['search_by']; } - if (isset($http_data['by']) && !in_array($http_data['by'], self::$exposed_fields)) { - return $this->json_error('Incorrect by field specified.'); + + if (isset($http_data['by']) && !is_array($http_data['by'])) { + if ($http_data['by'] === 'provides') { + return $this->json_error("The 'provides' by field " . + "may only be used via multiple by[] search."); + } elseif (!in_array($http_data['by'], self::$exposed_fields)) { + return $this->json_error('Incorrect by field specified.'); + } } $this->dbh = DB::connect(); @@ -360,7 +367,7 @@ class AurJSON { } elseif ($this->version >= 2) { if ($this->version == 2 || $this->version == 3) { $fields = implode(',', self::$fields_v2); - } else if ($this->version == 4 || $this->version == 5) { + } else if ($this->version == 4 || $this->version == 5 || $this->version == 6) { $fields = implode(',', self::$fields_v4); } $query = "SELECT {$fields} " . @@ -470,6 +477,10 @@ class AurJSON { * @return mixed Returns an array of package matches. */ private function search($http_data) { + if ($this->version == 6 && is_array($http_data['by'])) { + return $this->search_v6($http_data); + } + $keyword_string = $http_data['arg']; if (isset($http_data['by'])) { @@ -497,6 +508,8 @@ class AurJSON { $keyword_string = $this->dbh->quote($keyword_string); $where_condition = "Users.Username = $keyword_string "; } + } else if ($search_by === 'provides') { + return $this->json_error("The 'provides' by field is only available via multiple by[] search."); } else if (in_array($search_by, self::$exposed_depfields)) { if (empty($keyword_string)) { return $this->json_error('Query arg is empty.'); @@ -515,6 +528,89 @@ class AurJSON { return $this->process_query('search', $where_condition); } + /* + * Returns multiple-by, multiple-arg and/or search information + * a feature not included in the v5 and before RPC API, + * which introduces the beginnings of the v6 API. + * + * @param array $http_data Query parameters. + * + * @return mixed Returns an array of results containing the package data + */ + private function search_v6($http_data) { + if (!is_array($http_data['arg'])) { + $http_data['arg'] = array($http_data['arg']); + } + + $max_results = config_get_int('options', 'max_rpc_results'); + $fields = implode(',', self::$fields_v4); + + $query = "SELECT {$fields} FROM " . + "Packages LEFT JOIN PackageBases " . + "ON PackageBases.ID = Packages.PackageBaseID " . + "LEFT JOIN Users " . + "ON PackageBases.MaintainerUID = Users.ID " . + "LEFT JOIN PackageRelations " . + "ON PackageRelations.PackageID = Packages.ID " . + "AND PackageRelations.RelTypeID = 2 "; + + $where = array(); + + foreach($http_data['by'] as $by) { + if($by == 'provides') { + foreach ($http_data['arg'] as $provide) { + array_push($where, + "PackageRelations.RelName = " . + $this->dbh->quote($provide)); + } + } elseif($by == 'name') { + foreach ($http_data['arg'] as $name) { + array_push($where, "Packages.NAME LIKE " . + $this->dbh->quote('%' . $name . '%_')); + array_push($where, "Packages.Name = " . + $this->dbh->quote($name)); + } + } elseif($by == 'name-desc') { + foreach ($http_data['arg'] as $name) { + array_push($where, "Packages.NAME LIKE " . + $this->dbh->quote('%' . $name . '%_')); + array_push($where, "Packages.Name = " . + $this->dbh->quote($name)); + array_push($where, "Packages.Description LIKE " . + $this->dbh->quote('%' . $name . '%_')); + } + } elseif (in_array($by, self::$exposed_depfields)) { + foreach ($http_data['arg'] as $dep) { + $subquery = "SELECT PackageDepends.DepName FROM PackageDepends "; + $subquery .= "LEFT JOIN DependencyTypes "; + $subquery .= "ON PackageDepends.DepTypeID = DependencyTypes.ID "; + $subquery .= "WHERE PackageDepends.PackageID = Packages.ID "; + $subquery .= "AND DependencyTypes.Name = " . $this->dbh->quote($by); + $sub_condition = $this->dbh->quote($dep) . " IN (${subquery})"; + array_push($where, $sub_condition); + } + } else { + return $this->json_error( + "${by} is not supported in v6 multi-argument by search." + ); + } + } + + $query .= "WHERE " . implode(" OR ", $where) . " "; + "AND PackageBases.PackagerUID IS NOT NULL " . + "LIMIT ${max_results}"; + + $packages = array(); // Final list of packages + $result = $this->dbh->query($query); + + if ($result) { + while ($row = $result->fetch(PDO::FETCH_ASSOC)) + array_push($packages, $row); + } + + return $this->json_results('search', count($packages), $packages, NULL); + } + /* * Returns the info on a specific package. * @@ -569,7 +665,6 @@ class AurJSON { return $this->process_query('multiinfo', $where_condition); } - /* * Returns all the packages for a specific maintainer. * -- 2.20.1

1 0

PR Regarding v6 multiple by and provides field
by Kevin Morris 12 Feb '19

12 Feb '19

Hello AUR Maintainers, This pull request is in reference to https://patchwork.archlinux.org/patch/578/ and satisfies back-compat requirements and multiple by[] search implementation that the original requester intended, including the 'provides' field search. > The following changes since commit > 042f3f2622e1adbc7546cfa25a5d138486b99341: > Quote MySql 8.0 reserved keywords (2019-01-14 14:45:37 -0500) > are available in the Git repository at: > https://github.com/kevr/aurweb master > for you to fetch changes up to ac80bee46ae3fe0362c134fe598304d58d1db3fb: > Introduce API v6 (2019-02-12 13:39:08 -0800) > ---------------------------------------------------------------- > Kevin Morris (1): > Introduce API v6 > doc/rpc.txt | 15 ++++++++++++++- > web/lib/aurjson.class.php | 99 > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---- > 2 files changed, 109 insertions(+), 5 deletions(-) Please let me know if there are any blockers or questions you may have, I hope it works out well! Regards, Kevin -- Kevin Morris Personal Inquiries: kevr.gtalk(a)gmail.com Technologies: C++, Python, Django, Ruby, Rails, ReactJS, jQuery, Javascript, SQL, Redux

2 1

Search or info by "provides"
by Sergey Petrenko 02 Feb '19

02 Feb '19

Hi, I’m interested in implementing this one way or another. I have found thread from march 2018 concerning this feature, but without any final resolution. What is the status? Will you accept extension of /search over "provides" or I’ll have to wait till RPC v6 is ready?

1 0