- Aur-dev - lists.archlinux.org

[aur-dev] various implementations of a json api query endpoint for the aur
by elij 27 Mar '11

27 Mar '11

Greetings. I have implemented stand-alone versions of the aur rpc interface, in 3 languages. https://github.com/cactus/spew-ruby (ruby + sinatra) https://github.com/cactus/spew-js (nodejs + coffee-script) https://github.com/cactus/spew-python (python + tornado) The ruby version is probably the most complete (and arguably my favorite of the 3 so far), but each is a usable implementation. The nodejs version may node be idiomatic node code, as I am still learning my way around that ecosystem. The python version dosen't have a good implementation of the '/search' livesearch example endpoint, but that is just due to laziness on my part (I could add it easily if anyone else was interested in it besides me). Currently all 3 versions utilize the mysql database format that the AUR itself uses. One nice aspect of having the rpc interface separated from the php aur code itself are: - less api endpoint downtime when performing maintenance on the aur - decoupled from the aur codebase which could provide an arena for faster development of new features for api clients - nicer languages to work with (I am not a fan of php, due to having used it extensively in the past ;). - ability to have a more restful interface (utilize http response codes: 404 for packages not found, 204 for no content, etc) - provides ability to version the api. this would be a way to support legacy clients for a while, after changes to the api. fake example: /api/v3/maint?q=bob -> /api/v4/maintainer?q=bob The api I have currently implemented (v2) supports _search_ and _info_ type queries. Supporting _maintainer_ queries would be very easy to add to any of the implementations. The v2 interface is slightly different than the current rpc interface (more resftul, supports conditional get support for clients that can cache and perform conditional get requests -- to reduce bandwidth, etc). If there is any interest in utilizing any of the versions listed for Arch, I would be glad to continue working on the codebase and clean up any rough edges and add the maintainer feature. Creating an AUR query interface is one of the projects I use when learning a new language, so even if no version ever gets used for anything, I do not consider it a waste of my time -- so don't be concerned that I will feel bad if the answer is 'no'. Thanks.

2 2

[aur-dev] [PATCH] Bump up default per-page value to 50
by Dan McGee 24 Mar '11

24 Mar '11

25 is woefully small for the number of packages many searches can return, and with 28000+ packages in AUR, it makes sense to show a lot more per page by default. The new choices of (50, 100, 250) happen to match those from the main site. Signed-off-by: Dan McGee <dan(a)archlinux.org> --- Whoops- sent an old version of the patch without the bump from 100 to 250 on the upper limit. web/lib/pkgfuncs.inc | 12 ++++++------ web/template/pkg_search_form.php | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/web/lib/pkgfuncs.inc b/web/lib/pkgfuncs.inc index 03deea2..b38c7da 100644 --- a/web/lib/pkgfuncs.inc +++ b/web/lib/pkgfuncs.inc @@ -419,13 +419,13 @@ function pkg_search_page($SID="") { if (isset($_GET["PP"])) { $_GET["PP"] = intval($_GET["PP"]); - if ($_GET["PP"] < 25) - $_GET["PP"] = 25; - else if ($_GET["PP"] > 100) - $_GET["PP"] = 100; + if ($_GET["PP"] < 50) + $_GET["PP"] = 50; + else if ($_GET["PP"] > 250) + $_GET["PP"] = 250; } else { - $_GET["PP"] = 25; + $_GET["PP"] = 50; } // FIXME: pull out DB-related code. all of it. @@ -560,7 +560,7 @@ function pkg_search_page($SID="") { if ($total > 1 || $total == 0) { # calculation of pagination links - $per_page = ($_GET['PP'] > 0) ? $_GET['PP'] : 25; + $per_page = ($_GET['PP'] > 0) ? $_GET['PP'] : 50; $current = ceil($first / $per_page); $pages = ceil($total / $per_page); $templ_pages = array(); diff --git a/web/template/pkg_search_form.php b/web/template/pkg_search_form.php index 896d47f..e570dfb 100644 --- a/web/template/pkg_search_form.php +++ b/web/template/pkg_search_form.php @@ -89,7 +89,7 @@ <label><?php print __("Per page"); ?></label> <select name='PP'> <?php - $pages = array(25, 50, 75, 100); + $pages = array(50, 100, 250); foreach ($pages as $i): if (isset($_REQUEST['PP']) && $_REQUEST['PP'] == $i): ?> -- 1.7.4.1

2 1

[aur-dev] [PATCH] Bump up default per-page value to 50
by Dan McGee 23 Mar '11

23 Mar '11

25 is woefully small for the number of packages many searches can return, and with 28000+ packages in AUR, it makes sense to show a lot more per page by default. The new choices of (50, 100, 250) happen to match those from the main site. Signed-off-by: Dan McGee <dan(a)archlinux.org> --- web/lib/pkgfuncs.inc | 8 ++++---- web/template/pkg_search_form.php | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/web/lib/pkgfuncs.inc b/web/lib/pkgfuncs.inc index 03deea2..65091e3 100644 --- a/web/lib/pkgfuncs.inc +++ b/web/lib/pkgfuncs.inc @@ -419,13 +419,13 @@ function pkg_search_page($SID="") { if (isset($_GET["PP"])) { $_GET["PP"] = intval($_GET["PP"]); - if ($_GET["PP"] < 25) - $_GET["PP"] = 25; + if ($_GET["PP"] < 50) + $_GET["PP"] = 50; else if ($_GET["PP"] > 100) $_GET["PP"] = 100; } else { - $_GET["PP"] = 25; + $_GET["PP"] = 50; } // FIXME: pull out DB-related code. all of it. @@ -560,7 +560,7 @@ function pkg_search_page($SID="") { if ($total > 1 || $total == 0) { # calculation of pagination links - $per_page = ($_GET['PP'] > 0) ? $_GET['PP'] : 25; + $per_page = ($_GET['PP'] > 0) ? $_GET['PP'] : 50; $current = ceil($first / $per_page); $pages = ceil($total / $per_page); $templ_pages = array(); diff --git a/web/template/pkg_search_form.php b/web/template/pkg_search_form.php index 896d47f..e570dfb 100644 --- a/web/template/pkg_search_form.php +++ b/web/template/pkg_search_form.php @@ -89,7 +89,7 @@ <label><?php print __("Per page"); ?></label> <select name='PP'> <?php - $pages = array(25, 50, 75, 100); + $pages = array(50, 100, 250); foreach ($pages as $i): if (isset($_REQUEST['PP']) && $_REQUEST['PP'] == $i): ?> -- 1.7.4.1

1 0

[aur-dev] [PATCH] Fix main site package search URL
by Dan McGee 15 Mar '11

15 Mar '11

I've been wanting to kill this one for a long time. Signed-off-by: Dan McGee <dan(a)archlinux.org> --- web/template/pkg_details.php | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/web/template/pkg_details.php b/web/template/pkg_details.php index eac7b69..1a1e611 100644 --- a/web/template/pkg_details.php +++ b/web/template/pkg_details.php @@ -118,7 +118,7 @@ $out_of_date_time = ($row["OutOfDateTS"] == 0) ? $msg : gmdate("r", intval($row[ # $darr[3] is the DepCondition echo " <a href='packages.php?ID=".$darr[0]."'>".$darr[1].$darr[3]."</a>"; } else { - echo " <a href='http://www.archlinux.org/packages/search/?q=".$darr[1]."'>".$darr[1].$darr[3]."</a>"; + echo " <a href='http://www.archlinux.org/packages/?q=".$darr[1]."'>".$darr[1].$darr[3]."</a>"; } } @@ -135,7 +135,7 @@ $out_of_date_time = ($row["OutOfDateTS"] == 0) ? $msg : gmdate("r", intval($row[ if ($darr[2] == 0) { echo " <a href='packages.php?ID=".$darr[0]."'>".$darr[1]."</a>"; } else { - print "<a href='http://www.archlinux.org/packages/search/?q=".$darr[1]."'>".$darr[1]."</a>"; + print "<a href='http://www.archlinux.org/packages/?q=".$darr[1]."'>".$darr[1]."</a>"; } } -- 1.7.4.1

2 1

[aur-dev] AUR 1.8.1 released
by Lukas Fleischer 12 Mar '11

12 Mar '11

We've just released 1.8.1 [1]! :) This one's primarily a bug fix release. It improves blacklist handling and promotes full InnoDB conversion, including some database schema cleanups. IMPORTANT: It also contains a fix for a potential SQL injection vulnerability - all maintainers of AUR setups are encouraged to upgrade, soon! Thanks to Dan for pointing this out and for providing patches for this a lot more. Other changes include: * Per-user session limits (FS#12898, defaulting to 8 sessions per user). * Searching for non-out-of-date packages (fixes FS#17896). * Packages with subdirectories are rejected (fixes FS#22995). * Automatic adoption when updating an orphan package (fixes FS#22992). * ZIP bomb protection (fixes FS#22991). Check the Git log [2] for a complete list. The official Arch Linux AUR setup [3] will be upgraded soon! [1] http://projects.archlinux.org/aur.git/commit/?id=29b2f3b3 [2] http://projects.archlinux.org/aur.git/log/?id=29b2f3b3 [3] https://aur.archlinux.org/

2 1

[aur-dev] [PATCH] Notify package maintainers about deleted dependencies (FS#10372).
by Lukas Fleischer 11 Mar '11

11 Mar '11

Signed-off-by: Lukas Fleischer <archlinux(a)cryptocrack.de> --- While upgrading our database to InnoDB, I noticed that there are missing dependencies in the package details view for some packages. This happens when some dependency of a package gets deleted as we recently added an "ON DELETE CASCADE" foreign key on package IDs to the "PackageDepends" table (the problem already existed before, tho). While thinking about a proper fix, it reminded me of FS#10372 which is related and would seem to me to be kinda useful. Reviews and comments welcome! web/lib/pkgfuncs.inc | 29 +++++++++++++++++++++++++++++ 1 files changed, 29 insertions(+), 0 deletions(-) diff --git a/web/lib/pkgfuncs.inc b/web/lib/pkgfuncs.inc index d5e0771..bfa3e1f 100644 --- a/web/lib/pkgfuncs.inc +++ b/web/lib/pkgfuncs.inc @@ -709,6 +709,35 @@ function pkg_delete ($atype, $ids) { } $dbh = db_connect(); + + # Notify of deleted dependencies. + $f_name = username_from_sid($_COOKIE['AURSID']); + $f_uid = uid_from_sid($_COOKIE['AURSID']); + $q = "SELECT Users.Email, DepPkgs.ID, DepPkgs.Name, "; + $q .= "GROUP_CONCAT(Packages.Name SEPARATOR ', ') AS DepList "; + $q .= "FROM Packages "; + $q .= "LEFT JOIN PackageDepends ON PackageDepends.DepPkgId = Packages.ID "; + $q .= "LEFT JOIN Packages DepPkgs ON DepPkgs.ID = PackageDepends.PackageID "; + $q .= "LEFT JOIN Users ON Users.ID = DepPkgs.MaintainerUID "; + $q .= "WHERE Packages.ID IN (" . implode(",", $ids) . ") "; + $q .= "GROUP BY DepPkgs.ID;"; + $result = db_query($q, $dbh); + if (mysql_num_rows($result)) { + while ($row = mysql_fetch_assoc($result)) { + # construct email + $body = "The following dependencies of your package " . $row['Name'] . " [1] "; + $body .= "have been removed by " . $f_name . " [1]: " . $row['DepList'] . ".\n\n"; + $body .= "Please check if the \"depends\" array of your PKGBUILD needs to be updated "; + $body .= "(shouldn't be necessary if a package was moved to the official repos) "; + $body .= "and re-upload your package.\n\n"; + $body .= "[1] http://aur.archlinux.org/packages.php?ID=" . $row['ID'] . "\n"; + $body .= "[2] http://aur.archlinux.org/account.php?Action=AccountInfo&ID=" . $f_uid; + $body = wordwrap($body, 70); + $headers = "Reply-to: nobody@archlinux.org\nFrom:aur-notify@archlinux.org\nX-Mailer: PHP\nX-MimeOLE: Produced By AUR\n"; + @mail($row['Email'], "AUR Dependency Removal Notification for " . $row['Name'], $body, $headers); + } + } + $q = "DELETE FROM Packages WHERE ID IN (" . implode(",", $ids) . ")"; $result = db_query($q, $dbh); -- 1.7.4.1

1 0

[aur-dev] A few more patches
by Dan McGee 10 Mar '11

10 Mar '11

Mostly to Lukas, but posting it here- I have a few more patches on my working branch you might be interested in: http://code.toofishes.net/cgit/dan/aur.git/log/?h=working * Perform a second query to find total search count * Split package search query into parts * More PHP Notice undefined fixups -Dan

2 1

[aur-dev] [PATCH] Specify utf8 in database schema.
by Loui Chang 28 Feb '11

28 Feb '11

Signed-off-by: Loui Chang <louipc.ist(a)gmail.com> --- support/schema/aur-schema.sql | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/support/schema/aur-schema.sql b/support/schema/aur-schema.sql index b713fc8..1649055 100644 --- a/support/schema/aur-schema.sql +++ b/support/schema/aur-schema.sql @@ -2,7 +2,7 @@ -- is also included such as AccountTypes, etc. -- DROP DATABASE AUR; -CREATE DATABASE AUR; +CREATE DATABASE AUR DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; USE AUR; -- Define the Account Types for the AUR. @@ -159,7 +159,7 @@ CREATE TABLE PackageComments ( ID BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, PackageID INTEGER UNSIGNED NOT NULL, UsersID INTEGER UNSIGNED NOT NULL, - Comments TEXT NOT NULl DEFAULT '', + Comments TEXT NOT NULL DEFAULT '', CommentTS BIGINT UNSIGNED NOT NULL DEFAULT 0, DelUsersID INTEGER UNSIGNED NULL DEFAULT NULL, PRIMARY KEY (ID), @@ -193,8 +193,8 @@ CREATE TABLE PackageBlacklist ( -- CREATE TABLE IF NOT EXISTS TU_VoteInfo ( ID int(10) unsigned NOT NULL auto_increment, - Agenda text collate latin1_general_ci NOT NULL, - User VARCHAR(32) collate latin1_general_ci NOT NULL, + Agenda text COLLATE utf8_general_ci NOT NULL, + User VARCHAR(32) COLLATE utf8_general_ci NOT NULL, Submitted bigint(20) unsigned NOT NULL, End bigint(20) unsigned NOT NULL, SubmitterID int(10) unsigned NOT NULL, -- 1.7.4.1

2 1

[aur-dev] Russian comments are '???'
by Joker-jar 26 Feb '11

26 Feb '11

Hi. Russian comments are like '???' now in the aur. Look: http://aur.archlinux.org/packages.php?ID=40365 I think, problem in database encoding

6 5

[aur-dev] AUR 1.8.0 released
by Lukas Fleischer 22 Feb '11

22 Feb '11

1.8.0 is finally out [1]! Please note that this release removes all the code for automatic tarball extraction. From now on, only PKGBUILDs and source tarballs will be accessible via the AUR web interface. 1.8.0 also introduces a package blacklist and a blacklist helper which can be used to avoid submission of packages from the official repos to the AUR. Other changes include: * Original submitters in package details. * Variable parser improvements. * "python2" shebang for all Python scripts. * "First" and "Last" links in search results. * Timestamps for out-of-date packages. * Clickable links in comments. * Auto redirection from confirmation screens. * RTL support. * Updated translations. * InnoDB compatibility for all tables. * Database cleanups. * Fixes for some XSS vulnerabilities. * Fixes for all bugs reported on the bug tracker [2] :) Check the Git log [3] for a complete list. The official Arch Linux AUR setup [4] will be upgraded soon! [1] http://projects.archlinux.org/aur.git/commit/?id=a7742012 [2] https://bugs.archlinux.org/index.php?project=2 [3] http://projects.archlinux.org/aur.git/log/?id=a7742012 [4] https://aur.archlinux.org/

4 3