June 2014 - Aur-dev - lists.archlinux.org

[aur-dev] [HEADS-UP] Meta data and split package support in the AUR
by Lukas Fleischer 27 Jul '14

27 Jul '14

Hello, I plan to do the next major AUR release by late May or June and I thought this might be a good time to let people know what is going on behind the scenes. AUR 3.0.0 will be able to read metadata from source packages. These metadata will be read from a file called .AURINFO (contained in the source tarball). Dave wrote several tools [1], including mkaurball which can be used to automatically build a source tarball with metadata so you don't need to worry about creating this file on your own. Having this information available means that: 1. We can drop the PKGBUILD parser from the AUR. The parser will still be available in the upcoming release but it will be marked as deprecated and a warning will be displayed whenever someone tries to upload a source tarball without metadata. 2. We can implement other things that were blocked by the AUR PKGBUILD parser being incomplete and inaccurate. Specifically, the next release will support split packages. Things that are on our TODO list: * Test the new AUR code. The split package code is still experimental. * Fix and extend the AUR RPC interface. * Test Dave's pkgbuild-introspection. * Move pkgbuild-introspection (or at least mkaurball) to [community]. It would also be nice to get metadata generation integrated into makepkg so that people no longer need to install mkaurball to generate source tarballs for the AUR. Regards, Lukas [1] https://github.com/falconindy/pkgbuild-introspection

1 1

[aur-dev] [PATCH 1/6] Add support for filing package requests
by Lukas Fleischer 01 Jul '14

01 Jul '14

Add a new entry to the package actions box that allows for filing deletion and orphan requests. When choosing that action, the user is redirected to a new page that allows for selecting a request type and entering a comment. When submitting the request, a new entry in the request database is created and an email is sent to a configurable mailing list (defaults to aur-general). Signed-off-by: Lukas Fleischer <archlinux(a)cryptocrack.de> --- UPGRADING | 31 ++++++++++++++ schema/aur-schema.sql | 28 +++++++++++++ web/html/index.php | 3 ++ web/html/pkgbase.php | 2 + web/html/pkgreq.php | 55 +++++++++++++++++++++++++ web/lib/config.inc.php.proto | 3 ++ web/lib/pkgbasefuncs.inc.php | 88 ++++++++++++++++++++++++++++++++++++++++ web/template/pkg_details.php | 1 + web/template/pkgbase_details.php | 1 + 9 files changed, 212 insertions(+) create mode 100644 web/html/pkgreq.php diff --git a/UPGRADING b/UPGRADING index 863fde3..455118e 100644 --- a/UPGRADING +++ b/UPGRADING @@ -1,6 +1,37 @@ Upgrading ========= +From 3.1.0 to 3.2.0 +------------------- + +1. Add support for package requests to the database: + +---- +CREATE TABLE RequestTypes ( + ID TINYINT UNSIGNED NOT NULL AUTO_INCREMENT, + Name VARCHAR(32) NOT NULL DEFAULT '', + PRIMARY KEY (ID) +) ENGINE = InnoDB; +INSERT INTO RequestTypes VALUES (1, 'deletion'); +INSERT INTO RequestTypes VALUES (2, 'orphan'); + +CREATE TABLE PackageRequests ( + ID BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, + ReqTypeID TINYINT UNSIGNED NOT NULL, + PackageBaseID INTEGER UNSIGNED NULL, + PackageBaseName VARCHAR(255) NOT NULL, + UsersID INTEGER UNSIGNED NULL DEFAULT NULL, + Comments TEXT NOT NULL DEFAULT '', + RequestTS BIGINT UNSIGNED NOT NULL DEFAULT 0, + PRIMARY KEY (ID), + INDEX (UsersID), + INDEX (PackageBaseID), + FOREIGN KEY (ReqTypeID) REFERENCES RequestTypes(ID) ON DELETE NO ACTION, + FOREIGN KEY (UsersID) REFERENCES Users(ID) ON DELETE SET NULL, + FOREIGN KEY (PackageBaseID) REFERENCES PackageBases(ID) ON DELETE SET NULL +) ENGINE = InnoDB; +---- + From 3.0.0 to 3.1.0 ------------------- diff --git a/schema/aur-schema.sql b/schema/aur-schema.sql index 0efae93..88f9974 100644 --- a/schema/aur-schema.sql +++ b/schema/aur-schema.sql @@ -288,6 +288,34 @@ CREATE TABLE PackageBlacklist ( UNIQUE (Name) ) ENGINE = InnoDB; +-- Define package request types +-- +CREATE TABLE RequestTypes ( + ID TINYINT UNSIGNED NOT NULL AUTO_INCREMENT, + Name VARCHAR(32) NOT NULL DEFAULT '', + PRIMARY KEY (ID) +) ENGINE = InnoDB; +INSERT INTO RequestTypes VALUES (1, 'deletion'); +INSERT INTO RequestTypes VALUES (2, 'orphan'); + +-- Package requests +-- +CREATE TABLE PackageRequests ( + ID BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, + ReqTypeID TINYINT UNSIGNED NOT NULL, + PackageBaseID INTEGER UNSIGNED NULL, + PackageBaseName VARCHAR(255) NOT NULL, + UsersID INTEGER UNSIGNED NULL DEFAULT NULL, + Comments TEXT NOT NULL DEFAULT '', + RequestTS BIGINT UNSIGNED NOT NULL DEFAULT 0, + PRIMARY KEY (ID), + INDEX (UsersID), + INDEX (PackageBaseID), + FOREIGN KEY (ReqTypeID) REFERENCES RequestTypes(ID) ON DELETE NO ACTION, + FOREIGN KEY (UsersID) REFERENCES Users(ID) ON DELETE SET NULL, + FOREIGN KEY (PackageBaseID) REFERENCES PackageBases(ID) ON DELETE SET NULL +) ENGINE = InnoDB; + -- Vote information -- CREATE TABLE IF NOT EXISTS TU_VoteInfo ( diff --git a/web/html/index.php b/web/html/index.php index 921839a..40063f0 100644 --- a/web/html/index.php +++ b/web/html/index.php @@ -75,6 +75,9 @@ if (!empty($tokens[1]) && '/' . $tokens[1] == get_pkg_route()) { $_GET['N'] = $tokens[2]; include('voters.php'); return; + case "request": + include('pkgreq.php'); + return; default: header("HTTP/1.0 404 Not Found"); include "./404.php"; diff --git a/web/html/pkgbase.php b/web/html/pkgbase.php index 0d1b74a..dd09977 100644 --- a/web/html/pkgbase.php +++ b/web/html/pkgbase.php @@ -94,6 +94,8 @@ if (check_token()) { list($ret, $output) = pkgbase_delete_comment($atype); } elseif (current_action("do_ChangeCategory")) { list($ret, $output) = pkgbase_change_category($base_id, $atype); + } elseif (current_action("do_FileRequest")) { + list($ret, $output) = pkgbase_file_request($ids, $_POST['type'], $_POST['comments']); } if (isset($_REQUEST['comment'])) { diff --git a/web/html/pkgreq.php b/web/html/pkgreq.php new file mode 100644 index 0000000..c8dd673 --- /dev/null +++ b/web/html/pkgreq.php @@ -0,0 +1,55 @@ +<?php + +set_include_path(get_include_path() . PATH_SEPARATOR . '../lib'); + +include_once("aur.inc.php"); +include_once("pkgfuncs.inc.php"); + +set_lang(); +check_sid(); + +html_header(__("File Request")); + +if (!check_user_privileges()) { + header('Location: /'); + exit(); +} +?> + +<div class="box"> + <h2><?= __('File Request: %s', htmlspecialchars($pkgbase_name)) ?></h2> + <p> + <?= __('Use this form to file a request against package base %s%s%s which includes the following packages:', + '<strong>', htmlspecialchars($pkgbase_name), '</strong>'); ?> + </p> + <ul> + <?php foreach(pkgbase_get_pkgnames($base_id) as $pkgname): ?> + <li><?= htmlspecialchars($pkgname) ?></li> + <?php endforeach; ?> + </ul> + <form action="<?= get_uri('/pkgbase/'); ?>" method="post"> + <fieldset> + <input type="hidden" name="IDs[<?= $base_id ?>]" value="1" /> + <input type="hidden" name="ID" value="<?= $base_id ?>" /> + <input type="hidden" name="token" value="<?= htmlspecialchars($_COOKIE['AURSID']) ?>" /> + <p> + <label for="id_type"><?= __("Request type") ?>:</label> + <select name="type" id="id_type"> + <option value="deletion"><?= __('Deletion') ?></option> + <option value="orphan"><?= __('Orphan') ?></option> + </select> + </p> + <p> + <label for="id_comments"><?= __("Comments") ?>:</label> + <textarea name="comments" id="id_comments" rows="5" cols="50"></textarea> + </p> + <p> + <input type="submit" class="button" name="do_FileRequest" value="<?= __("File Request") ?>" /> + </p> + </fieldset> + </form> +</div> + +<?php +html_footer(AUR_VERSION); + diff --git a/web/lib/config.inc.php.proto b/web/lib/config.inc.php.proto index 1fe7dbc..10ed07e 100644 --- a/web/lib/config.inc.php.proto +++ b/web/lib/config.inc.php.proto @@ -59,3 +59,6 @@ $USE_VIRTUAL_URLS = true; # Maximum number of package results to return through an RPC connection. # Avoid setting this too high and having a PHP too much memory error. $MAX_RPC_RESULTS = 5000; + +# Mailing list to send package request notifications to. +$AUR_REQUEST_ML = "aur-general(a)archlinux.org"; diff --git a/web/lib/pkgbasefuncs.inc.php b/web/lib/pkgbasefuncs.inc.php index 9f80ef2..a9fe949 100644 --- a/web/lib/pkgbasefuncs.inc.php +++ b/web/lib/pkgbasefuncs.inc.php @@ -962,3 +962,91 @@ function pkgbase_update_category($base_id, $category_id) { $category_id, $base_id); $dbh->exec($q); } + +/** + * File a deletion/orphan request against a package base + * + * @global string $AUR_LOCATION The AUR's URL used for notification e-mails + * @global string $AUR_REQUEST_ML The request notification mailing list + * @param string $ids The package base IDs to file the request against + * @param string $type The type of the request + * @param string $comments The comments to be added to the request + * + * @return void + */ +function pkgbase_file_request($ids, $type, $comments) { + global $AUR_LOCATION; + global $AUR_REQUEST_ML; + + if (empty($comments)) { + return array(false, __("The comment field must not be empty.")); + } + + $dbh = DB::connect(); + $uid = uid_from_sid($_COOKIE["AURSID"]); + + /* TODO: Allow for filing multiple requests at once. */ + $base_id = $ids[0]; + $pkgbase_name = pkgbase_name_from_id($base_id); + + $q = "SELECT ID FROM RequestTypes WHERE Name = " . $dbh->quote($type); + $result = $dbh->query($q); + if ($row = $result->fetch(PDO::FETCH_ASSOC)) { + $type_id = $row['ID']; + } else { + return array(false, __("Invalid request type.")); + } + + $q = "INSERT INTO PackageRequests "; + $q.= "(ReqTypeID, PackageBaseID, PackageBaseName, UsersID, "; + $q.= "Comments, RequestTS) VALUES (" . $type_id . ", "; + $q.= intval($base_id) . ", " . $dbh->quote($pkgbase_name) . ", "; + $q.= $uid . ", " . $dbh->quote($comments) . ", UNIX_TIMESTAMP())"; + $dbh->exec($q); + + /* + * Send e-mail notifications. + * TODO: Move notification logic to separate function where it belongs. + */ + $q = "SELECT Users.Email "; + $q.= "FROM Users INNER JOIN PackageBases "; + $q.= "ON PackageBases.MaintainerUID = Users.ID "; + $q.= "WHERE PackageBases.ID = " . intval($base_id); + $result = $dbh->query($q); + if ($row = $result->fetch(PDO::FETCH_ASSOC)) { + $bcc = $row['Email']; + } else { + unset($bcc); + } + + $q = "SELECT Name FROM PackageBases WHERE ID = "; + $q.= intval($base_id); + $result = $dbh->query($q); + $row = $result->fetch(PDO::FETCH_ASSOC); + + /* + * TODO: Add native language emails for users, based on their + * preferences. Simply making these strings translatable won't + * work, users would be getting emails in the language that the + * user who posted the comment was in. + */ + $username = username_from_sid($_COOKIE['AURSID']); + $body = + $username . " [1] filed a " . $type . " request for " . + $row['Name'] . " [2]:\n\n" . $comments . "\n\n" . + "[1] " . $AUR_LOCATION . get_user_uri($username) . "\n" . + "[2] " . $AUR_LOCATION . get_pkgbase_uri($row['Name']) . "\n"; + $body = wordwrap($body, 70); + $headers = "MIME-Version: 1.0\r\n" . + "Content-type: text/plain; charset=UTF-8\r\n"; + if (!empty($bcc)) { + $headers .= "Bcc: $bcc\r\n"; + } + $headers .= "Reply-to: noreply(a)aur.archlinux.org\r\n" . + "From: notify(a)aur.archlinux.org\r\n" . + "X-Mailer: AUR"; + @mail($AUR_REQUEST_ML, "AUR " . ucfirst($type) . " Request for " . + $row['Name'], $body, $headers); + + return array(true, __("Added request successfully.")); +} diff --git a/web/template/pkg_details.php b/web/template/pkg_details.php index 6326d4e..065057a 100644 --- a/web/template/pkg_details.php +++ b/web/template/pkg_details.php @@ -106,6 +106,7 @@ $sources = pkg_sources($row["ID"]); </form> </li> <?php endif; ?> + <li><a href="<?= get_pkgbase_uri($row['BaseName']) . 'request/'; ?>"><?= __('File Request'); ?></a></li> <?php if ($atype == "Trusted User" || $atype == "Developer"): ?> <li><a href="<?= get_pkgbase_uri($row['BaseName']) . 'delete/'; ?>"><?= __('Delete Package'); ?></a></li> <li><a href="<?= get_pkgbase_uri($row['BaseName']) . 'merge/'; ?>"><?= __('Merge Package'); ?></a></li> diff --git a/web/template/pkgbase_details.php b/web/template/pkgbase_details.php index 6c617bf..1b40b84 100644 --- a/web/template/pkgbase_details.php +++ b/web/template/pkgbase_details.php @@ -81,6 +81,7 @@ $pkgs = pkgbase_get_pkgnames($base_id); </form> </li> <?php endif; ?> + <li><a href="<?= get_pkgbase_uri($row['BaseName']) . 'request/'; ?>"><?= __('File Request'); ?></a></li> <?php if ($atype == "Trusted User" || $atype == "Developer"): ?> <li><a href="<?= get_pkgbase_uri($row['Name']) . 'delete/'; ?>"><?= __('Delete Package'); ?></a></li> <li><a href="<?= get_pkgbase_uri($row['Name']) . 'merge/'; ?>"><?= __('Merge Package'); ?></a></li> -- 2.0.0

2 11

[aur-dev] [PATCH] aur-schema.sql: Use tabs for indentation everywhere
by Lukas Fleischer 30 Jun '14

30 Jun '14

Signed-off-by: Lukas Fleischer <archlinux(a)cryptocrack.de> --- schema/aur-schema.sql | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/schema/aur-schema.sql b/schema/aur-schema.sql index 1ec7385..8d89487 100644 --- a/schema/aur-schema.sql +++ b/schema/aur-schema.sql @@ -322,28 +322,28 @@ CREATE TABLE PackageRequests ( -- Vote information -- CREATE TABLE IF NOT EXISTS TU_VoteInfo ( - ID int(10) unsigned NOT NULL auto_increment, - Agenda text NOT NULL, - User VARCHAR(32) NOT NULL, - Submitted bigint(20) unsigned NOT NULL, - End bigint(20) unsigned NOT NULL, - Quorum decimal(2, 2) unsigned NOT NULL, - SubmitterID int(10) unsigned NOT NULL, - Yes tinyint(3) unsigned NOT NULL default '0', - No tinyint(3) unsigned NOT NULL default '0', - Abstain tinyint(3) unsigned NOT NULL default '0', - ActiveTUs tinyint(3) unsigned NOT NULL default '0', - PRIMARY KEY (ID), - FOREIGN KEY (SubmitterID) REFERENCES Users(ID) ON DELETE CASCADE + ID int(10) unsigned NOT NULL auto_increment, + Agenda text NOT NULL, + User VARCHAR(32) NOT NULL, + Submitted bigint(20) unsigned NOT NULL, + End bigint(20) unsigned NOT NULL, + Quorum decimal(2, 2) unsigned NOT NULL, + SubmitterID int(10) unsigned NOT NULL, + Yes tinyint(3) unsigned NOT NULL default '0', + No tinyint(3) unsigned NOT NULL default '0', + Abstain tinyint(3) unsigned NOT NULL default '0', + ActiveTUs tinyint(3) unsigned NOT NULL default '0', + PRIMARY KEY (ID), + FOREIGN KEY (SubmitterID) REFERENCES Users(ID) ON DELETE CASCADE ) ENGINE = InnoDB; -- Individual vote records -- CREATE TABLE IF NOT EXISTS TU_Votes ( - VoteID int(10) unsigned NOT NULL, - UserID int(10) unsigned NOT NULL, - FOREIGN KEY (VoteID) REFERENCES TU_VoteInfo(ID) ON DELETE CASCADE, - FOREIGN KEY (UserID) REFERENCES Users(ID) ON DELETE CASCADE + VoteID int(10) unsigned NOT NULL, + UserID int(10) unsigned NOT NULL, + FOREIGN KEY (VoteID) REFERENCES TU_VoteInfo(ID) ON DELETE CASCADE, + FOREIGN KEY (UserID) REFERENCES Users(ID) ON DELETE CASCADE ) ENGINE = InnoDB; -- Malicious user banning -- 2.0.1

1 0

[aur-dev] [PATCH] Do not highlight closed requests
by Lukas Fleischer 29 Jun '14

29 Jun '14

Since d8dbad0 (Highlight package requests after 14 days, 2014-06-25), requests, which have been created more than two weeks ago, are highlighted. This is useful because Trusted Users are generally supposed to wait for 14 days before accepting a request. However, it does not help to highlight requests that have already been closed. Change the logic so that only *open* requests which are at least 14 days old are marked. Signed-off-by: Lukas Fleischer <archlinux(a)cryptocrack.de> --- web/template/pkgreq_results.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/template/pkgreq_results.php b/web/template/pkgreq_results.php index 042cc3a..7933964 100644 --- a/web/template/pkgreq_results.php +++ b/web/template/pkgreq_results.php @@ -45,7 +45,7 @@ <td> <a href="<?= get_uri('/account/') . htmlspecialchars($row['User'], ENT_QUOTES) ?>" title="<?= __('View account information for %s', htmlspecialchars($row['User'])) ?>"><?= htmlspecialchars($row['User']) ?></a> </td> - <td<?php if (time() - intval($row['RequestTS']) > $REQUEST_IDLE_TIME): ?> class="flagged"<?php endif; ?>><?= gmdate("Y-m-d H:i", intval($row['RequestTS'])) ?></td> + <td<?php if ($row['Status'] == 0 && time() - intval($row['RequestTS']) > $REQUEST_IDLE_TIME): ?> class="flagged"<?php endif; ?>><?= gmdate("Y-m-d H:i", intval($row['RequestTS'])) ?></td> <?php if ($row['Status'] == 0): ?> <td> <?php if ($row['BaseID']): ?> -- 2.0.1

1 0

[aur-dev] [PATCH 1/6] Allow regular users to file package requests
by Lukas Fleischer 25 Jun '14

25 Jun '14

Move the permission check so that regular users can file requests, whereas the request list is only available to Trusted Users and developers. Signed-off-by: Lukas Fleischer <archlinux(a)cryptocrack.de> --- web/html/pkgreq.php | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/web/html/pkgreq.php b/web/html/pkgreq.php index 2b46b02..d7a4354 100644 --- a/web/html/pkgreq.php +++ b/web/html/pkgreq.php @@ -10,12 +10,12 @@ check_sid(); html_header(__("File Request")); -if (!check_user_privileges()) { - header('Location: /'); - exit(); -} - if (!isset($base_id)) { + if (!check_user_privileges()) { + header('Location: /'); + exit(); + } + $results = pkgbase_request_list(); $total = count($results); -- 2.0.0

1 5

[aur-dev] [PATCH] Make results consistent in RPC replies
by Lukas Fleischer 25 Jun '14

25 Jun '14

Add RPC v3 which always returns a list of objects in RPC responses, regardless of the request type. An empty list is returned when an error occurs. The error message is moved to a separate "error" field. Fixes FS#40963. Signed-off-by: Lukas Fleischer <archlinux(a)cryptocrack.de> --- web/lib/aurjson.class.php | 41 ++++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 13 deletions(-) diff --git a/web/lib/aurjson.class.php b/web/lib/aurjson.class.php index 9abb2d6..8187bef 100644 --- a/web/lib/aurjson.class.php +++ b/web/lib/aurjson.class.php @@ -61,7 +61,7 @@ class AurJSON { if (isset($http_data['v'])) { $this->version = intval($http_data['v']); } - if ($this->version < 1 || $this->version > 2) { + if ($this->version < 1 || $this->version > 3) { return $this->json_error('Invalid version specified.'); } @@ -109,7 +109,11 @@ class AurJSON { */ private function json_error($msg) { header('content-type: application/json'); - return $this->json_results('error', 0, $msg); + if ($this->version < 3) { + return $this->json_results('error', 0, $msg, NULL); + } elseif ($this->version >= 3) { + return $this->json_results('error', 0, array(), $msg); + } } /* @@ -117,16 +121,23 @@ class AurJSON { * * @param $type The response method type. * @param $data The result data to return + * @param $error An error message to include in the response * * @return mixed A json formatted result response. */ - private function json_results($type, $count, $data) { - return json_encode(array( + private function json_results($type, $count, $data, $error) { + $json_array = array( 'version' => $this->version, 'type' => $type, 'resultcount' => $count, 'results' => $data - )); + ); + + if ($error) { + $json_array['error'] = $error; + } + + return json_encode($json_array); } private function get_extended_fields($pkgid) { @@ -195,7 +206,7 @@ class AurJSON { "WHERE ${where_condition} " . "GROUP BY Packages.ID " . "LIMIT $MAX_RPC_RESULTS"; - } elseif ($this->version == 2) { + } elseif ($this->version >= 2) { $fields = implode(',', self::$fields_v2); $query = "SELECT {$fields} " . "FROM Packages LEFT JOIN PackageBases " . @@ -225,14 +236,18 @@ class AurJSON { $row[$field] = intval($row[$field]); } - if ($this->version == 2 && ($type == 'info' || $type == 'multiinfo')) { + if ($this->version >= 2 && ($type == 'info' || $type == 'multiinfo')) { $row = array_merge($row, $this->get_extended_fields($row['ID'])); } - if ($type == 'info') { - $search_data = $row; - break; - } else { + if ($this->version < 3) { + if ($type == 'info') { + $search_data = $row; + break; + } else { + array_push($search_data, $row); + } + } elseif ($this->version >= 3) { array_push($search_data, $row); } } @@ -241,9 +256,9 @@ class AurJSON { return $this->json_error('Too many package results.'); } - return $this->json_results($type, $resultcount, $search_data); + return $this->json_results($type, $resultcount, $search_data, NULL); } else { - return $this->json_results($type, 0, array()); + return $this->json_results($type, 0, array(), NULL); } } -- 2.0.0

1 0

[aur-dev] Git repositories for AUR packages
by Lukas Fleischer 24 Jun '14

24 Jun '14

Hi list, The next major AUR release will come with Git integration. In this email, I am going to summarize previous ideas and make things a bit more concrete (sketch implementation details etc.) If you are new to this topic, please read [1] and [2] first. Our plan is to create one Git repository per package base. Each of the Git repositories contains a PKGBUILD and zero or more additionally required files, so you will no longer need to build a source tarball. There will be a pkgbuild-introspection utility to generate and write .AURINFO metadata without automatically putting that file inside the tarball. Florian had the idea to use a shared Git object database for space efficiency. When creating a new account, users will be able to upload an SSH public key. This key will be stored in the AUR database and can also be changed using the profile edit page. Using this public key, users will be able to push to repositories they maintain using Git over SSH. For authentication, a custom script doing a lookup in the AUR user database will be used as AuthorizedKeysCommand. If a matching key is found, that script prints the corresponding authorized_keys line with a forced command that invokes a wrapper around the Git shell. That wrapper in turn does authorization by checking maintainership and then calls git-shell(1). Using Git hooks, the .AURINFO metadata of a package is parsed when pushing and the AUR package database is updated accordingly. Hooks will also take care of checking the tree objects for huge files etc. The receive.denyNonFastForwards configuration option will be enabled to prevent users from rewriting the history. In order to submit new packages, you will be able to generate empty Git repositories via the AUR web interface. During the transition period, all existing source tarballs will be converted to bare repositories with one initial commit whose tree equals the tarball contents. Instead of the source tarball download link and the PKGBUILD preview, there will be a public clone URL on the package details page. In a second step, cgit will be configured to provide a web-based interface to all the repositories. Then, links to tarball snapshots and to the PKGBUILD preview will be added to the details page as a replacement for the "Download tarball" and "View PKGBUILD" links (all this is supposed to happen before the first release). As soon as all this is set up, we can add support for multiple maintainers and related features. If there are any questions or suggestions regarding this setup, please feel free to ask/reply. Regards, Lukas [1] https://mailman.archlinux.org/pipermail/aur-dev/2013-March/002411.html [2] https://mailman.archlinux.org/pipermail/aur-dev/2014-January/002592.html

7 10

[aur-dev] [PATCH] Fix latest_pkgs() again
by Lukas Fleischer 22 Jun '14

22 Jun '14

In 4cd6841 (pkgfuncs.inc.php: Fix latest_pkgs(), 2014-05-29), latest_pkgs() was modified to work with the new database layout. However, since all columns were selected, package names have been overwritten by package base names since that change. Qualify the *-shorthand to avoid this. Signed-off-by: Lukas Fleischer <archlinux(a)cryptocrack.de> --- web/lib/pkgfuncs.inc.php | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/web/lib/pkgfuncs.inc.php b/web/lib/pkgfuncs.inc.php index c736051..f515864 100644 --- a/web/lib/pkgfuncs.inc.php +++ b/web/lib/pkgfuncs.inc.php @@ -954,7 +954,8 @@ function pkg_add_lic($pkgid, $licid) { function latest_pkgs($numpkgs) { $dbh = DB::connect(); - $q = "SELECT * FROM Packages LEFT JOIN PackageBases ON "; + $q = "SELECT Packages.*, MaintainerUID, SubmittedTS "; + $q.= "FROM Packages LEFT JOIN PackageBases ON "; $q.= "PackageBases.ID = Packages.PackageBaseID "; $q.= "ORDER BY SubmittedTS DESC "; $q.= "LIMIT " . intval($numpkgs); -- 2.0.0

1 0

[aur-dev] [PATCH/RFC 0/5] Use Git repositories to store AUR packages
by Lukas Fleischer 18 Jun '14

18 Jun '14

This patch series adds Git support to the AUR. The scripts are not thoroughly tested and there is no documentation as of now. First comments and reviews are welcome, though! If you want to test this, note that the scripts currently require a patched sshd(8) [1]. I am trying to get this upstream. [1] https://github.com/ScottDuckworth/openssh-akcenv Lukas Fleischer (5): Add support for adding SSH public keys to profiles Add basic Git authentication/authorization scripts Add update hook template Use Git repositories to store packages Add public clone URLs to package details UPGRADING | 10 +- schema/aur-schema.sql | 1 + scripts/git-integration/config.sample | 17 + scripts/git-integration/git-auth.py | 41 + scripts/git-integration/git-serve.py | 106 ++ scripts/git-integration/git-update.py | 334 ++++++ web/html/account.php | 7 +- web/html/pkgsubmit.php | 495 -------- web/lib/Archive/PEAR.php | 1063 ------------------ web/lib/Archive/PEAR5.php | 33 - web/lib/Archive/Tar.php | 1993 --------------------------------- web/lib/acctfuncs.inc.php | 78 +- web/lib/aurjson.class.php | 1 - web/lib/config.inc.php.proto | 4 +- web/lib/pkgbasefuncs.inc.php | 39 - web/lib/pkgbuild-parser.inc.php | 139 --- web/lib/pkgfuncs.inc.php | 195 ---- web/lib/routing.inc.php | 1 - web/template/account_edit_form.php | 5 + web/template/header.php | 1 - web/template/pkg_details.php | 10 +- web/template/pkgbase_details.php | 10 +- 22 files changed, 602 insertions(+), 3981 deletions(-) create mode 100644 scripts/git-integration/config.sample create mode 100755 scripts/git-integration/git-auth.py create mode 100755 scripts/git-integration/git-serve.py create mode 100755 scripts/git-integration/git-update.py delete mode 100644 web/html/pkgsubmit.php delete mode 100644 web/lib/Archive/PEAR.php delete mode 100644 web/lib/Archive/PEAR5.php delete mode 100644 web/lib/Archive/Tar.php delete mode 100644 web/lib/pkgbuild-parser.inc.php -- 2.0.0

4 9

Re: [aur-dev] Git repositories for AUR packages
by Kuba Serafinowski 05 Jun '14

05 Jun '14

> Hi list, > > The next major AUR release will come with Git integration. In this > email, I am going to summarize previous ideas and make things a bit more > concrete (sketch implementation details etc.) > > If you are new to this topic, please read [1] and [2] first. > > Our plan is to create one Git repository per package base. Each of the > Git repositories contains a PKGBUILD and zero or more additionally > required files, so you will no longer need to build a source tarball. > There will be a pkgbuild-introspection utility to generate and write > .AURINFO metadata without automatically putting that file inside the > tarball. Florian had the idea to use a shared Git object database for > space efficiency. > > When creating a new account, users will be able to upload an SSH public > key. This key will be stored in the AUR database and can also be changed > using the profile edit page. Using this public key, users will be able > to push to repositories they maintain using Git over SSH. For > authentication, a custom script doing a lookup in the AUR user database > will be used as AuthorizedKeysCommand. If a matching key is found, that > script prints the corresponding authorized_keys line with a forced > command that invokes a wrapper around the Git shell. That wrapper in > turn does authorization by checking maintainership and then calls > git-shell(1). > > Using Git hooks, the .AURINFO metadata of a package is parsed when > pushing and the AUR package database is updated accordingly. Hooks will > also take care of checking the tree objects for huge files etc. The > receive.denyNonFastForwards configuration option will be enabled to > prevent users from rewriting the history. > > In order to submit new packages, you will be able to generate empty Git > repositories via the AUR web interface. During the transition period, > all existing source tarballs will be converted to bare repositories with > one initial commit whose tree equals the tarball contents. As has been suggested in either [1] or [2] thread, can we get initial importing of git history? I personally haven't been using the same scheme for version control, but from what I've read it's quite simple to separate a directory into its own repository, maintaining the commit history. It would be really nice to have, to be able to fully migrate off of personal solutions without losing anything. > > Instead of the source tarball download link and the PKGBUILD preview, > there will be a public clone URL on the package details page. In a > second step, cgit will be configured to provide a web-based interface to > all the repositories. Then, links to tarball snapshots and to the > PKGBUILD preview will be added to the details page as a replacement for > the "Download tarball" and "View PKGBUILD" links (all this is supposed > to happen before the first release). > > As soon as all this is set up, we can add support for multiple > maintainers and related features. > > If there are any questions or suggestions regarding this setup, please > feel free to ask/reply. > > Regards, > Lukas > > [1] https://mailman.archlinux.org/pipermail/aur-dev/2013-March/002411.html > [2] https://mailman.archlinux.org/pipermail/aur-dev/2014-January/002592.html Cheers Kuba

1 0