October 2016 - Aur-dev - lists.archlinux.org

[aur-dev] aurweb 4.4.1 released
by Lukas Fleischer 18 Oct '16

18 Oct '16

Just for the record: The first maintenance release for aurweb 4.4 was released yesterday and the official setup [1] was upgraded. For a comprehensive list of changes, consult the Git log [2], and please report bugs to the aurweb bug tracker [3]. [1] https://aur.archlinux.org/ [2] https://projects.archlinux.org/aurweb.git/log/?id=v4.4.1 [3] https://bugs.archlinux.org/index.php?project=2

1 0

[aur-dev] [PATCH 1/3] aurweb/git: Add missing __init__.py file
by Lukas Fleischer 17 Oct '16

17 Oct '16

Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- aurweb/git/__init__.py | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 aurweb/git/__init__.py diff --git a/aurweb/git/__init__.py b/aurweb/git/__init__.py new file mode 100644 index 0000000..e69de29 -- 2.10.0

1 2

[aur-dev] [PATCH] Do not show current day if registration date is unknown
by Lukas Fleischer 17 Oct '16

17 Oct '16

The registration date field on the account details page currently defaults to the current day if the user's registration date is unknown. To avoid confusion, show "unknown" in these cases instead. Fixes FS#51405. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- web/template/account_details.php | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/web/template/account_details.php b/web/template/account_details.php index b79685b..024bd9c 100644 --- a/web/template/account_details.php +++ b/web/template/account_details.php @@ -61,7 +61,11 @@ </tr> <tr> <th><?= __("Registration date:") ?></th> + <?php if ($row["RegistrationTS"]): ?> <td><?= (new DateTime($row["RegistrationTS"]))->format('Y-m-d') ?></td> + <?php else: ?> + <td><?= __("unknown") ?></td> + <?php endif; ?> </tr> <?php if (has_credential(CRED_ACCOUNT_LAST_LOGIN)): ?> <tr> -- 2.10.0

1 0

[aur-dev] aurweb 4.4.0 released
by Lukas Fleischer 16 Oct '16

16 Oct '16

Dear list, I am sending this a bit later than usual -- aurweb 4.4.0 was released yesterday and the official aurweb setup [1] was upgraded a couple of hours ago. You can now change package ownership using the SSH interface, check the new commands in the `ssh aur(a)aur.archlinux.org help` output for details. For a comprehensive list of changes, please consult the Git log [2]. As usual, bugs should be reported to the aurweb bug tracker [3]. Wish you all a nice Sunday! [1] https://aur.archlinux.org/ [2] https://projects.archlinux.org/aurweb.git/log/?id=v4.4.0 [3] https://bugs.archlinux.org/index.php?project=2

1 0

[aur-dev] [PATCH] git-serve: Close orphan requests upon disown
by Lukas Fleischer 11 Oct '16

11 Oct '16

When disowning a package base via the SSH interface, auto-accept all pending orphan requests for the affected package. Also, add a test case that checks whether (only) orphan requests belonging to disowned packages are closed correctly. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- aurweb/git/serve.py | 44 +++++++++++++++++++++++++++++++++++++++++++- test/t1200-git-serve.sh | 23 +++++++++++++++++++++++ 2 files changed, 66 insertions(+), 1 deletion(-) diff --git a/aurweb/git/serve.py b/aurweb/git/serve.py index ebfef94..476aea8 100755 --- a/aurweb/git/serve.py +++ b/aurweb/git/serve.py @@ -183,6 +183,44 @@ def pkgbase_set_comaintainers(pkgbase, userlist, user, privileged): conn.close() +def pkgreq_by_pkgbase(pkgbase_id, reqtype): + conn = aurweb.db.Connection() + + cur = conn.execute("SELECT PackageRequests.ID FROM PackageRequests " + + "INNER JOIN RequestTypes ON " + + "RequestTypes.ID = PackageRequests.ReqTypeID " + + "WHERE PackageRequests.Status = 0 " + + "AND PackageRequests.PackageBaseID = ?" + + "AND RequestTypes.Name = ?", [pkgbase_id, reqtype]) + + return [row[0] for row in cur.fetchall()] + + +def pkgreq_close(reqid, reason, comments, autoclose=False): + statusmap = {'accepted': 2, 'rejected': 3} + if reason not in statusmap: + die('{:s}: invalid reason: {:s}'.format(action, reason)) + status = statusmap[reason] + + conn = aurweb.db.Connection() + + if autoclose: + userid = 0 + else: + cur = conn.execute("SELECT ID FROM Users WHERE Username = ?", [user]) + userid = cur.fetchone()[0] + if userid == 0: + die('{:s}: unknown user: {:s}'.format(action, user)) + + conn.execute("UPDATE PackageRequests SET Status = ?, ClosureComment = ? " + + "WHERE ID = ?", [status, comments, reqid]) + conn.commit() + conn.close() + + subprocess.Popen((notify_cmd, 'request-close', str(userid), str(reqid), + reason)).wait() + + def pkgbase_disown(pkgbase, user, privileged): pkgbase_id = pkgbase_from_name(pkgbase) if not pkgbase_id: @@ -193,7 +231,11 @@ def pkgbase_disown(pkgbase, user, privileged): die('{:s}: permission denied: {:s}'.format(action, user)) # TODO: Support disowning package bases via package request. - # TODO: Scan through pending orphan requests and close them. + + # Scan through pending orphan requests and close them. + comment = 'The user {:s} disowned the package.'.format(user) + for reqid in pkgreq_by_pkgbase(pkgbase_id, 'orphan'): + pkgreq_close(reqid, 'accepted', comment, True) comaintainers = [] new_maintainer_userid = None diff --git a/test/t1200-git-serve.sh b/test/t1200-git-serve.sh index 2f1926e..5054ce3 100755 --- a/test/t1200-git-serve.sh +++ b/test/t1200-git-serve.sh @@ -317,4 +317,27 @@ test_expect_success "Force-disown a package base and check (co-)maintainer list. test_cmp expected actual ' +test_expect_success "Check whether package requests are closed when disowning." ' + SSH_ORIGINAL_COMMAND="adopt foobar" AUR_USER=user AUR_PRIVILEGED=0 \ + "$GIT_SERVE" 2>&1 && + cat <<-EOD | sqlite3 aur.db && + INSERT INTO PackageRequests (ID, ReqTypeID, PackageBaseID, PackageBaseName, UsersID) VALUES (1, 2, 3, "foobar", 4); + INSERT INTO PackageRequests (ID, ReqTypeID, PackageBaseID, PackageBaseName, UsersID) VALUES (2, 3, 3, "foobar", 5); + INSERT INTO PackageRequests (ID, ReqTypeID, PackageBaseID, PackageBaseName, UsersID) VALUES (3, 2, 2, "foobar2", 6); + EOD + >sendmail.out && + SSH_ORIGINAL_COMMAND="disown foobar" AUR_USER=user AUR_PRIVILEGED=0 \ + "$GIT_SERVE" 2>&1 && + cat <<-EOD >expected && + Subject: [PRQ#1] Request Accepted + EOD + grep "^Subject.*PRQ" sendmail.out >sendmail.parts && + test_cmp sendmail.parts expected && + cat <<-EOD >expected && + 1|2|3|foobar||4||The user user disowned the package.|0|2 + EOD + echo "SELECT * FROM PackageRequests WHERE Status = 2;" | sqlite3 aur.db >actual && + test_cmp actual expected +' + test_done -- 2.10.0

1 0

[aur-dev] [PATCH] upgrading/4.4.0.txt: Warn about new scripts
by Lukas Fleischer 11 Oct '16

11 Oct '16

The location of the Git interface wrapper scripts was changed by commit d4fe77a (Reorganize Git interface scripts, 2016-10-08). Add a note to the upgrade instructions to remind users to update their configuration files. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- upgrading/4.4.0.txt | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/upgrading/4.4.0.txt b/upgrading/4.4.0.txt index 1cc55b3..afe01be 100644 --- a/upgrading/4.4.0.txt +++ b/upgrading/4.4.0.txt @@ -10,3 +10,8 @@ ALTER TABLE Packages MODIFY URL VARCHAR(8000) NULL DEFAULT NULL; ALTER TABLE PackageSources MODIFY Source VARCHAR(8000) NOT NULL DEFAULT "/dev/null"; ---- + +3. The location of the Git interface scripts was changed. Make sure you update + your aurweb configuration, as well as the SSH daemon and AUR Git repository + configurations to point to the new wrapper scripts which are located in + /usr/local/bin/ by default. -- 2.10.0

1 0

[aur-dev] [PATCH 1/2] Make URL columns 8000 characters wide
by Lukas Fleischer 11 Oct '16

11 Oct '16

According to RFC 7230, URLs can be up too 8000 characters long. Resize all URL fields accordingly. Also, add a test to verify that URLs with more than 8000 characters are rejected by the update hook. Reported-by: Andreas Linz <klingt.net(a)gmail.com> Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- aurweb/git/update.py | 5 +++-- schema/aur-schema.sql | 4 ++-- test/t1300-git-update.sh | 16 ++++++++++++++++ upgrading/4.4.0.txt | 12 ++++++++++++ 4 files changed, 33 insertions(+), 4 deletions(-) create mode 100644 upgrading/4.4.0.txt diff --git a/aurweb/git/update.py b/aurweb/git/update.py index 7337341..af2dfed 100755 --- a/aurweb/git/update.py +++ b/aurweb/git/update.py @@ -324,8 +324,9 @@ def main(): die_commit('invalid package name: {:s}'.format( pkginfo['pkgname']), str(commit.id)) - for field in ('pkgname', 'pkgdesc', 'url'): - if field in pkginfo and len(pkginfo[field]) > 255: + max_len = {'pkgname': 255, 'pkgdesc': 255, 'url': 8000} + for field in max_len.keys(): + if field in pkginfo and len(pkginfo[field]) > max_len[field]: die_commit('{:s} field too long: {:s}'.format(field, pkginfo[field]), str(commit.id)) diff --git a/schema/aur-schema.sql b/schema/aur-schema.sql index 030370b..30209bd 100644 --- a/schema/aur-schema.sql +++ b/schema/aur-schema.sql @@ -119,7 +119,7 @@ CREATE TABLE Packages ( Name VARCHAR(255) NOT NULL, Version VARCHAR(255) NOT NULL DEFAULT '', Description VARCHAR(255) NULL DEFAULT NULL, - URL VARCHAR(255) NULL DEFAULT NULL, + URL VARCHAR(8000) NULL DEFAULT NULL, PRIMARY KEY (ID), UNIQUE (Name), FOREIGN KEY (PackageBaseID) REFERENCES PackageBases(ID) ON DELETE CASCADE @@ -227,7 +227,7 @@ CREATE INDEX RelationsRelName ON PackageRelations (RelName); -- CREATE TABLE PackageSources ( PackageID INTEGER UNSIGNED NOT NULL, - Source VARCHAR(255) NOT NULL DEFAULT "/dev/null", + Source VARCHAR(8000) NOT NULL DEFAULT "/dev/null", SourceArch VARCHAR(255) NULL DEFAULT NULL, FOREIGN KEY (PackageID) REFERENCES Packages(ID) ON DELETE CASCADE ) ENGINE = InnoDB; diff --git a/test/t1300-git-update.sh b/test/t1300-git-update.sh index b642089..abab7ea 100755 --- a/test/t1300-git-update.sh +++ b/test/t1300-git-update.sh @@ -309,6 +309,22 @@ test_expect_success 'Pushing .SRCINFO with invalid epoch.' ' grep -q "^error: invalid epoch: !$" actual ' +test_expect_success 'Pushing .SRCINFO with too long URL.' ' + old=$(git -C aur.git rev-parse HEAD) && + url="http://$(printf "%7993s" x | sed "s/ /x/g")/" && + test_when_finished "git -C aur.git reset --hard $old" && + ( + cd aur.git && + sed "s#.*url.*#\\0\\nurl = $url#" .SRCINFO >.SRCINFO.new + mv .SRCINFO.new .SRCINFO + git commit -q -am "Change URL" + ) && + new=$(git -C aur.git rev-parse HEAD) && + AUR_USER=user AUR_PKGBASE=foobar AUR_PRIVILEGED=0 \ + test_must_fail "$GIT_UPDATE" refs/heads/master "$old" "$new" >actual 2>&1 && + grep -q "^error: url field too long: $url\$" actual +' + test_expect_success 'Missing install file.' ' old=$(git -C aur.git rev-parse HEAD) && test_when_finished "git -C aur.git reset --hard $old" && diff --git a/upgrading/4.4.0.txt b/upgrading/4.4.0.txt new file mode 100644 index 0000000..1cc55b3 --- /dev/null +++ b/upgrading/4.4.0.txt @@ -0,0 +1,12 @@ +1. Resize the URL column of the Packages table: + +---- +ALTER TABLE Packages MODIFY URL VARCHAR(8000) NULL DEFAULT NULL; +---- + +2. Resize the Source column of the PackageSources table: + +---- +ALTER TABLE PackageSources + MODIFY Source VARCHAR(8000) NOT NULL DEFAULT "/dev/null"; +---- -- 2.10.0

1 1

[aur-dev] 4.4.0 string freeze
by Lukas Fleischer 08 Oct '16

08 Oct '16

Dear translators, Strings for 4.4.0 have been submitted to Transifex. I plan to tag the release on Saturday, 2016-10-15, so there are ~7 days left to translate the new messages. Thank you for contributing! Regards, Lukas

1 0

[aur-dev] [PATCH] Update .gitignore
by Lukas Fleischer 08 Oct '16

08 Oct '16

Remove obsolete files. Add byte-compiled Python files and test results. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- .gitignore | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.gitignore b/.gitignore index c41bf3e..2cb5bdc 100644 --- a/.gitignore +++ b/.gitignore @@ -3,8 +3,8 @@ dummy-data.sql* po/*.mo po/*.po~ po/POTFILES -scripts/git-integration/__pycache__/ -scripts/git-integration/templates/ web/locale/*/ aur.git/ -git-interface/git-auth.sh +__pycache__/ +*.py[cod] +test/test-results/ -- 2.10.0

1 0

[aur-dev] [PATCH] Reorganize Git interface scripts
by Lukas Fleischer 08 Oct '16

08 Oct '16

Move the Git interface scripts from git-interface/ to aurweb/git/. Use setuptools to automatically create wrappers which can be installed using `python3 setup.py install`. Update the configuration files, the test suite as well as the INSTALL and README files to reflect these changes. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- INSTALL | 28 ++++++++++------------ README | 5 +--- git-interface/git-auth.py => aurweb/git/auth.py | 0 git-interface/git-serve.py => aurweb/git/serve.py | 0 .../git-update.py => aurweb/git/update.py | 0 conf/config.proto | 4 ++-- git-interface/Makefile | 18 -------------- git-interface/__init__.py | 0 git-interface/config.mk | 1 - git-interface/git-auth.sh.in | 3 --- setup.py | 7 ++++++ test/setup.sh | 8 +++---- 12 files changed, 26 insertions(+), 48 deletions(-) rename git-interface/git-auth.py => aurweb/git/auth.py (100%) rename git-interface/git-serve.py => aurweb/git/serve.py (100%) rename git-interface/git-update.py => aurweb/git/update.py (100%) delete mode 100644 git-interface/Makefile delete mode 100644 git-interface/__init__.py delete mode 100644 git-interface/config.mk delete mode 100644 git-interface/git-auth.sh.in diff --git a/INSTALL b/INSTALL index dab48cc..395915a 100644 --- a/INSTALL +++ b/INSTALL @@ -37,11 +37,16 @@ Setup on Arch Linux $ mysql -uaur -p AUR </srv/http/aurweb/schema/aur-schema.sql -5) Create a new user: +5) Install Python modules and dependencies: + + # pacman -S python-mysql-connector python-pygit2 python-srcinfo + # python3 setup.py install + +6) Create a new user: # useradd -U -d /srv/http/aurweb -c 'AUR user' aur -6) Initialize the Git repository: +7) Initialize the Git repository: # mkdir /srv/http/aurweb/aur.git/ # cd /srv/http/aurweb/aur.git/ @@ -49,29 +54,20 @@ Setup on Arch Linux # git config --local transfer.hideRefs '^refs/' # git config --local --add transfer.hideRefs '!refs/' # git config --local --add transfer.hideRefs '!HEAD' - # ln -s ../../git-interface/git-update.py hooks/update + # ln -s /usr/local/bin/aurweb-git-update hooks/update # chown -R aur . -7) Install needed Python modules: - - # pacman -S python-mysql-connector python-pygit2 python-srcinfo - -8) Install the git-auth wrapper script: - - # cd /srv/http/aurweb/git-interface/ - # make && make install - -9) Configure sshd(8) for the AUR. Add the following lines at the end of your +8) Configure sshd(8) for the AUR. Add the following lines at the end of your sshd_config(5) and restart the sshd. Note that OpenSSH 6.9 or newer is needed! Match User aur PasswordAuthentication no - AuthorizedKeysCommand /usr/local/bin/aur-git-auth "%t" "%k" + AuthorizedKeysCommand /usr/local/bin/aurweb-git-auth "%t" "%k" AuthorizedKeysCommandUser aur -10) If you want to enable smart HTTP support with nginx and fcgiwrap, you can - use the following directives: +9) If you want to enable smart HTTP support with nginx and fcgiwrap, you can + use the following directives: location ~ "^/([a-z0-9][a-z0-9.+_-]*?)(\.git)?/(git-(receive|upload)-pack|HEAD|info/refs|objects/(info/(http-)?alternates|packs)|[0-9a-f]{2}/[0-9a-f]{38}|pack/pack-[0-9a-f]{40}\.(pack|idx))$" { fastcgi_pass unix:/run/fcgiwrap.sock; diff --git a/README b/README index c110ef8..e633ec3 100644 --- a/README +++ b/README @@ -18,7 +18,7 @@ Directory Layout ---------------- aurweb:: - Shared aurweb Python modules. + aurweb Python modules. conf:: Configuration and configuration templates. @@ -26,9 +26,6 @@ conf:: doc:: Project documentation. -git-interface:: - Scripts that make up the Git and SSH interface. - po:: Translation files for strings in the aurweb interface. diff --git a/git-interface/git-auth.py b/aurweb/git/auth.py similarity index 100% rename from git-interface/git-auth.py rename to aurweb/git/auth.py diff --git a/git-interface/git-serve.py b/aurweb/git/serve.py similarity index 100% rename from git-interface/git-serve.py rename to aurweb/git/serve.py diff --git a/git-interface/git-update.py b/aurweb/git/update.py similarity index 100% rename from git-interface/git-update.py rename to aurweb/git/update.py diff --git a/conf/config.proto b/conf/config.proto index 21441a9..96fad80 100644 --- a/conf/config.proto +++ b/conf/config.proto @@ -46,14 +46,14 @@ RSA = SHA256:Ju+yWiMb/2O+gKQ9RJCDqvRg7l+Q95KFAeqM5sr6l2s [auth] valid-keytypes = ssh-rsa ssh-dss ecdsa-sha2-nistp256 ecdsa-sha2-nistp384 ecdsa-sha2-nistp521 ssh-ed25519 username-regex = [a-zA-Z0-9]+[.\-_]?[a-zA-Z0-9]+$ -git-serve-cmd = /srv/http/aurweb/git-interface/git-serve.py +git-serve-cmd = /usr/local/bin/aurweb-git-serve ssh-options = restrict [serve] repo-path = /srv/http/aurweb/aur.git/ repo-regex = [a-z0-9][a-z0-9.+_-]*$ git-shell-cmd = /usr/bin/git-shell -git-update-cmd = /srv/http/aurweb/git-interface/git-update.py +git-update-cmd = /usr/local/bin/aurweb-git-update ssh-cmdline = ssh aur(a)aur.archlinux.org [update] diff --git a/git-interface/Makefile b/git-interface/Makefile deleted file mode 100644 index 8865790..0000000 --- a/git-interface/Makefile +++ /dev/null @@ -1,18 +0,0 @@ -GIT_INTERFACE_DIR := $(abspath $(dir $(lastword $(MAKEFILE_LIST)))) - -include config.mk - -git-auth.sh: - sed 's#%GIT_INTERFACE_DIR%#$(GIT_INTERFACE_DIR)#' <git-auth.sh.in >git-auth.sh - chmod +x git-auth.sh - -install: git-auth.sh - install -Dm0755 git-auth.sh "$(DESTDIR)$(PREFIX)/bin/aur-git-auth" - -uninstall: - rm -f "$(DESTDIR)$(PREFIX)/bin/aur-git-auth" - -clean: - rm -f git-auth.sh - -.PHONY: install uninstall clean diff --git a/git-interface/__init__.py b/git-interface/__init__.py deleted file mode 100644 index e69de29..0000000 diff --git a/git-interface/config.mk b/git-interface/config.mk deleted file mode 100644 index 4d794a1..0000000 --- a/git-interface/config.mk +++ /dev/null @@ -1 +0,0 @@ -PREFIX = /usr/local diff --git a/git-interface/git-auth.sh.in b/git-interface/git-auth.sh.in deleted file mode 100644 index 223816a..0000000 --- a/git-interface/git-auth.sh.in +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh - -%GIT_INTERFACE_DIR%/git-auth.py "$1" "$2" diff --git a/setup.py b/setup.py index 48eb176..b64e71c 100644 --- a/setup.py +++ b/setup.py @@ -17,4 +17,11 @@ setup( name="aurweb", version=version, packages=find_packages(), + entry_points={ + 'console_scripts': [ + 'aurweb-git-auth = aurweb.git.auth:main', + 'aurweb-git-serve = aurweb.git.serve:main', + 'aurweb-git-update = aurweb.git.update:main', + ], + }, ) diff --git a/test/setup.sh b/test/setup.sh index dc9cff2..d02d298 100644 --- a/test/setup.sh +++ b/test/setup.sh @@ -8,9 +8,9 @@ PYTHONPATH="$TOPLEVEL" export PYTHONPATH # Configure paths to the Git interface scripts. -GIT_AUTH="$TOPLEVEL/git-interface/git-auth.py" -GIT_SERVE="$TOPLEVEL/git-interface/git-serve.py" -GIT_UPDATE="$TOPLEVEL/git-interface/git-update.py" +GIT_AUTH="$TOPLEVEL/aurweb/git/auth.py" +GIT_SERVE="$TOPLEVEL/aurweb/git/serve.py" +GIT_UPDATE="$TOPLEVEL/aurweb/git/update.py" MKPKGLISTS="$TOPLEVEL/scripts/mkpkglists.py" TUVOTEREMINDER="$TOPLEVEL/scripts/tuvotereminder.py" PKGMAINT="$TOPLEVEL/scripts/pkgmaint.py" @@ -38,7 +38,7 @@ reply-to = noreply(a)aur.archlinux.org [auth] valid-keytypes = ssh-rsa ssh-dss ecdsa-sha2-nistp256 ecdsa-sha2-nistp384 ecdsa-sha2-nistp521 ssh-ed25519 username-regex = [a-zA-Z0-9]+[.\-_]?[a-zA-Z0-9]+$ -git-serve-cmd = /srv/http/aurweb/git-interface/git-serve.py +git-serve-cmd = $GIT_SERVE ssh-options = restrict [serve] -- 2.10.0

1 0