- Aur-dev - lists.archlinux.org

[aur-dev] [PATCH] t1100: Test AUR_OVERWRITE
by Lukas Fleischer 10 Aug '17

10 Aug '17

Since c5302d3 (Require TUs to explicitly request to overwrite a pkgbase, 2017-07-24), non-fast-forward pushes require setting the AUR_OVERWRITE environment variable. Make sure that git-auth passes this variable to git-serve when it should (and does not pass it if it shouldn't). Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- test/t1100-git-auth.sh | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/test/t1100-git-auth.sh b/test/t1100-git-auth.sh index 71d526f..dd20bea 100755 --- a/test/t1100-git-auth.sh +++ b/test/t1100-git-auth.sh @@ -25,4 +25,21 @@ test_expect_success 'Test authentication with a wrong key.' ' test_must_be_empty out ' +test_expect_success 'Test AUR_OVERWRITE passthrough.' ' + AUR_OVERWRITE=1 \ + "$GIT_AUTH" "$AUTH_KEYTYPE_TU" "$AUTH_KEYTEXT_TU" >out && + grep -q AUR_OVERWRITE=1 out +' + +test_expect_success 'Make sure that AUR_OVERWRITE is unset by default.' ' + "$GIT_AUTH" "$AUTH_KEYTYPE_TU" "$AUTH_KEYTEXT_TU" >out && + grep -q AUR_OVERWRITE=0 out +' + +test_expect_success 'Make sure regular users cannot set AUR_OVERWRITE.' ' + AUR_OVERWRITE=1 \ + "$GIT_AUTH" "$AUTH_KEYTYPE_USER" "$AUTH_KEYTEXT_USER" >out && + grep -q AUR_OVERWRITE=0 out +' + test_done -- 2.14.0

1 0

[aur-dev] [PATCH] git-serve: Fix broken SQL statement
by Lukas Fleischer 08 Aug '17

08 Aug '17

Add a missing space to the SQL statement performing the disown operation. Fixes FS#55068. Note that the broken query was not discovered by the test suite since SQLite parses "?AND" inside prepared statements gracefully while MySQL does not. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- aurweb/git/serve.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aurweb/git/serve.py b/aurweb/git/serve.py index eea0c9c..dfaddd1 100755 --- a/aurweb/git/serve.py +++ b/aurweb/git/serve.py @@ -192,7 +192,7 @@ def pkgreq_by_pkgbase(pkgbase_id, reqtype): "INNER JOIN RequestTypes ON " + "RequestTypes.ID = PackageRequests.ReqTypeID " + "WHERE PackageRequests.Status = 0 " + - "AND PackageRequests.PackageBaseID = ?" + + "AND PackageRequests.PackageBaseID = ? " + "AND RequestTypes.Name = ?", [pkgbase_id, reqtype]) return [row[0] for row in cur.fetchall()] -- 2.14.0

1 0

[aur-dev] [PATCH] t1300: Fix test cases for non-fast-forward pushes
by Lukas Fleischer 08 Aug '17

08 Aug '17

Since commit c5302d3 (Require TUs to explicitly request to overwrite a pkgbase, 2017-07-24), non-fast-forward pushes are denied even for Trusted Users, unless the AUR_OVERWRITE environment variable is set. Mark the test case performing a non-fast-forward push from a TU account as test_must_fail and add another test case performing the same operation with AUR_OVERWRITE=1. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- test/t1300-git-update.sh | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/test/t1300-git-update.sh b/test/t1300-git-update.sh index f16e2ad..109351a 100755 --- a/test/t1300-git-update.sh +++ b/test/t1300-git-update.sh @@ -125,7 +125,18 @@ test_expect_success 'Performing a non-fast-forward ref update.' ' test_expect_success 'Performing a non-fast-forward ref update as Trusted User.' ' old=$(git -C aur.git rev-parse HEAD) && new=$(git -C aur.git rev-parse HEAD^) && + cat >expected <<-EOD && + error: denying non-fast-forward (you should pull first) + EOD AUR_USER=tu AUR_PKGBASE=foobar AUR_PRIVILEGED=1 \ + test_must_fail "$GIT_UPDATE" refs/heads/master "$old" "$new" 2>&1 && + test_cmp expected actual +' + +test_expect_success 'Performing a non-fast-forward ref update with AUR_OVERWRITE=1.' ' + old=$(git -C aur.git rev-parse HEAD) && + new=$(git -C aur.git rev-parse HEAD^) && + AUR_USER=tu AUR_PKGBASE=foobar AUR_PRIVILEGED=1 AUR_OVERWRITE=1 \ "$GIT_UPDATE" refs/heads/master "$old" "$new" 2>&1 ' -- 2.14.0

1 0

[aur-dev] Fixed translation
by Daniel Milde 03 Aug '17

03 Aug '17

Hi all, I fixed one annoying wrong translation in submenu for czech language. Patch in attachement. Thanks Daniel

2 2

[aur-dev] [PATCH] INSTALL: Add new dependencies
by Lukas Fleischer 01 Aug '17

01 Aug '17

Add installation instructions for python-bleach and python-markdown which are required for the rendercomment script. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- INSTALL | 1 + 1 file changed, 1 insertion(+) diff --git a/INSTALL b/INSTALL index 369e1e3..ba24a45 100644 --- a/INSTALL +++ b/INSTALL @@ -48,6 +48,7 @@ read the instructions below. 5) Install Python modules and dependencies: # pacman -S python-mysql-connector python-pygit2 python-srcinfo + # pacman -S python-bleach python-markdown # python3 setup.py install 6) Create a new user: -- 2.13.3

1 0

[aur-dev] [PATCH] mkpkglists: Generate a list of user names
by Lukas Fleischer 01 Aug '17

01 Aug '17

In addition to the packages list and the package base list, also create a list of registered user names. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- aurweb/scripts/mkpkglists.py | 7 +++++++ conf/config.proto | 1 + test/setup.sh | 1 + test/t2100-mkpkglists.sh | 18 ++++++++++++++++++ 4 files changed, 27 insertions(+) diff --git a/aurweb/scripts/mkpkglists.py b/aurweb/scripts/mkpkglists.py index 8a0f2e9..686ee7d 100755 --- a/aurweb/scripts/mkpkglists.py +++ b/aurweb/scripts/mkpkglists.py @@ -8,6 +8,7 @@ import aurweb.db packagesfile = aurweb.config.get('mkpkglists', 'packagesfile') pkgbasefile = aurweb.config.get('mkpkglists', 'pkgbasefile') +userfile = aurweb.config.get('mkpkglists', 'userfile') def main(): @@ -16,6 +17,7 @@ def main(): datestr = datetime.datetime.utcnow().strftime("%a, %d %b %Y %H:%M:%S GMT") pkglist_header = "# AUR package list, generated on " + datestr pkgbaselist_header = "# AUR package base list, generated on " + datestr + userlist_header = "# AUR user name list, generated on " + datestr with gzip.open(packagesfile, "w") as f: f.write(bytes(pkglist_header + "\n", "UTF-8")) @@ -31,6 +33,11 @@ def main(): "WHERE PackagerUID IS NOT NULL") f.writelines([bytes(x[0] + "\n", "UTF-8") for x in cur.fetchall()]) + with gzip.open(userfile, "w") as f: + f.write(bytes(userlist_header + "\n", "UTF-8")) + cur = conn.execute("SELECT UserName FROM Users") + f.writelines([bytes(x[0] + "\n", "UTF-8") for x in cur.fetchall()]) + conn.close() diff --git a/conf/config.proto b/conf/config.proto index 6c637b0..1750929 100644 --- a/conf/config.proto +++ b/conf/config.proto @@ -71,3 +71,4 @@ server = ftp://mirrors.kernel.org/archlinux/%s/os/x86_64 [mkpkglists] packagesfile = /srv/http/aurweb/web/html/packages.gz pkgbasefile = /srv/http/aurweb/web/html/pkgbase.gz +userfile = /srv/http/aurweb/web/html/users.gz diff --git a/test/setup.sh b/test/setup.sh index efa1ab8..d98c49c 100644 --- a/test/setup.sh +++ b/test/setup.sh @@ -61,6 +61,7 @@ server = file://$(pwd)/remote/ [mkpkglists] packagesfile = packages.gz pkgbasefile = pkgbase.gz +userfile = users.gz EOF cat >sendmail.sh <<-\EOF diff --git a/test/t2100-mkpkglists.sh b/test/t2100-mkpkglists.sh index 1e2edc3..fc11d07 100755 --- a/test/t2100-mkpkglists.sh +++ b/test/t2100-mkpkglists.sh @@ -44,4 +44,22 @@ test_expect_success 'Test package list generation.' ' test_cmp actual expected ' +test_expect_success 'Test user list generation.' ' + "$MKPKGLISTS" && + cat <<-EOD >expected && + dev + tu + tu2 + tu3 + tu4 + user + user2 + user3 + user4 + EOD + gunzip users.gz && + sed "/^#/d" users >actual && + test_cmp actual expected +' + test_done -- 2.13.3

1 0

[aur-dev] [PATCH] Unset PackagerUID before deleting an account
by Lukas Fleischer 01 Aug '17

01 Aug '17

When removing an account, remove the user from all last packager fields before deletion to make sure that no package bases are deleted, even if propagation constraints are missing. Fixes FS#53956. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- web/lib/acctfuncs.inc.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/lib/acctfuncs.inc.php b/web/lib/acctfuncs.inc.php index e45d735..bdcaaa8 100644 --- a/web/lib/acctfuncs.inc.php +++ b/web/lib/acctfuncs.inc.php @@ -924,7 +924,7 @@ function user_delete($id) { $fields_set_null = array( array("PackageBases", "SubmitterUID"), array("PackageBases", "MaintainerUID"), - array("PackageBases", "SubmitterUID"), + array("PackageBases", "PackagerUID"), array("PackageComments", "UsersID"), array("PackageComments", "DelUsersID"), array("PackageRequests", "UsersID"), -- 2.13.3

1 0

[aur-dev] [PATCH v4] Require TUs to explicitly request to overwrite a pkgbase
by Eli Schwartz 25 Jul '17

25 Jul '17

AUR_PRIVILEGED allows people with privileged AUR accounts to evade the block on non-fast-forward commits. While valid in this case, we should not do so by default, since in at least one case a TU did this without realizing there was an existing package. ( https://aur.archlinux.org/packages/rtmidi/ ) Switch to using allow_overwrite to check for destructive actions. Use .ssh/config "SendEnv" on the TU's side and and sshd_config "AcceptEnv" in the AUR server to specifically request overwrite access. TUs should use: `AUR_OVERWRITE=1 git push --force` Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- INSTALL | 1 + aurweb/git/auth.py | 2 ++ aurweb/git/update.py | 3 ++- doc/git-interface.txt | 6 ++++++ 4 files changed, 11 insertions(+), 1 deletion(-) diff --git a/INSTALL b/INSTALL index 8c9c4dd..369e1e3 100644 --- a/INSTALL +++ b/INSTALL @@ -76,6 +76,7 @@ read the instructions below. PasswordAuthentication no AuthorizedKeysCommand /usr/local/bin/aurweb-git-auth "%t" "%k" AuthorizedKeysCommandUser aur + AcceptEnv AUR_OVERWRITE 9) If you want to enable smart HTTP support with nginx and fcgiwrap, you can use the following directives: diff --git a/aurweb/git/auth.py b/aurweb/git/auth.py index 022b0ff..d02390d 100755 --- a/aurweb/git/auth.py +++ b/aurweb/git/auth.py @@ -1,5 +1,6 @@ #!/usr/bin/python3 +import os import shlex import re import sys @@ -52,6 +53,7 @@ def main(): env_vars = { 'AUR_USER': user, 'AUR_PRIVILEGED': '1' if account_type > 1 else '0', + 'AUR_OVERWRITE' : os.environ.get('AUR_OVERWRITE', '0') if account_type > 1 else '0', } key = keytype + ' ' + keytext diff --git a/aurweb/git/update.py b/aurweb/git/update.py index c9a98d0..16fa2b0 100755 --- a/aurweb/git/update.py +++ b/aurweb/git/update.py @@ -238,6 +238,7 @@ def main(): user = os.environ.get("AUR_USER") pkgbase = os.environ.get("AUR_PKGBASE") privileged = (os.environ.get("AUR_PRIVILEGED", '0') == '1') + allow_overwrite = (os.environ.get("AUR_OVERWRITE", '0') == '1') warn_or_die = warn if privileged else die if len(sys.argv) == 2 and sys.argv[1] == "restore": @@ -258,7 +259,7 @@ def main(): conn = aurweb.db.Connection() # Detect and deny non-fast-forwards. - if sha1_old != "0" * 40 and not privileged: + if sha1_old != "0" * 40 and not allow_overwrite: walker = repo.walk(sha1_old, pygit2.GIT_SORT_TOPOLOGICAL) walker.hide(sha1_new) if next(walker, None) is not None: diff --git a/doc/git-interface.txt b/doc/git-interface.txt index 7758512..f940961 100644 --- a/doc/git-interface.txt +++ b/doc/git-interface.txt @@ -33,6 +33,11 @@ users cannot access anything on the server except for the aurweb SSH interface. The forced command can be configured in the aurweb configuration file and it usually points to the git-serve program. +If SSH has been configured to pass on the AUR_OVERWRITE environment variable +(via SendEnv, see ssh_config(5) for details) and the user's account is a +registered Trusted User or Developer, this will be passed on to the git-update +program in order to enable a non-fast-forward push. + The INSTALL file in the top-level directory contains detailed instructions on how to configure sshd(8) to use git-auth for authentication. @@ -67,6 +72,7 @@ The Git update hook, called git-update, performs several subtasks: * Prevent from creating branches or tags other than master. * Deny non-fast-forwards, except for Trusted Users and Developers. +* Deny blacklisted packages, except for Trusted Users and Developers. * Verify each new commit (validate meta data, impose file size limits, ...) * Update package base information and package information in the database. * Update the named branch and the namespaced HEAD ref of the package. -- 2.13.3

2 1

[aur-dev] [PATCH] Document new git-serve commands
by Lukas Fleischer 22 Jul '17

22 Jul '17

Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- doc/git-interface.txt | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/doc/git-interface.txt b/doc/git-interface.txt index 475fda6..7758512 100644 --- a/doc/git-interface.txt +++ b/doc/git-interface.txt @@ -41,11 +41,15 @@ The Shell: git-serve The git-serve command, the "aurweb shell", provides different subcommands: +* The adopt/disown commands can be used to change ownership of a package base. +* The flag/unflag commands can be used to flag/unflag a package. * The help command shows a list of available commands. * The list-repos command lists all repositories of the authenticated user. +* The restore command can be used to restore a deleted package base. +* The set-comaintainers command modifies the co-maintainers of a package base. * The set-keywords command modifies the keywords assigned to a package base. * The setup-repo command can be used to create a new repository. -* The restore command can be used to restore a deleted package base. +* The vote/unvote command can be used to vote/unvote for a package base. * The git-{receive,upload}-pack commands are redirected to git-shell(1). The command is extracted from the SSH_ORIGINAL_COMMAND environment variable -- 2.13.3

1 0

[aur-dev] [PATCH v2 2/2] Require TUs to explicitly request to overwrite a pkgbase
by Eli Schwartz 21 Jul '17

21 Jul '17

AUR_PRIVILEGED allows people with privileged AUR accounts to evade the block on non-fast-forward commits. While valid in this case, we should not do so by default, since in at least one case a TU did this without realizing there was an existing package. ( https://aur.archlinux.org/packages/rtmidi/ ) Switch to using allow_overwrite to check for destructive actions. Use .ssh/config "SendEnv" on the TU's side and and sshd_config "AcceptEnv" in the AUR server to specifically request overwrite access. TUs should use: `export AUR_OVERWRITE=1; git push` Signed-off-by: Eli Schwartz <eschwartz(a)archlinux.org> --- v2: only require confirmation for force-pushing history. If you want to restrict who can use this feature, it should be as simple as adding a new account_type and modifying the env_vars passthrough, which can be implemented separately. INSTALL | 1 + aurweb/git/auth.py | 1 + aurweb/git/update.py | 3 ++- 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/INSTALL b/INSTALL index 8c9c4dd..369e1e3 100644 --- a/INSTALL +++ b/INSTALL @@ -76,6 +76,7 @@ read the instructions below. PasswordAuthentication no AuthorizedKeysCommand /usr/local/bin/aurweb-git-auth "%t" "%k" AuthorizedKeysCommandUser aur + AcceptEnv AUR_OVERWRITE 9) If you want to enable smart HTTP support with nginx and fcgiwrap, you can use the following directives: diff --git a/aurweb/git/auth.py b/aurweb/git/auth.py index 022b0ff..fe018cb 100755 --- a/aurweb/git/auth.py +++ b/aurweb/git/auth.py @@ -52,6 +52,7 @@ def main(): env_vars = { 'AUR_USER': user, 'AUR_PRIVILEGED': '1' if account_type > 1 else '0', + 'AUR_OVERWRITE' : os.environ.get('AUR_OVERWRITE', '0') if account_type > 1 else '0', } key = keytype + ' ' + keytext diff --git a/aurweb/git/update.py b/aurweb/git/update.py index 3b9ff97..5419338 100755 --- a/aurweb/git/update.py +++ b/aurweb/git/update.py @@ -238,6 +238,7 @@ def main(): user = os.environ.get("AUR_USER") pkgbase = os.environ.get("AUR_PKGBASE") privileged = (os.environ.get("AUR_PRIVILEGED", '0') == '1') + allow_overwrite = (os.environ.get("AUR_OVERWRITE", '0') == '1') warn_or_die = warn if privileged else die if len(sys.argv) == 2 and sys.argv[1] == "restore": @@ -262,7 +263,7 @@ def main(): walker = repo.walk(sha1_old, pygit2.GIT_SORT_TOPOLOGICAL) walker.hide(sha1_new) if next(walker, None) is not None: - if privileged: + if allow_overwrite: warn("non-fast-forward push (are you absolutely sure you mean this?)") else: die("denying non-fast-forward (you should pull first)") -- 2.13.3

2 3