- Aur-dev - lists.archlinux.org

[aur-dev] [PATCH] db.py: Support pyformat paramstyle
by Lukas Fleischer 14 Aug '16

14 Aug '16

This is used by the MySQL database backend. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- git-interface/db.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/git-interface/db.py b/git-interface/db.py index 060689b..75d2283 100644 --- a/git-interface/db.py +++ b/git-interface/db.py @@ -32,7 +32,7 @@ class Connection: raise ValueError('unsupported database backend') def execute(self, query, params=()): - if self._paramstyle == 'format': + if self._paramstyle in ('format', 'pyformat'): query = query.replace('%', '%%').replace('?', '%s') elif self._paramstyle == 'qmark': pass -- 2.9.2

1 0

[aur-dev] [PATCH 1/2] Display registration date on account details page
by Johannes Löthberg 13 Aug '16

13 Aug '16

Signed-off-by: Johannes Löthberg <johannes(a)kyriasis.com> --- Apparently the RegistrationTS isn't actually a UNIX timestamp. It uses CURRENT_TIMESTAMP which is just an alias to NOW in MySQL, leading to a date in "Y-m-d H:M" format, so use DateTime to only print the date part. web/template/account_details.php | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/web/template/account_details.php b/web/template/account_details.php index 8b76382..b79685b 100644 --- a/web/template/account_details.php +++ b/web/template/account_details.php @@ -59,6 +59,10 @@ <?= $row["InactivityTS"] ? __("Inactive since") . ' ' . date("Y-m-d H:i", $row["InactivityTS"]) : __("Active"); ?> </td> </tr> + <tr> + <th><?= __("Registration date:") ?></th> + <td><?= (new DateTime($row["RegistrationTS"]))->format('Y-m-d') ?></td> + </tr> <?php if (has_credential(CRED_ACCOUNT_LAST_LOGIN)): ?> <tr> <th><?= __("Last Login") . ":" ?></th> -- 2.9.2

2 2

[aur-dev] [PATCH 1/2] Lazy-add new package bases
by Lukas Fleischer 10 Aug '16

10 Aug '16

Create new package bases just before saving package metadata. This protects from stray package bases left behind when new packages are rejected, i.e. when the user tries to push a package that is available from the official repositories already. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- git-interface/git-serve.py | 5 +---- git-interface/git-update.py | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/git-interface/git-serve.py b/git-interface/git-serve.py index d3a32c3..19c3ab2 100755 --- a/git-interface/git-serve.py +++ b/git-interface/git-serve.py @@ -145,10 +145,7 @@ if action == 'git-upload-pack' or action == 'git-receive-pack': if not re.match(repo_regex, pkgbase): die('{:s}: invalid repository name: {:s}'.format(action, pkgbase)) - if not pkgbase_exists(pkgbase): - create_pkgbase(pkgbase, user) - - if action == 'git-receive-pack': + if action == 'git-receive-pack' and pkgbase_exists(pkgbase): if not privileged and not pkgbase_has_write_access(pkgbase, user): die('{:s}: permission denied: {:s}'.format(action, user)) diff --git a/git-interface/git-update.py b/git-interface/git-update.py index e6f6410..40d834d 100755 --- a/git-interface/git-update.py +++ b/git-interface/git-update.py @@ -58,6 +58,25 @@ def parse_dep(depstring): return (depname, depcond) +def create_pkgbase(conn, pkgbase, user): + cur = conn.execute("SELECT ID FROM Users WHERE Username = ?", [user]) + userid = cur.fetchone()[0] + + now = int(time.time()) + cur = conn.execute("INSERT INTO PackageBases (Name, SubmittedTS, " + + "ModifiedTS, SubmitterUID, MaintainerUID) VALUES " + + "(?, ?, ?, ?, ?)", [pkgbase, now, now, userid, userid]) + pkgbase_id = cur.lastrowid + + cur = conn.execute("INSERT INTO PackageNotifications " + + "(PackageBaseID, UserID) VALUES (?, ?)", + [pkgbase_id, userid]) + + conn.commit() + + return pkgbase_id + + def save_metadata(metadata, conn, user): # Obtain package base ID and previous maintainer. pkgbase = metadata['pkgbase'] @@ -362,6 +381,10 @@ for pkgname in srcinfo.utils.get_package_names(metadata): if cur.fetchone()[0] > 0: die('cannot overwrite package: {:s}'.format(pkgname)) +# Create a new package base if it does not exist yet. +if pkgbase_id == 0: + pkgbase_id = create_pkgbase(conn, pkgbase, user) + # Store package base details in the database. save_metadata(metadata, conn, user) -- 2.9.2

1 1

[aur-dev] 4.3.0 string freeze
by Lukas Fleischer 09 Aug '16

09 Aug '16

Dear translators, I have pushed the final 4.3.0 update to Transifex a couple of hours ago which brings two new strings (8 new strings since 4.2.0 in total). I would like to tag the release on Sunday, August 14th, so translations are due by Saturday (or early Sunday). Thanks everybody for the time spent improving the AUR. Regards, Lukas

1 0

[aur-dev] [PATCH 0/3] More tests
by Lukas Fleischer 06 Aug '16

06 Aug '16

This is a follow-up to the series I submitted earlier today. All patches are based on pu. Lukas Fleischer (3): t0001: Add more git-auth tests t0002: Add more git-serve tests t0003: Add more git-update tests git-interface/test/setup.sh | 91 ++++++++-- git-interface/test/t0001-auth.sh | 9 + git-interface/test/t0002-serve.sh | 62 +++++++ git-interface/test/t0003-update.sh | 350 ++++++++++++++++++++++++++++++++++++- 4 files changed, 496 insertions(+), 16 deletions(-) -- 2.9.2

1 4

[aur-dev] [PATCH] git-update: Remove package details before updating
by Lukas Fleischer 05 Aug '16

05 Aug '16

Explicitly remove all package sources, dependencies, relations, licenses and groups before inserting new ones. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- Based on pu. git-interface/git-update.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/git-interface/git-update.py b/git-interface/git-update.py index 2820720..e6f6410 100755 --- a/git-interface/git-update.py +++ b/git-interface/git-update.py @@ -78,8 +78,13 @@ def save_metadata(metadata, conn, user): conn.execute("UPDATE PackageBases SET MaintainerUID = ? " + "WHERE ID = ? AND MaintainerUID IS NULL", [user_id, pkgbase_id]) - conn.execute("DELETE FROM Packages WHERE PackageBaseID = ?", - [pkgbase_id]) + for table in ('Sources', 'Depends', 'Relations', 'Licenses', 'Groups'): + conn.execute("DELETE FROM Package" + table + " WHERE EXISTS (" + + "SELECT * FROM Packages " + + "WHERE Packages.PackageBaseID = ? AND " + + "Package" + table + ".PackageID = Packages.ID)", + [pkgbase_id]) + conn.execute("DELETE FROM Packages WHERE PackageBaseID = ?", [pkgbase_id]) for pkgname in srcinfo.utils.get_package_names(metadata): pkginfo = srcinfo.utils.get_merged_package(pkgname, metadata) -- 2.9.2

1 0

[aur-dev] [PATCH 0/8] Introduce a test suite for the Git/SSH interface
by Lukas Fleischer 05 Aug '16

05 Aug '16

This larger patch set adds a test suite for the Git/SSH interface. It also includes some very basic tests of git-auth, git-serve and git-update. The series consists of three major parts: Patches 1, 4, 5, 6 and 7 prepare the code and the database schema such that one can alternatively use SQLite as database backend. Patches 2 and 3 add support for overriding the path to the configuration file using an environment variable. The last patch finally adds the test suite and some tests. Since it makes use of the new SQLite backend, no MySQL server is required. The whole topic will be merged to pu and sit there for a while. Comments welcome! Lukas Fleischer (8): git-interface: Add database abstraction layer git-interface: Factor out configuration file parsing git-interface: Add AUR_CONFIG environment variable git-interface: Do not use rowcount git-interface: Do not use UNIX_TIMESTAMP git-interface: Support SQLite as database backend Cleanup database schema git-interface: Add test suite and basic tests conf/config.proto | 2 +- git-interface/config.py | 31 ++ git-interface/db.py | 51 +++ git-interface/git-auth.py | 33 +- git-interface/git-serve.py | 96 ++--- git-interface/git-update.py | 162 ++++--- git-interface/test/Makefile | 11 + git-interface/test/setup.sh | 121 ++++++ git-interface/test/sharness.sh | 851 +++++++++++++++++++++++++++++++++++++ git-interface/test/t0001-auth.sh | 19 + git-interface/test/t0002-serve.sh | 26 ++ git-interface/test/t0003-update.sh | 20 + schema/aur-schema.sql | 67 ++- 13 files changed, 1288 insertions(+), 202 deletions(-) create mode 100644 git-interface/config.py create mode 100644 git-interface/db.py create mode 100644 git-interface/test/Makefile create mode 100644 git-interface/test/setup.sh create mode 100644 git-interface/test/sharness.sh create mode 100755 git-interface/test/t0001-auth.sh create mode 100755 git-interface/test/t0002-serve.sh create mode 100755 git-interface/test/t0003-update.sh -- 2.9.2

1 8

[aur-dev] [PATCH 1/3] git-update: Fix some issues reported by pyflakes
by Lukas Fleischer 05 Aug '16

05 Aug '16

Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- git-interface/git-update.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/git-interface/git-update.py b/git-interface/git-update.py index 41b38a6..5fc5562 100755 --- a/git-interface/git-update.py +++ b/git-interface/git-update.py @@ -167,11 +167,13 @@ def save_metadata(metadata, db, cur, user): "PackageBaseID = %s AND UserID = %s", [pkgbase_id, user_id]) if cur.fetchone()[0] == 0: - cur.execute("INSERT INTO PackageNotifications (PackageBaseID, UserID) " + - "VALUES (%s, %s)", [pkgbase_id, user_id]) + cur.execute("INSERT INTO PackageNotifications " + + "(PackageBaseID, UserID) VALUES (%s, %s)", + [pkgbase_id, user_id]) db.commit() + def update_notify(db, cur, user, pkgbase_id): # Obtain the user ID of the new maintainer. cur.execute("SELECT ID FROM Users WHERE Username = %s", [user]) @@ -180,6 +182,7 @@ def update_notify(db, cur, user, pkgbase_id): # Execute the notification script. subprocess.Popen((notify_cmd, 'update', str(user_id), str(pkgbase_id))) + def die(msg): sys.stderr.write("error: {:s}\n".format(msg)) exit(1) -- 2.9.2

1 2

[aur-dev] [PATCH] Change default SSH options to "restrict"
by Lukas Fleischer 05 Aug '16

05 Aug '16

>From the sshd(8) man page: Enable all restrictions, i.e. disable port, agent and X11 forwarding, as well as disabling PTY allocation and execution of ~/.ssh/rc. If any future restriction capabilities are added to authorized_keys files they will be included in this set. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- conf/config.proto | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/conf/config.proto b/conf/config.proto index 64af774..d5778a0 100644 --- a/conf/config.proto +++ b/conf/config.proto @@ -47,7 +47,7 @@ RSA = SHA256:Ju+yWiMb/2O+gKQ9RJCDqvRg7l+Q95KFAeqM5sr6l2s valid-keytypes = ssh-rsa ssh-dss ecdsa-sha2-nistp256 ecdsa-sha2-nistp384 ecdsa-sha2-nistp521 ssh-ed25519 username-regex = [a-zA-Z0-9]+[.\-_]?[a-zA-Z0-9]+$ git-serve-cmd = /srv/http/aurweb/git-interface/git-serve.py -ssh-options = no-port-forwarding,no-X11-forwarding,no-pty +ssh-options = restrict [serve] repo-path = /srv/http/aurweb/aur.git/ -- 2.9.2

1 0

[aur-dev] [PATCH] git-serve: Deprecate setup-repo
by Lukas Fleischer 05 Aug '16

05 Aug '16

Since 02dd9c5 (git-serve.py: Automatically create repositories, 2015-01-06), one can create new package bases by running `git push`. It is no longer necessary to run setup-repo manually. Signed-off-by: Lukas Fleischer <lfleischer(a)archlinux.org> --- git-interface/git-serve.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/git-interface/git-serve.py b/git-interface/git-serve.py index aa5f1c9..81b7e93 100755 --- a/git-interface/git-serve.py +++ b/git-interface/git-serve.py @@ -133,6 +133,10 @@ def die_with_help(msg): die(msg + "\nTry `{:s} help` for a list of commands.".format(ssh_cmdline)) +def warn(msg): + sys.stderr.write("warning: {:s}\n".format(msg)) + + user = os.environ.get('AUR_USER') privileged = (os.environ.get('AUR_PRIVILEGED', '0') == '1') ssh_cmd = os.environ.get('SSH_ORIGINAL_COMMAND') @@ -186,6 +190,7 @@ elif action == 'setup-repo': die_with_help("{:s}: missing repository name".format(action)) if len(cmdargv) > 2: die_with_help("{:s}: too many arguments".format(action)) + warn('{:s} is deprecated. Use `git push` to create new repositories.') create_pkgbase(cmdargv[1], user) elif action == 'restore': if len(cmdargv) < 2: -- 2.9.2

1 1