June 2011 - Aur-dev - lists.archlinux.org

[aur-dev] [PATCH] Wrap comments in <pre> tags, fixes FS#14391
by Manuel Tortosa 01 Jul '11

01 Jul '11

Exactly this, wrap the comments in <pre> tags allowing patches or at least better build output formatting. Greez

4 4

[aur-dev] [PATCH] Convert the language selection bar in a select form
by Manuel Tortosa 29 Jun '11

29 Jun '11

Subject says all. Greez. Manuel

2 3

[aur-dev] [PATCH] Fix empty depends database insert
by Manuel Tortosa 29 Jun '11

29 Jun '11

In pkgsubmit.php in this part: foreach ($depends as $dep) { $deppkgname = preg_replace("/(<|<=|=|>=|>).*/", "", $dep); $depcondition = str_replace($deppkgname, "", $dep); if ($deppkgname == "#") { break; } $q = sprintf("INSERT INTO PackageDepends (PackageID, DepName, DepCondition) VALUES (%d, '%s', '%s')", $packageID, mysql_real_escape_string($deppkgname), mysql_real_escape_string($depcondition)); db_query($q, $dbh); } This will pass through the loop at least once in case of pkgdesc=(), adding an empty pkgdepends entry to the database. Easy to avoid by the fact $depends = explode(" ", $new_pkgbuild['depends']); returns false. Best Regards.

2 1

[aur-dev] [PATCH 1/3] aurblup: blacklist processing query changes
by Dan McGee 28 Jun '11

28 Jun '11

* Do all list building and freeing outside of the transaction to keep it as short as possible. * Remove ability to blacklist without transactions as we now only support InnoDB/transactional engines with proper relations. * No need to turn autocommit off; BEGIN TRANSACTION operates regardless of this setting. Signed-off-by: Dan McGee <dan(a)archlinux.org> --- scripts/aurblup/aurblup.c | 23 +++++------------------ scripts/aurblup/config.h.proto | 3 --- 2 files changed, 5 insertions(+), 21 deletions(-) diff --git a/scripts/aurblup/aurblup.c b/scripts/aurblup/aurblup.c index fea7ffc..1f7aefc 100644 --- a/scripts/aurblup/aurblup.c +++ b/scripts/aurblup/aurblup.c @@ -122,19 +122,11 @@ blacklist_sync(alpm_list_t *pkgs_cur, alpm_list_t *pkgs_new) { alpm_list_t *pkgs_add, *pkgs_rem, *p; -#if MYSQL_USE_TRANSACTIONS - if (mysql_autocommit(c, 0)) - mysql_die("failed to turn MySQL autocommit off: %s\n"); + pkgs_add = alpm_list_diff(pkgs_new, pkgs_cur, (alpm_list_fn_cmp)strcmp); + pkgs_rem = alpm_list_diff(pkgs_cur, pkgs_new, (alpm_list_fn_cmp)strcmp); if (mysql_query(c, "START TRANSACTION;")) mysql_die("failed to start MySQL transaction: %s\n"); -#else - if (mysql_query(c, "LOCK TABLES PackageBlacklist WRITE;")) - mysql_die("failed to lock MySQL table: %s\n"); -#endif - - pkgs_add = alpm_list_diff(pkgs_new, pkgs_cur, (alpm_list_fn_cmp)strcmp); - pkgs_rem = alpm_list_diff(pkgs_cur, pkgs_new, (alpm_list_fn_cmp)strcmp); for (p = pkgs_add; p; p = alpm_list_next(p)) blacklist_add(alpm_list_getdata(p)); @@ -142,16 +134,11 @@ blacklist_sync(alpm_list_t *pkgs_cur, alpm_list_t *pkgs_new) for (p = pkgs_rem; p; p = alpm_list_next(p)) blacklist_remove(alpm_list_getdata(p)); - alpm_list_free(pkgs_add); - alpm_list_free(pkgs_rem); - -#if MYSQL_USE_TRANSACTIONS if (mysql_query(c, "COMMIT;")) mysql_die("failed to commit MySQL transaction: %s\n"); -#else - if (mysql_query(c, "UNLOCK TABLES;")) - mysql_die("failed to unlock MySQL tables: %s\n"); -#endif + + alpm_list_free(pkgs_add); + alpm_list_free(pkgs_rem); } alpm_list_t * diff --git a/scripts/aurblup/config.h.proto b/scripts/aurblup/config.h.proto index fc1a87b..daf1744 100644 --- a/scripts/aurblup/config.h.proto +++ b/scripts/aurblup/config.h.proto @@ -6,9 +6,6 @@ #define CONFIG_KEY_PASSWD "AUR_db_pass" #define CONFIG_KEY_DB "AUR_db_name" -/* unset this to use "LOCK TABLE" instead of transactions */ -#define MYSQL_USE_TRANSACTIONS 1 - /* libalpm options */ #define ALPM_DBPATH "/var/lib/aurblup/" #define ALPM_MIRROR "ftp://mirrors.kernel.org/archlinux/%s/os/i686" -- 1.7.5.2

2 4

[aur-dev] Python 3 AUR module
by Xyne 27 Jun '11

27 Jun '11

Hi all, I've written a Python 3 AUR module that some of you might find interesting or useful: http://xyne.archlinux.ca/projects/python-aur/ Regards, Xyne

1 0

[aur-dev] misc patches
by Florian Pritz 25 Jun '11

25 Jun '11

I've tested some basic AUR functionality and it still worked fine, so please merge. Cgit (with pull URL) is available at http://git.server-speed.net/users/flo/aur/?h=working

4 10

[aur-dev] epoch fields for packages are completely ignored
by Slavi Pantaleev 24 Jun '11

24 Jun '11

Hey. The epoch field in PKGBUILD files is completely ignored now, and the final Version field for the package consists only of pkgver and pkgrel (example: 5.0-1) This means that various AUR helpers cannot perform updates on packages that have epoch > 0, because the local package is from that higher epoch and the one in the AUR (as reported by rpc.php) is from epoch 0. I'm providing a patch that supposedly fixes the problem by inspecting the epoch field, and if not 0 prepending it to the Version (example: 1:5.0-1). Comments welcome.

2 2

[aur-dev] [PATCH 1/2] Fix performance issues with new PackageDepends lookups
by Dan McGee 24 Jun '11

24 Jun '11

We do a lookup by DepName in the package details view, but I made the silly mistake of forgetting this index addition in the upgrade steps. Signed-off-by: Dan McGee <dan(a)archlinux.org> --- Not sure if this fell through the cracks way back in April... UPGRADING | 6 ++++-- support/schema/aur-schema.sql | 1 + 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/UPGRADING b/UPGRADING index ad7ad96..51afad3 100644 --- a/UPGRADING +++ b/UPGRADING @@ -9,9 +9,11 @@ each AUR upgrade by running `make install` in the "po/" directory. 2. Remove the "NewPkgNotify" column from the "Users" table: ----- ALTER TABLE Users DROP COLUMN NewPkgNotify; ----- + +3. Fix up issues with depends performance on large dataset. + +ALTER TABLE PackageDepends ADD INDEX (DepName); 3. Rename "web/lib/config.inc" to "web/lib/config.inc.php". diff --git a/support/schema/aur-schema.sql b/support/schema/aur-schema.sql index e64ae82..88d074e 100644 --- a/support/schema/aur-schema.sql +++ b/support/schema/aur-schema.sql @@ -123,6 +123,7 @@ CREATE TABLE PackageDepends ( DepName VARCHAR(64) NOT NULL, DepCondition VARCHAR(20), INDEX (PackageID), + INDEX (DepName), FOREIGN KEY (PackageID) REFERENCES Packages(ID) ON DELETE CASCADE ) ENGINE = InnoDB; -- 1.7.5.4

2 3

[aur-dev] Internal server error while searching for "es"
by Tuxce 22 Jun '11

22 Jun '11

Hi, as yaourt maintainer, I received a bug ticket about yaourt giving an error while searching for "es" on AUR. It seems that searching for "es" with rpc (and maybe only "es") returns an internal server error. http://aur.archlinux.org/rpc.php?type=search&arg=es But I cannot reproduce it on a local installed aur (even at version 1.8.2). If it's better to open a task on the bugtracker, just let me know. Regards.

2 1

[aur-dev] [PATCH 1/1] Make cache type selectable based on config value
by elij 22 Jun '11

22 Jun '11

Provie a mechanism to specify cache type from NONE, APC, or MEMCACHE based on a config variable. If MEMCACHE type is selected, a list of servers can be specified to provide multiserver support. Note that php-memcaced is required for MEMCACHE support. --- web/lib/aur.inc.php | 49 +----------------------- web/lib/cachefuncs.inc.php | 85 ++++++++++++++++++++++++++++++++++++++++++ web/lib/config.inc.php.proto | 10 +++++ 3 files changed, 96 insertions(+), 48 deletions(-) create mode 100644 web/lib/cachefuncs.inc.php diff --git a/web/lib/aur.inc.php b/web/lib/aur.inc.php index 7cf43e6..e65677d 100644 --- a/web/lib/aur.inc.php +++ b/web/lib/aur.inc.php @@ -13,12 +13,7 @@ set_lang(); include_once("config.inc.php"); include_once("version.inc.php"); include_once("acctfuncs.inc.php"); - -# Check if APC extension is loaded, and set cache prefix if it is -if (!defined('EXTENSION_LOADED_APC')) { - define('EXTENSION_LOADED_APC', extension_loaded('apc')); - define('APC_PREFIX', 'aur:'); -} +include_once("cachefuncs.inc.php"); # see if the visitor is already logged in # @@ -263,48 +258,6 @@ function db_query($query="", $db_handle="") { return $result; } -# set a value in the cache (currently apc) if cache -# is available for use. if not available, this becomes -# effectively a no-op (return value is false) -# accepts an optional TTL (defaults to 600 seconds) -function set_cache_value($key, $value, $ttl=600) { - $status = false; - if (EXTENSION_LOADED_APC) { - $status = apc_store(APC_PREFIX.$key, $value, $ttl); - } - return $status; -} - -# get a value from the cache (currently apc) if cache -# is available for use. if not available, this -# returns false (optionally sets passed in variable $status -# to false, much like apc_fetch behaves). this allows -# for testing the fetch result appropriately even in the event -# that a 'false' value was the value in the cache. -function get_cache_value($key, &$status=false) { - if(EXTENSION_LOADED_APC) { - $ret = apc_fetch(APC_PREFIX.$key, $status); - if ($status) { - return $ret; - } - } - return $status; -} - -# run a simple db query, retrieving and/or caching the value if APC -# is available for use -# accepts an optioanal TTL value (defaults to 600 seconds) -function db_cache_value($dbq, $dbh, $key, $ttl=600) { - $status = false; - $value = get_cache_value($key, $status); - if (!$status) { - $result = db_query($dbq, $dbh); - $row = mysql_fetch_row($result); - $value = $row[0]; - set_cache_value($key, $value, $ttl); - } - return $value; -} # set up the visitor's language # diff --git a/web/lib/cachefuncs.inc.php b/web/lib/cachefuncs.inc.php new file mode 100644 index 0000000..8e32370 --- /dev/null +++ b/web/lib/cachefuncs.inc.php @@ -0,0 +1,85 @@ +<?php + +if (!defined('CACHE_TYPE')) { + define('CACHE_TYPE', 'NONE'); +} + +# Check if APC extension is loaded, and set cache prefix if it is +if (CACHE_TYPE == 'APC' && !defined('EXTENSION_LOADED_APC')) { + define('EXTENSION_LOADED_APC', extension_loaded('apc')); + define('CACHE_PREFIX', 'aur:'); +} + +# Check if memcache extension is loaded, and set cache prefix if it is +if (CACHE_TYPE == 'MEMCACHE' && !defined('EXTENSION_LOADED_MEMCACHE')) { + define('EXTENSION_LOADED_MEMCACHE', extension_loaded('memcached')); + define('CACHE_PREFIX', 'aur:'); + global $memcache; + $memcache = new Memcached(); + $mcs = defined('MEMCACHE_SERVERS') ? MEMCACHE_SERVERS : '127.0.0.1:11211'; + foreach (explode(',', $mcs) as $elem) { + $telem = trim($elem); + $mcserver = explode(':', $telem); + $memcache->addServer($mcserver[0], intval($mcserver[1])); + } +} + +# set a value in the cache (currently apc) if cache +# is available for use. if not available, this becomes +# effectively a no-op (return value is false) +# accepts an optional TTL (defaults to 600 seconds) +function set_cache_value($key, $value, $ttl=600) { + $status = false; + if (defined('EXTENSION_LOADED_APC')) { + $status = apc_store(CACHE_PREFIX.$key, $value, $ttl); + } + if (defined('EXTENSION_LOADED_MEMCACHE')) { + global $memcache; + $status = $memcache->set(CACHE_PREFIX.$key, $value, $ttl); + } + return $status; +} + +# get a value from the cache (currently apc) if cache +# is available for use. if not available, this +# returns false (optionally sets passed in variable $status +# to false, much like apc_fetch behaves). this allows +# for testing the fetch result appropriately even in the event +# that a 'false' value was the value in the cache. +function get_cache_value($key, &$status=false) { + if(defined('EXTENSION_LOADED_APC')) { + $ret = apc_fetch(CACHE_PREFIX.$key, $status); + if ($status) { + return $ret; + } + } + if (defined('EXTENSION_LOADED_MEMCACHE')) { + global $memcache; + $ret = $memcache->get(CACHE_PREFIX.$key); + if (!$ret) { + $status = false; + } + else { + $status = true; + } + return $ret; + } + return $status; +} + +# run a simple db query, retrieving and/or caching the value if APC +# is available for use +# accepts an optioanal TTL value (defaults to 600 seconds) +function db_cache_value($dbq, $dbh, $key, $ttl=600) { + $status = false; + $value = get_cache_value($key, $status); + if (!$status) { + $result = db_query($dbq, $dbh); + $row = mysql_fetch_row($result); + $value = $row[0]; + set_cache_value($key, $value, $ttl); + } + return $value; +} + +?> diff --git a/web/lib/config.inc.php.proto b/web/lib/config.inc.php.proto index 43c64d2..5351a4a 100644 --- a/web/lib/config.inc.php.proto +++ b/web/lib/config.inc.php.proto @@ -24,6 +24,16 @@ define("DEFAULT_LANG", "en"); # development. Should not be enabled in production. Default to 0 (off). define("SQL_DEBUG", 0); +# set cache type. Either "APC", "MEMCACHE", or "NONE" +# defaults to NONE +# +#define("CACHE_TYPE", "APC"); +#define("CACHE_TYPE", "MEMCACHE"); +# if using memcache cache_type, list servers. you can separate multiple +# servers with a comma, ex: '127.0.0.1:11211,127.0.0.1:11212' +# if undefined, defaults to '127.0.0.1:11211' +#define("MEMCACHE_SERVERS", '127.0.0.1:11211'); + # Languages we have translations for $SUPPORTED_LANGS = array( "ca" => "Català", -- 1.7.2.5

2 4