- Aur-dev - lists.archlinux.org

[aur-dev] Fighting spam on the AUR
by Lukas Fleischer 18 Mar '13

18 Mar '13

Status quo: 06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date The account suspension feature does not help here. Options: * Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments. * Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1]. * Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR. * Block IP addresses. Bye-bye, Tor users! Comments and suggestions welcome! We need to find a proper solution as soon as possible! [1] http://www.google.com/recaptcha

11 17

[aur-dev] Using git as a backend for the AUR
by William Giokas 17 Mar '13

17 Mar '13

All, So in my spare time I was thinking about the AUR and how it could be better. Back in January I commented on a bug[1] about integrating the AUR and git to have a powerful, robust backend for the AUR. I think that the original idea of creating one massive repository was inherently flawed for most users, as it requires that the entire repository be cloned, and that is no trivial task for people with only moderate internet speeds. Were it similar to the way the official repositories git setup is, then it would be okay for fetching, but using single repositories for each project would make access control much less of a nightmare. At the moment I have no experience with PHP or messing with the AUR, but I have been playing around with git and using it to track packages and the like, as well as some minimal experience with access control and git. Currently, this is kind of the roadmap I see in my head: * Set up access controls for the AUR users to access over ssh using keys. While I could see this being somewhat arduous, it shouldn't be terribly hard to automate or set up. Something like gitolite or whatever should make it simpler. * Each repository on the server would contain a single package (if someone decides to do a split package on the AUR it would contain the whole set of packages), allowing for multiple users to have push access and maintain the packages. * The repositories would be limited to 5M, unless given special permission (certain kernel packages with lots of massive configuration files) which should make enough room for the needed files, while helping to enforce the idea of not putting retrievable sources in the source tarball. I did some quick math using the number of packages on the AUR right now, and if every repo used that 5M limit, it would require a bit over 200G. Granted, I doubt that this limit would be reached for most packages. * With the repository set up, give maintainers a week or so to upload history for their packages (in case they already keep their repos in this set up), and new packages would get put directly into a new repo. * Once the week for transition is over, begin moving all of the current packages in the AUR to the git format. This basically concludes setting up the git stuff, which I have done on a local machine on a very small scale (5 packages) in the past. One thing I was thinking of was the use of `makepkg -S` tarballs still being available. cryptocrack came up with a good point, that just simply copying things over could overwrite the .git directory in the designated directory. I don't think that would be too hard to get around, but I'm no expert. I wrote a quick and dirty script that extracts and commits to the respective repository. Currently, almost no packages use a .changelog file, but there could be some minimal parsing done of a packages .changelog file to craft a git commit message from a tarball. Obviously there needs to be some kind of security check, but that could be done by the frontend using the system we have now for source tarballs. At the moment, I am unsure exactly how the AUR parses the PKGBUILDs it receives, but I'm sure that this could easily be adapted to the git system. One major advantage to having the AUR managed this way is that it would allow users to check for updates on the AUR without the need for complex helpers outside of git. Secondly, it would make mirroring the AUR, or just parts of the AUR extremely easy. In case something happens to the AUR, people can still get their packages and maintainers can still update them very easily using git. It would also still allow people to grab tarballs of the current master branch. This way git is not a requirement to use the AUR, it just makes things a butt ton easier. Also, it would allow users to see the entire source of the package, not just the PKGBUILD, similar to how the official repository works, just that things would be in distinct git repositories not branches. I would really like to see this kind of thing come to fruition, and if anyone would be willing to help, then please say so. Pardon my scattered thoughts, -- William Giokas | KaiSforza GnuPG Key: 0x73CD09CF Fingerprint: F73F 50EF BBE2 9846 8306 E6B8 6902 06D8 73CD 09CF [1]: https://bugs.archlinux.org/task/23010

5 9

[aur-dev] [PATCH] pkgsubmit.php: Parse .SRCINFO metadata
by Lukas Fleischer 17 Mar '13

17 Mar '13

This allows for adding a metadata file called ".SRCINFO" to source tarballs to overwrite specific PKGBUILD fields. .SRCINFO files are parsed line by line. The syntax for each line is "key = value", where key is any of the following field names: * pkgname * pkgver * pkgdesc * url * license * depend Multiple "depend" lines can be specified to add multiple dependencies. This format closely matches the .PKGINFO format that is used for binary packages in pacman/libalpm. It can be extended by field name prefixes or sections to support split packages later. Signed-off-by: Lukas Fleischer <archlinux(a)cryptocrack.de> --- web/html/pkgsubmit.php | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/web/html/pkgsubmit.php b/web/html/pkgsubmit.php index d9bb6bc..4c14be4 100644 --- a/web/html/pkgsubmit.php +++ b/web/html/pkgsubmit.php @@ -81,8 +81,8 @@ if ($uid): if (!$error) { $tar = new Archive_Tar($_FILES['pfile']['tmp_name']); - # Extract PKGBUILD into a string - $pkgbuild_raw = ''; + # Extract PKGBUILD and .SRCINFO into a string + $pkgbuild_raw = $srcinfo_raw = ''; $dircount = 0; foreach ($tar->listContent() as $tar_file) { if ($tar_file['typeflag'] == 0) { @@ -93,6 +93,9 @@ if ($uid): elseif (substr($tar_file['filename'], -9) == '/PKGBUILD') { $pkgbuild_raw = $tar->extractInString($tar_file['filename']); } + elseif (substr($tar_file['filename'], -9) == '/.SRCINFO') { + $srcinfo_raw = $tar->extractInString($tar_file['filename']); + } } elseif ($tar_file['typeflag'] == 5) { if (substr_count($tar_file['filename'], "/") > 1) { @@ -254,6 +257,31 @@ if ($uid): } } + # Parse .SRCINFO and overwrite PKGBUILD fields accordingly + unset($pkg_version); + $depends = array(); + foreach (explode("\n", $srcinfo_raw) as $line) { + if ($line[0] = '#') { + continue; + } + list($key, $value) = explode(' = ', $line, 2); + + switch ($key) { + case 'pkgname': + case 'pkgdesc': + case 'url': + case 'license': + $new_pkgbuild[$key] = $value; + break; + case 'pkgver': + $pkg_version = $value; + break; + case 'depend': + $depends[] = $value; + break; + } + } + # Validate package name if (!$error) { $pkg_name = $new_pkgbuild['pkgname']; @@ -266,7 +294,7 @@ if ($uid): } # Determine the full package version with epoch - if (!$error) { + if (!$error && !isset($pkg_version)) { if (isset($new_pkgbuild['epoch']) && (int)$new_pkgbuild['epoch'] > 0) { $pkg_version = sprintf('%d:%s-%s', $new_pkgbuild['epoch'], $new_pkgbuild['pkgver'], $new_pkgbuild['pkgrel']); } else { @@ -389,8 +417,10 @@ if ($uid): } # Update package depends - if (!empty($new_pkgbuild['depends'])) { + if (empty($depends) && !empty($new_pkgbuild['depends'])) { $depends = explode(" ", $new_pkgbuild['depends']); + } + if (!empty($depends)) { foreach ($depends as $dep) { $deppkgname = preg_replace("/(<|<=|=|>=|>).*/", "", $dep); $depcondition = str_replace($deppkgname, "", $dep); -- 1.8.2.rc2.352.g908df73

3 8

[aur-dev] 2.1.0 string freeze
by Lukas Fleischer 12 Mar '13

12 Mar '13

I added the 2.1.0 release to Transifex yesterday. The string freeze starts today and ends on 2013-03-17. Not a lot of time, but the top ten translations are already 100% complete and the following four languages require <10 new translations: * French * Greek * Romanian * Spanish Thanks again to all contributors!

1 0

[aur-dev] [PATCH] pkgsubmit.php: Parse .AURINFO metadata
by Lukas Fleischer 08 Mar '13

08 Mar '13

This allows for adding a metadata file called ".AURINFO" to source tarballs to overwrite specific PKGBUILD fields. .AURINFO files are parsed line by line. The syntax for each line is "key = value", where key is any of the following field names: * pkgname * pkgver * pkgdesc * url * license * depend Multiple "depend" lines can be specified to add multiple dependencies. This format closely matches the .PKGINFO format that is used for binary packages in pacman/libalpm. It can be extended by field name prefixes or sections to support split packages later. Signed-off-by: Lukas Fleischer <archlinux(a)cryptocrack.de> --- As shortly discussed with Allan on IRC, rename the metadata file to ".AURINFO" until it is officially supported in makepkg(8). Further comments welcome. I will soon merge this into master and prepare a release. web/html/pkgsubmit.php | 37 +++++++++++++++++++++++++++++++++---- 1 file changed, 33 insertions(+), 4 deletions(-) diff --git a/web/html/pkgsubmit.php b/web/html/pkgsubmit.php index d9bb6bc..d2fe512 100644 --- a/web/html/pkgsubmit.php +++ b/web/html/pkgsubmit.php @@ -81,8 +81,8 @@ if ($uid): if (!$error) { $tar = new Archive_Tar($_FILES['pfile']['tmp_name']); - # Extract PKGBUILD into a string - $pkgbuild_raw = ''; + # Extract PKGBUILD and .AURINFO into a string + $pkgbuild_raw = $srcinfo_raw = ''; $dircount = 0; foreach ($tar->listContent() as $tar_file) { if ($tar_file['typeflag'] == 0) { @@ -93,6 +93,9 @@ if ($uid): elseif (substr($tar_file['filename'], -9) == '/PKGBUILD') { $pkgbuild_raw = $tar->extractInString($tar_file['filename']); } + elseif (substr($tar_file['filename'], -9) == '/.AURINFO') { + $srcinfo_raw = $tar->extractInString($tar_file['filename']); + } } elseif ($tar_file['typeflag'] == 5) { if (substr_count($tar_file['filename'], "/") > 1) { @@ -254,6 +257,30 @@ if ($uid): } } + # Parse .AURINFO and overwrite PKGBUILD fields accordingly + unset($pkg_version); + $depends = array(); + foreach (explode("\n", $srcinfo_raw) as $line) { + if (empty($line) || $line[0] == '#') { + continue; + } + list($key, $value) = explode(' = ', $line, 2); + switch ($key) { + case 'pkgname': + case 'pkgdesc': + case 'url': + case 'license': + $new_pkgbuild[$key] = $value; + break; + case 'pkgver': + $pkg_version = $value; + break; + case 'depend': + $depends[] = $value; + break; + } + } + # Validate package name if (!$error) { $pkg_name = $new_pkgbuild['pkgname']; @@ -266,7 +293,7 @@ if ($uid): } # Determine the full package version with epoch - if (!$error) { + if (!$error && !isset($pkg_version)) { if (isset($new_pkgbuild['epoch']) && (int)$new_pkgbuild['epoch'] > 0) { $pkg_version = sprintf('%d:%s-%s', $new_pkgbuild['epoch'], $new_pkgbuild['pkgver'], $new_pkgbuild['pkgrel']); } else { @@ -389,8 +416,10 @@ if ($uid): } # Update package depends - if (!empty($new_pkgbuild['depends'])) { + if (empty($depends) && !empty($new_pkgbuild['depends'])) { $depends = explode(" ", $new_pkgbuild['depends']); + } + if (!empty($depends)) { foreach ($depends as $dep) { $deppkgname = preg_replace("/(<|<=|=|>=|>).*/", "", $dep); $depcondition = str_replace($deppkgname, "", $dep); -- 1.8.2.rc2.352.g908df73

1 0

[aur-dev] [PATCH 1/2] pkgsubmit.php: Simplify package name validation
by Lukas Fleischer 05 Mar '13

05 Mar '13

Remove redundant filters -- single quotes are already removed in $pkgbuild_new and we do not pass the package name to a shell (additionally, the regular expression already checks for potentially evil characters). Also, move the $pkg_name extraction up to fix the split package check. Signed-off-by: Lukas Fleischer <archlinux(a)cryptocrack.de> --- web/html/pkgsubmit.php | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/web/html/pkgsubmit.php b/web/html/pkgsubmit.php index fefb31e..685d5cb 100644 --- a/web/html/pkgsubmit.php +++ b/web/html/pkgsubmit.php @@ -268,19 +268,13 @@ if ($uid): } } - # Now we've parsed the pkgbuild, let's move it to where it belongs - if (!$error && $pkg_name[0] == '(') { - $error = __("Error - The AUR does not support split packages!"); - } - + # Validate package name if (!$error) { - $pkg_name = str_replace("'", "", $new_pkgbuild['pkgname']); - $pkg_name = escapeshellarg($pkg_name); - $pkg_name = str_replace("'", "", $pkg_name); - - $presult = preg_match("/^[a-z0-9][a-z0-9\.+_-]*$/", $pkg_name); - - if (!$presult) { + $pkg_name = $new_pkgbuild['pkgname']; + if ($pkg_name[0] == '(') { + $error = __("Error - The AUR does not support split packages!"); + } + if (!preg_match("/^[a-z0-9][a-z0-9\.+_-]*$/", $pkg_name)) { $error = __("Invalid name: only lowercase letters are allowed."); } } -- 1.8.2.rc2.352.g908df73

1 1

[aur-dev] Data left behind from a deleted package
by Connor Behan 02 Mar '13

02 Mar '13

When I recently looked through the AUR source, it seemed to me that when a package is deleted, a record is only removed from the `Packages` table and that there is data for the corresponding package still in `CommentNotify`, `PackageComments`, `PackageContents`, `PackageSources` and `PackageVotes`. Does this orphan data get deleted by another mechanism I haven't seen? If not, could I send patches that do a better job of cleaning it up?

2 1

[aur-dev] [PATCH] Fix typo in delete permission warning
by Connor Behan 02 Mar '13

02 Mar '13

Signed-off-by: Connor Behan <connor.behan(a)gmail.com> --- web/lib/pkgfuncs.inc.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/lib/pkgfuncs.inc.php b/web/lib/pkgfuncs.inc.php index 5ad53b2..53b7e54 100644 --- a/web/lib/pkgfuncs.inc.php +++ b/web/lib/pkgfuncs.inc.php @@ -851,7 +851,7 @@ function pkg_delete ($atype, $ids, $mergepkgid) { # If they're a TU or dev, can delete if ($atype != "Trusted User" && $atype != "Developer") { - return __("You do have permission to delete packages."); + return __("You do not have permission to delete packages."); } $ids = sanitize_ids($ids); -- 1.8.1.4

1 0

[aur-dev] cookies + suspended account
by Daniel Wallace 01 Mar '13

01 Mar '13

Hello, I have been having to deal with some idiot who is pissed off in the aur for some reason. He keeps marking all my packages out of date. And somehow he is able to continually do this even after I have suspended his account. I am not sure if this is because of the cookie still working and him still being logged in. Would it be possible to add captchas to flag packages out of date, or to make it so that suspending an account kills the cookie? https://aur.archlinux.org/account/YyTe/ https://aur.archlinux.org/account/293oHrnk/ https://aur.archlinux.org/account/iou https://aur.archlinux.org/account/b2qLe1Np/ Thanks, -- Daniel Wallace Archlinux Trusted User (gtmanfred) Georgia Institute of Technology

9 12

[aur-dev] [PATCH] Fix hashes in arrays
by Neer Sighted 28 Feb '13

28 Feb '13

I've fixed the parsing of hashes in arrays, as they would have been treated as comments before. If a line with a hash in it is found, and it ends in `')` or `")`, it will not be treated as a comment. I've already submitted this patch as an attachment, but here it is again in the 'proper' form. Neer Sighted (1): pkgsubmit.php: Fix comment parsing in arrays web/html/pkgsubmit.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 1.8.1.4

1 1