Officially, the tarballs uploaded to the AUR should be named after their package, contain a directory named after their package, contain no dot files and most importantly contain no binaries. Officially, these requirements are very important. Here are a bunch of non-conforming packages. Maybe 90% of them. (A few errors slip though my scanner.) Of the +700 packages with binaries, most are a simple desktop icon. Should these be base64 encoded if someone can't find hosting? If no one can think of a better way to deal with the nonconforming packages, I'll write a bot to post insulting comments. Personally, I really like this solution. The AUR has always had a wild west frontier / insane asylum feel to it. The less regulation, the better it works. But a few well placed suggestions could help make the two thousand maintainers do a better job. -Kyle http://kmkeen.com
On Friday 03 December 2010 19:46:10 keenerd wrote:
Officially, the tarballs uploaded to the AUR should be named after their package, contain a directory named after their package, contain no dot files and most importantly contain no binaries. Officially, these requirements are very important.
Here are a bunch of non-conforming packages. Maybe 90% of them. (A few errors slip though my scanner.)
Of the +700 packages with binaries, most are a simple desktop icon. Should these be base64 encoded if someone can't find hosting?
If no one can think of a better way to deal with the nonconforming packages, I'll write a bot to post insulting comments. Personally, I really like this solution. The AUR has always had a wild west frontier / insane asylum feel to it. The less regulation, the better it works. But a few well placed suggestions could help make the two thousand maintainers do a better job.
-Kyle
please send insults, i'll find my eventually wrong packages faster :p
Honestly if there was a parser that would just inform us of our sucktitude ever now and then we would most likely become better maintainers. I am all for the insults, diplomatic insults of course On Fri, Dec 3, 2010 at 11:55 AM, Ike Devolder <ike.devolder@gmail.com>wrote:
On Friday 03 December 2010 19:46:10 keenerd wrote:
Officially, the tarballs uploaded to the AUR should be named after their package, contain a directory named after their package, contain no dot files and most importantly contain no binaries. Officially, these requirements are very important.
Here are a bunch of non-conforming packages. Maybe 90% of them. (A few errors slip though my scanner.)
Of the +700 packages with binaries, most are a simple desktop icon. Should these be base64 encoded if someone can't find hosting?
If no one can think of a better way to deal with the nonconforming packages, I'll write a bot to post insulting comments. Personally, I really like this solution. The AUR has always had a wild west frontier / insane asylum feel to it. The less regulation, the better it works. But a few well placed suggestions could help make the two thousand maintainers do a better job.
-Kyle
please send insults, i'll find my eventually wrong packages faster :p
Am Fri, 3 Dec 2010 12:00:25 -0700 schrieb Thomas S Hatch <thatch45@gmail.com>:
Honestly if there was a parser that would just inform us of our sucktitude ever now and then we would most likely become better maintainers.
I am all for the insults, diplomatic insults of course
Isn't there `makepkg --source` which should be used to build the tarballs for the AUR upload? It's mentioned on the submit page (http://aur.archlinux.org/pkgsubmit.php): "Upload your source packages here. Create source packages with `makepkg --source`." Heiko
On 4 December 2010 05:31, Heiko Baums <lists@baums-on-web.de> wrote:
Am Fri, 3 Dec 2010 12:00:25 -0700 schrieb Thomas S Hatch <thatch45@gmail.com>:
Honestly if there was a parser that would just inform us of our sucktitude ever now and then we would most likely become better maintainers.
I am all for the insults, diplomatic insults of course
Isn't there `makepkg --source` which should be used to build the tarballs for the AUR upload?
Yes, I don't see the need for any guideline here, aside from that which is enforced by running makepkg. Simply stab anyone who tries to be funny and includes Apple trailers.
Am 03.12.2010 19:46, schrieb keenerd:
Officially, the tarballs uploaded to the AUR should be named after their package, contain a directory named after their package, contain no dot files and most importantly contain no binaries. Officially, these requirements are very important.
Here are a bunch of non-conforming packages. Maybe 90% of them. (A few errors slip though my scanner.)
Of the +700 packages with binaries, most are a simple desktop icon. Should these be base64 encoded if someone can't find hosting?
If no one can think of a better way to deal with the nonconforming packages, I'll write a bot to post insulting comments. Personally, I really like this solution. The AUR has always had a wild west frontier / insane asylum feel to it. The less regulation, the better it works. But a few well placed suggestions could help make the two thousand maintainers do a better job.
-Kyle
Hello, I think, icon files should be tolerated, and always have been (since I use Arch Linux), if there is a desktop file and no downloadable icon delivered upstream. Having desktop files which point to an icon but not having the icon itself does not make much sense to me. Yes, taken verbatim, icons fall under binaries. But the spirit behind the restriction is that binaries often meen "executable binaries" which are virtually always downloadable or build by the makepkg step. Regards Stefan
On 2010-12-03 20:33 +0100 (48:5) Stefan Husmann wrote:
Am 03.12.2010 19:46, schrieb keenerd:
Officially, the tarballs uploaded to the AUR should be named after their package, contain a directory named after their package, contain no dot files and most importantly contain no binaries. Officially, these requirements are very important.
Here are a bunch of non-conforming packages. Maybe 90% of them. (A few errors slip though my scanner.)
Of the +700 packages with binaries, most are a simple desktop icon. Should these be base64 encoded if someone can't find hosting?
If no one can think of a better way to deal with the nonconforming packages, I'll write a bot to post insulting comments. Personally, I really like this solution. The AUR has always had a wild west frontier / insane asylum feel to it. The less regulation, the better it works. But a few well placed suggestions could help make the two thousand maintainers do a better job.
-Kyle
Hello,
I think, icon files should be tolerated, and always have been (since I use Arch Linux), if there is a desktop file and no downloadable icon delivered upstream. Having desktop files which point to an icon but not having the icon itself does not make much sense to me.
Yes, taken verbatim, icons fall under binaries. But the spirit behind the restriction is that binaries often meen "executable binaries" which are virtually always downloadable or build by the makepkg step.
Regards Stefan
I agree with Stefan. Also, base64 encoding files would only increase the size of the package (however insignificantly) without any benefit. The "no binaries" rule really means "don't include compiled files, large files, sandwiches, or anything else that shouldn't be in here" :P
The day was 03/12/10 21:24 when , Xyne had this to say......:
On 2010-12-03 20:33 +0100 (48:5) Stefan Husmann wrote:
Am 03.12.2010 19:46, schrieb keenerd:
Officially, the tarballs uploaded to the AUR should be named after their package, contain a directory named after their package, contain no dot files and most importantly contain no binaries. Officially, these requirements are very important.
Here are a bunch of non-conforming packages. Maybe 90% of them. (A few errors slip though my scanner.)
Of the +700 packages with binaries, most are a simple desktop icon. Should these be base64 encoded if someone can't find hosting?
If no one can think of a better way to deal with the nonconforming packages, I'll write a bot to post insulting comments. Personally, I really like this solution. The AUR has always had a wild west frontier / insane asylum feel to it. The less regulation, the better it works. But a few well placed suggestions could help make the two thousand maintainers do a better job.
-Kyle
Hello,
I think, icon files should be tolerated, and always have been (since I use Arch Linux), if there is a desktop file and no downloadable icon delivered upstream. Having desktop files which point to an icon but not having the icon itself does not make much sense to me.
Yes, taken verbatim, icons fall under binaries. But the spirit behind the restriction is that binaries often meen "executable binaries" which are virtually always downloadable or build by the makepkg step.
Regards Stefan
I agree with Stefan. Also, base64 encoding files would only increase the size of the package (however insignificantly) without any benefit.
The "no binaries" rule really means "don't include compiled files, large files, sandwiches, or anything else that shouldn't be in here" :P
I to hope that icons as ok, I've tested base64 encoding with ine of my pkg's that has an icon in and the size increased by over a third (from 60223 B to 82412 B). Also worth mentioning is that i only included the icon when someone requested a .desktop file, and as Stefan mentioned they're a bit 'off' with no matching icon.
If no one can think of a better way to deal with the nonconforming
packages, I'll write a bot to post insulting comments. Personally, I really like this solution.
Please don't send me horrible comments! If anyone's interested I used this site for the encoding. http://www.motobit.com/util/base64-decoder-encoder.asp --
On 12/04/2010 03:16 AM, Simon Stoakley wrote:
The day was 03/12/10 21:24 when , Xyne had this to say......:
On 2010-12-03 20:33 +0100 (48:5) Stefan Husmann wrote:
Am 03.12.2010 19:46, schrieb keenerd:
Officially, the tarballs uploaded to the AUR should be named after their package, contain a directory named after their package, contain no dot files and most importantly contain no binaries. Officially, these requirements are very important.
Here are a bunch of non-conforming packages. Maybe 90% of them. (A few errors slip though my scanner.)
Of the +700 packages with binaries, most are a simple desktop icon. Should these be base64 encoded if someone can't find hosting?
If no one can think of a better way to deal with the nonconforming packages, I'll write a bot to post insulting comments. Personally, I really like this solution. The AUR has always had a wild west frontier / insane asylum feel to it. The less regulation, the better it works. But a few well placed suggestions could help make the two thousand maintainers do a better job.
-Kyle
Hello,
I think, icon files should be tolerated, and always have been (since I use Arch Linux), if there is a desktop file and no downloadable icon delivered upstream. Having desktop files which point to an icon but not having the icon itself does not make much sense to me.
Yes, taken verbatim, icons fall under binaries. But the spirit behind the restriction is that binaries often meen "executable binaries" which are virtually always downloadable or build by the makepkg step.
Regards Stefan
I agree with Stefan. Also, base64 encoding files would only increase the size of the package (however insignificantly) without any benefit.
The "no binaries" rule really means "don't include compiled files, large files, sandwiches, or anything else that shouldn't be in here" :P
I to hope that icons as ok, I've tested base64 encoding with ine of my pkg's that has an icon in and the size increased by over a third (from 60223 B to 82412 B). Also worth mentioning is that i only included the icon when someone requested a .desktop file, and as Stefan mentioned they're a bit 'off' with no matching icon.
If no one can think of a better way to deal with the nonconforming
packages, I'll write a bot to post insulting comments. Personally, I really like this solution.
Please don't send me horrible comments!
If anyone's interested I used this site for the encoding.
The raw icon data gets encoded to approx. 4/3 of it's original size. This is because base64 encodes 3 bytes to 4 printable characters. (And it's only approximately 4/3 because the data is padded to have a size that is a multiple of 3. You may have noticed the '=' at the end of the encoded string) -- freenode/pyropeter "12:50 - Ich drücke Return."
Thanks for all the input. I'm pushing the posts now and it should be done in a few hours. For now it is just doing a single pass, but in the future I'll set it up to track the RSS. There is a lot of fun stuff in the AUR. More stats later. For now, here is my favorite: 472 packages had a single PNG. 1 package had 100 PNGs. For more fun, try to find the bot. He's tagging 4% of packages, so it should not be too hard. -Kyle http://kmkeen.com
On Sun, Dec 5, 2010 at 9:42 PM, keenerd <keenerd@gmail.com> wrote:
Thanks for all the input. I'm pushing the posts now and it should be done in a few hours. For now it is just doing a single pass, but in the future I'll set it up to track the RSS.
There is a lot of fun stuff in the AUR. More stats later. For now, here is my favorite:
472 packages had a single PNG. 1 package had 100 PNGs.
For more fun, try to find the bot. He's tagging 4% of packages, so it should not be too hard.
-Kyle http://kmkeen.com
I would suggest modifying the bot's comments to point out the fact it's an automated message and not an actual user, so people aren't replying to it and expecting a response. It may make more sense to include this functionality as part of the upload process and notify the maintainer at that point, so comment areas aren't filled with bot spam every time there's a new package revision for packages with legitimate "binary" file usage. Thanks, Slash
On Sun, Dec 05, 2010 at 10:19:01PM -0500, Slash wrote:
On Sun, Dec 5, 2010 at 9:42 PM, keenerd <keenerd@gmail.com> wrote:
Thanks for all the input. I'm pushing the posts now and it should be done in a few hours. For now it is just doing a single pass, but in the future I'll set it up to track the RSS.
There is a lot of fun stuff in the AUR. More stats later. For now, here is my favorite:
472 packages had a single PNG. 1 package had 100 PNGs.
For more fun, try to find the bot. He's tagging 4% of packages, so it should not be too hard.
-Kyle http://kmkeen.com
I would suggest modifying the bot's comments to point out the fact it's an automated message and not an actual user, so people aren't replying to it and expecting a response.
It may make more sense to include this functionality as part of the upload process and notify the maintainer at that point, so comment areas aren't filled with bot spam every time there's a new package revision for packages with legitimate "binary" file usage.
IMO, the AUR shouldn't even warn you about this. Either your tarball passes a set of criteria (described by the packaging guidelines) and it's accepted to the AUR, or its rejected reason and the user can try again. queue mentions of aur is in 'maintenance mode', etc etc... dave
On Sun, Dec 5, 2010 at 10:39 PM, Dave Reisner <d@falconindy.com> wrote:
IMO, the AUR shouldn't even warn you about this. Either your tarball passes a set of criteria (described by the packaging guidelines) and it's accepted to the AUR, or its rejected reason and the user can try again.
I think there is a a difference between pass/fail requirements and "best practices". The AUR _is_ warning me about a best practice- via automated bot. I don't think comments are the best place to send automated best practice notices. If the AUR had to notify me, the upload form would make more sense, IMO. Personally, I agree that the AUR probably shouldn't be doing this at all, but I was just making a suggestion assuming it was going to be done regardless. Anyways, judging by the comments in this thread, there is no one tool or best place to put something like this, currently (like namcap). If the bot will only comment once per package, I guess it won't be too intrusive- besides this one-time spam for existing packages ;) Slash
On Mon, Dec 6, 2010 at 12:14 AM, Slash <demodevil@gmail.com> wrote:
Anyways, judging by the comments in this thread, there is no one tool or best place to put something like this, currently (like namcap). If the bot will only comment once per package, I guess it won't be too intrusive- besides this one-time spam for existing packages ;)
It's an experiment I've been working on for some time. To appease Heiko I've removed all trace of personality and variety from the form message. I've also come across a bug in the AUR. In short, the tarball URL provided by the RPC interface is different from the tarball taken from the html page. The RPC tarball is *exactly* what was uploaded. While the html tarball has been sanitized. So let's say someone uploads something that is not even a tarball. The AUR fixes this and pushed it to the html. The RPC link goes to the original, and Mr Robot complains. Human looks at html tarball and sees nothing wrong. Confusion abound. I'll remove those comments. -Kyle http://kmkeen.com
On Sun, Dec 5, 2010 at 9:26 PM, keenerd <keenerd@gmail.com> wrote:
I've also come across a bug in the AUR. In short, the tarball URL provided by the RPC interface is different from the tarball taken from the html page. The RPC tarball is *exactly* what was uploaded. While the html tarball has been sanitized. ...
Christopher Brannon also mentioned this on the aur-dev list, giving links to the flyspray bug reports as well: http://mailman.archlinux.org/pipermail/aur-dev/2010-November/001345.html I don't understand why the URLPath RPC field is necessary if the source package files always have the same predictable URL... -- -Justin
Am Mon, 6 Dec 2010 00:26:22 -0500 schrieb keenerd <keenerd@gmail.com>:
It's an experiment I've been working on for some time. To appease Heiko I've removed all trace of personality and variety from the form message.
In most cases there's a reason for having binaries, icons and the like in a package. And whether such a package actually has a bad quality or its contents are necessary can't be decided by a bot. It's pretty the same as the case when someone thought in the past it was sufficient to just comparing two package names and to sending a removal request for one of these packages just because a part of the package names is equal without looking into these packages and without reading the PKGBUILD. So such a bot can probably help you to finding possible candidates with a bad packaging quality, but you have to verify those packages by yourself. So a bot should at the very most create a list of those packages for you, but should definitely not write comments to AUR. Then you should verify the packages on this list by looking into those packages and reading the PKGBUILDs. Only if you then find a package which really doesn't respect the policies you can post a comment for this package manually or create another list with those packages and let a bot sending the comments to the packages on this second list. But having a bot sending such comments just because there's one .desktop file or icon in the package is spam. And think about the responses of the maintainers or other users to those comments. And consider that this spam goes into the inboxes of up to hundreds of people. Btw., the QA in AUR is usually pretty good, because comments for a package are usually written pretty fast by other users or TUs if a package doesn't respect some guidelines, has bugs or a bad quality or isn't trustworthy. And if a maintainer doesn't respond to such comments or doesn't fix those issues users usually send an orphan request to the mailing list to be able to fix these issues themselves. So there's usually no need for such a bot.
I've also come across a bug in the AUR. In short, the tarball URL provided by the RPC interface is different from the tarball taken from the html page. The RPC tarball is *exactly* what was uploaded. While the html tarball has been sanitized. So let's say someone uploads something that is not even a tarball. The AUR fixes this and pushed it to the html. The RPC link goes to the original, and Mr Robot complains. Human looks at html tarball and sees nothing wrong. Confusion abound. I'll remove those comments.
I don't know how all the AUR scripts like yaourt, aurbuild, clyde etc. retrieve the tarballs from AUR, whether they get it from the HTML or the RPC interface. And I don't know how the HTML interface should sanitize packages and what you actually mean with sanitize. But I had absolutely no such problems with AUR packages, yet. Heiko
On Mon, Dec 06, 2010 at 03:20:06PM +0100, Heiko Baums wrote:
In most cases there's a reason for having binaries, icons and the like in a package. And whether such a package actually has a bad quality or its contents are necessary can't be decided by a bot.
In _all_ cases, binaries are not permissable as stated by the AUR guidelines [1]. Your opinion doesn't change this. A proposal to amend the guidelines can.
<snip>
Btw., the QA in AUR is usually pretty good, because comments for a package are usually written pretty fast by other users or TUs if a package doesn't respect some guidelines, has bugs or a bad quality or isn't trustworthy.
My day job is QA. I assure you, based on what Kyle found and in my own personal experience, it's not that good. A lot of things do get caught. A lot of things don't. How many packages still don't have an arch= declaration, or still make reference to $startdir? While less offensive, I could give you a list of packages that have 2777 permissions on the tarballed folder. Kyle hasn't even touched parsing the PKGBUILD itself, but I might be tempted to borrow his database and do such a thing.
And if a maintainer doesn't respond to such comments or doesn't fix those issues users usually send an orphan request to the mailing list to be able to fix these issues themselves.
So there's usually no need for such a bot.
I've also come across a bug in the AUR. In short, the tarball URL provided by the RPC interface is different from the tarball taken from the html page. The RPC tarball is *exactly* what was uploaded. While the html tarball has been sanitized. So let's say someone uploads something that is not even a tarball. The AUR fixes this and pushed it to the html. The RPC link goes to the original, and Mr Robot complains. Human looks at html tarball and sees nothing wrong. Confusion abound. I'll remove those comments.
I don't know how all the AUR scripts like yaourt, aurbuild, clyde etc. retrieve the tarballs from AUR, whether they get it from the HTML or the RPC interface. And I don't know how the HTML interface should sanitize packages and what you actually mean with sanitize. But I had absolutely no such problems with AUR packages, yet.
As an author of one of these abominations, I (cower) make an assumption that the package can be found at /packages/%s/%s.tar.gz, simply because the URLPath provided in the JSON reply is not trustworthy. It seems as though the database is populated on upload, rather than after parsing, which leads to these problems. It's not really a big deal. Frankly, if any change were to be made, I'd vote for URLPath to be removed from the JSON reply. dave [1] https://wiki.archlinux.org/index.php/AUR
Dave Reisner <d@falconindy.com> writes:
As an author of one of these abominations, I (cower) make an assumption that the package can be found at /packages/%s/%s.tar.gz, simply because the URLPath provided in the JSON reply is not trustworthy. It seems as though the database is populated on upload, rather than after parsing,
I think some of the old database entries are just stale. They weren't updated to reflect changes to the code. Case in point, hocr. Here's a curl command to retrieve the JSON data for that package: curl 'http://aur.archlinux.org/rpc.php?type=info&arg=hocr' -- Chris
On 6 December 2010 22:47, Dave Reisner <d@falconindy.com> wrote:
On Mon, Dec 06, 2010 at 03:20:06PM +0100, Heiko Baums wrote:
In most cases there's a reason for having binaries, icons and the like in a package. And whether such a package actually has a bad quality or its contents are necessary can't be decided by a bot.
In _all_ cases, binaries are not permissable as stated by the AUR guidelines [1]. Your opinion doesn't change this. A proposal to amend the guidelines can.
Binaries here means binary executables. Nobody told us to read between the lines to pick out technical file types (of which an image file would be a 'binary file'). In the same manner, even if a set of Python scripts is not comprised of 'binary files', they _are_ executables and should be contained in a tarball linked online. Bash (or rather any interpreted language) scripts can be included if they are to be part of the distribution framework, eg. init scripts. Of course, by common understanding, if there is a set of Bash scripts which form a 'project' or 'application', they should be uploaded somewhere. There is no need to ammend the guidelines. We have been including desktop files, images (needed by the desktop files most of the time) and init scripts all along, because it should be a common understanding. find /var/abs -name *.png | wc -l == 60
On Tue, Dec 07, 2010 at 03:58:25AM +0800, Ray Rashif wrote:
On 6 December 2010 22:47, Dave Reisner <d@falconindy.com> wrote:
On Mon, Dec 06, 2010 at 03:20:06PM +0100, Heiko Baums wrote:
In most cases there's a reason for having binaries, icons and the like in a package. And whether such a package actually has a bad quality or its contents are necessary can't be decided by a bot.
In _all_ cases, binaries are not permissable as stated by the AUR guidelines [1]. Your opinion doesn't change this. A proposal to amend the guidelines can.
Binaries here means binary executables. Nobody told us to read between the lines to pick out technical file types (of which an image file would be a 'binary file').
Fair enough. I took the strict interpretation of this -- non human readable files, including but not limited to: compiled code, images, tarballs, etc. Limiting to binary executables makes a bit more sense. dave
On Mon, Dec 6, 2010 at 2:58 PM, Ray Rashif <schiv@archlinux.org> wrote:
In the same manner, even if a set of Python scripts is not comprised of 'binary files', they _are_ executables and should be contained in a tarball linked online.
I've been working on that one too. Best I've got so far is to check for a source array without any http or ftp links. Though that mainly hits a large number of font packages, which are already flagged as containing binaries. Checking for +x brings up false positives that look like they were packages on windows machines.
find /var/abs -name *.png | wc -l == 60
Of +4800 packages, that is 1.2%. The AUR is more than twice that rate. But while we are running the numbers to determine best practices..... grep -r '|| return 1' /var/abs/ | wc -l == 6165 -Kyle http://kmkeen.com
Am Mon, 6 Dec 2010 16:53:08 -0500 schrieb keenerd <keenerd@gmail.com>:
find /var/abs -name *.png | wc -l == 60
Of +4800 packages, that is 1.2%. The AUR is more than twice that rate. But while we are running the numbers to determine best practices.....
This would be about 480000+ e-mails to users if your bot continues writing those AUR comments. That's too many. As I said before, please, don't do this. You can, of course, let such a bot help you finding "bad" packages. But you have to verify its results personally, before you write such AUR comments. Such automations are usually pretty unreliable except they are written very thoughtfully and are tested a lot. And regarding the 1.2%... Don't trust any statistics you did not even fake. Heiko
On Tue 07 Dec 2010 03:58 +0800, Ray Rashif wrote:
On 6 December 2010 22:47, Dave Reisner <d@falconindy.com> wrote:
On Mon, Dec 06, 2010 at 03:20:06PM +0100, Heiko Baums wrote:
In most cases there's a reason for having binaries, icons and the like in a package. And whether such a package actually has a bad quality or its contents are necessary can't be decided by a bot.
In _all_ cases, binaries are not permissable as stated by the AUR guidelines [1]. Your opinion doesn't change this. A proposal to amend the guidelines can.
There is no need to ammend the guidelines. We have been including desktop files, images (needed by the desktop files most of the time) and init scripts all along, because it should be a common understanding.
find /var/abs -name *.png | wc -l == 60
I might be kind of crazy here, but maybe desktop files and icons are things that should be distributed from upstream. So maintainers should work to get those included like a patch or whatnot.
Am Mon, 6 Dec 2010 21:02:00 -0500 schrieb Loui Chang <louipc.ist@gmail.com>:
I might be kind of crazy here, but maybe desktop files and icons are things that should be distributed from upstream. So maintainers should work to get those included like a patch or whatnot.
I generally agree, but not every upstream does include desktop files and it's not possible for packages which aren't maintained by upstream anymore. We already had this discussion on the mailing lists. So this should be handled quite flexible in AUR as well as in the repos. So if upstream doesn't provide such files or until upstream will provide such files, such files should be included into the downstream packages. And there are upstream devs who disagree and find that those files should be provided by downstream. You won't get the ideal in this point. Heiko
On Mon, Dec 6, 2010 at 9:25 PM, Heiko Baums <lists@baums-on-web.de> wrote:
Am Mon, 6 Dec 2010 21:02:00 -0500 schrieb Loui Chang <louipc.ist@gmail.com>:
I might be kind of crazy here, but maybe desktop files and icons are things that should be distributed from upstream. So maintainers should work to get those included like a patch or whatnot.
I generally agree, but not every upstream does include desktop files and it's not possible for packages which aren't maintained by upstream anymore.
We already had this discussion on the mailing lists. So this should be handled quite flexible in AUR as well as in the repos.
You won't get the ideal in this point.
In the official repositories we don't include these files generally. I think the AUR should be the place where maintainers can choose to include these files if they want. There are all kinds of PKGBUILD's in [unsupported] with all kinds of crazy patches. And including a *.desktop file that upstream doesn't include is essentially a patch. So if we're going to say that AUR PKGBUILD's should not include *.desktop files then we might as well say that AUR PKGBUILD's should not include patches. And if we're going to go with this level of communism then we might as well say that -svn, -git, etc. PKGBUILD's don't belong in [unsupported] because a development branch is basically a series of unofficially released patches. And if we do that we might as well scrap the whole AUR. On the other hand for [core], [extra], and [community] users expect a high level of standardization and QA. Therefore in my opinion things like *.desktop files and patches NEVER belong there except when critically necessary. Let me summarize: [unsupported] is not official so we should only suggest not enforce; [core], [extra], [community] are official and we should enforce our policy to the fullest extent. --Kaiting. -- Kiwis and Limes: http://kaitocracy.blogspot.com/
Kaiting Chen wrote:
On Mon, Dec 6, 2010 at 9:25 PM, Heiko Baums <lists@baums-on-web.de> wrote:
Am Mon, 6 Dec 2010 21:02:00 -0500 schrieb Loui Chang <louipc.ist@gmail.com>:
I might be kind of crazy here, but maybe desktop files and icons are things that should be distributed from upstream. So maintainers should work to get those included like a patch or whatnot.
I generally agree, but not every upstream does include desktop files and it's not possible for packages which aren't maintained by upstream anymore.
We already had this discussion on the mailing lists. So this should be handled quite flexible in AUR as well as in the repos.
You won't get the ideal in this point.
In the official repositories we don't include these files generally. I think the AUR should be the place where maintainers can choose to include these files if they want. There are all kinds of PKGBUILD's in [unsupported] with all kinds of crazy patches. And including a *.desktop file that upstream doesn't include is essentially a patch. So if we're going to say that AUR PKGBUILD's should not include *.desktop files then we might as well say that AUR PKGBUILD's should not include patches.
And if we're going to go with this level of communism then we might as well say that -svn, -git, etc. PKGBUILD's don't belong in [unsupported] because a development branch is basically a series of unofficially released patches. And if we do that we might as well scrap the whole AUR.
On the other hand for [core], [extra], and [community] users expect a high level of standardization and QA. Therefore in my opinion things like *.desktop files and patches NEVER belong there except when critically necessary.
Let me summarize: [unsupported] is not official so we should only suggest not enforce; [core], [extra], [community] are official and we should enforce our policy to the fullest extent. --Kaiting.
I basically agree with previous comments to the effect that some extra files should be allowed in AUR packages provided that they are relatively small. The validity of inclusion should be determined case by case. I definitely don't think that mass-spamming the AUR with a bot is the right way to do it.
On 7 December 2010 12:23, Kaiting Chen <kaitocracy@gmail.com> wrote:
In the official repositories we don't include these files generally.
We do, when needed.
So if we're going to say that AUR PKGBUILD's should not include *.desktop files then we might as well say that AUR PKGBUILD's should not include patches.
Nobody said that! :O
And if we're going to go with this level of communism then we might as well say that -svn, -git, etc. PKGBUILD's don't belong in [unsupported] because a development branch is basically a series of unofficially released patches. And if we do that we might as well scrap the whole AUR.
Yes, such nonsense!
On the other hand for [core], [extra], and [community] users expect a high level of standardization and QA. Therefore in my opinion things like *.desktop files and patches NEVER belong there except when critically necessary.
Let me summarize: [unsupported] is not official so we should only suggest not enforce; [core], [extra], [community] are official and we should enforce our policy to the fullest extent. --Kaiting.
I don't think there's a need to enforce anything here. Likewise.. On 7 December 2010 05:53, keenerd <keenerd@gmail.com> wrote:
But while we are running the numbers to determine best practices.....
grep -r '|| return 1' /var/abs/ | wc -l == 6165
This is something like "Hey, pacman has this cool feature for some time now to detect any kind of error. So the next time you update your builds, remember that you no longer need to return 1." It's not something to "enforce", you update as and when you can. KISS. All we really need right now is a reminder to check if included files are really necessary.
Here are some of my favorites. And some stats about what is in the AUR. -Kyle http://kmkeen.com
Just wanted to say I appreciated the one email I got from the bot that told me there was a .PKGINFO file in my source package. I adopted this package so I'm not sure how the original packager pulled that off. I like the idea. Yes, obviously it should be built into the AUR website itself. The problem is that it is a lot quicker and easier to make a bot. There's also the whole weird "maintenance mode" thing. The cool part is we've already beta tested this thing without modifying the AUR at all. I would not mess with binary files. I always assumed the packaging guidelines meant executable files instead of binaries. Yes executable (ELF) files are bad. I think this has been established earlier. One suggestion is that the comment adds Keenerd's email or some way to know where in the world it came from or who owns it. An anonymous bot! Fear! Another idea: instead of commenting en mass you could send an e-mail to the maintainer with a list of packages and their problems. If they have their e-mail available of course. -- -Justin
On 8 December 2010 03:47, keenerd <keenerd@gmail.com> wrote:
Here are some of my favorites. And some stats about what is in the AUR.
-Kyle http://kmkeen.com
You could also try the following: * PKGBUILDs with executable bit set * Install scriptlets with executable bit set * One or more included files from source array if it's a URL Anyway, we could implement things like these in AUR itself, and the maintainer would be informed upon upload (first submission or subsequent updates) without having to resort to posting comments, and depending on the severity of non-compliance either allow or reject the upload. Also, while you're on this, you can actually send the maintainer an e-mail, rather than posting a comment. That would be pretty slick, actually. But of course, first we need to decide what and what not to warn/inform about.
On Wed, Dec 8, 2010 at 03:16, Ray Rashif <schiv@archlinux.org> wrote:
On 8 December 2010 03:47, keenerd <keenerd@gmail.com> wrote:
Here are some of my favorites. And some stats about what is in the AUR.
-Kyle http://kmkeen.com
You could also try the following:
* PKGBUILDs with executable bit set * Install scriptlets with executable bit set * One or more included files from source array if it's a URL
I am not a TU but I maintain all my PKGBUILDs http://aur.archlinux.org/packages.php?SeB=m&K=skodabenz in a git repo in an ntfs partition so that I can edit even in windows (of course makepkg --source needs arch). Any damn file created in the ntfs partition gets executable bit set. There is a ntfs-3g mount option to disable executable bit for all files but I am not using it as I use some bash scripts which I run from that partition itself instead of copying to home/desktop dir. Just my suggestion. - Keshav
Anyway, we could implement things like these in AUR itself, and the maintainer would be informed upon upload (first submission or subsequent updates) without having to resort to posting comments, and depending on the severity of non-compliance either allow or reject the upload.
Also, while you're on this, you can actually send the maintainer an e-mail, rather than posting a comment. That would be pretty slick, actually. But of course, first we need to decide what and what not to warn/inform about.
On Wed, Dec 8, 2010 at 03:16, Ray Rashif <schiv@archlinux.org> wrote:
On 8 December 2010 03:47, keenerd <keenerd@gmail.com> wrote:
Here are some of my favorites. And some stats about what is in the AUR.
-Kyle http://kmkeen.com
You could also try the following:
* PKGBUILDs with executable bit set * Install scriptlets with executable bit set * One or more included files from source array if it's a URL
I am not a TU but I maintain all my PKGBUILDs http://aur.archlinux.org/packages.php?SeB=m&K=skodabenz in a git repo in an ntfs partition so that I can edit even in windows (of course makepkg --source needs arch). Any damn file created in the ntfs partition gets executable bit set. There is a ntfs-3g mount option to disable executable bit for all files but I am not using it as I use some bash scripts which I run from that partition itself instead of copying to home/desktop dir. So I don't think executable bit checking is a good idea. I would prefer checking for ELF executables instead of any binary. Icons and image files can be allowed. Just my suggestion. - Keshav
Anyway, we could implement things like these in AUR itself, and the maintainer would be informed upon upload (first submission or subsequent updates) without having to resort to posting comments, and depending on the severity of non-compliance either allow or reject the upload.
Also, while you're on this, you can actually send the maintainer an e-mail, rather than posting a comment. That would be pretty slick, actually. But of course, first we need to decide what and what not to warn/inform about.
KESHAV P.R. mentions ntfs and such partitions that, if I understand correctly, don't do per-file executable bit and are therefore typically mounted with everything "executable". Is there any legitimate use of an executable bit in a package? Perhaps makepkg --source could just strip all of the executable bits*. If they bit is really necessary for something, the PKGBUILD could use `chmod` or `install` to make something executable... I know, that's not particularly Arch-y. *(except directories of course) -Isaac
On 8 December 2010 16:38, KESHAV P.R. <skodabenz@gmail.com> wrote:
On Wed, Dec 8, 2010 at 03:16, Ray Rashif <schiv@archlinux.org> wrote:
On 8 December 2010 03:47, keenerd <keenerd@gmail.com> wrote:
Here are some of my favorites. And some stats about what is in the AUR.
-Kyle http://kmkeen.com
You could also try the following:
* PKGBUILDs with executable bit set * Install scriptlets with executable bit set * One or more included files from source array if it's a URL
I am not a TU but I maintain all my PKGBUILDs http://aur.archlinux.org/packages.php?SeB=m&K=skodabenz in a git repo in an ntfs partition so that I can edit even in windows (of course makepkg --source needs arch). Any damn file created in the ntfs partition gets executable bit set. There is a ntfs-3g mount option to disable executable bit for all files but I am not using it as I use some bash scripts which I run from that partition itself instead of copying to home/desktop dir. Just my suggestion.
I'm well aware of that, because I myself have the bulk of my files on an NTFS. I requested the check solely for informational purposes. On 8 December 2010 17:24, Isaac Dupree <ml@isaac.cedarswampstudios.org> wrote:
Is there any legitimate use of an executable bit in a package?
No. None of the files are meant to be executed. Having proper file permissions is just good practice, that's all.
On Tue, 2010-12-07 at 14:47 -0500, keenerd wrote:
Here are some of my favorites. And some stats about what is in the AUR.
-Kyle http://kmkeen.com
Point of curiosity, how were the listed packages created? Some things (like including icons) I can see, but things like .swp etc... doesn't everybody use makepkg --source?
On Wed, Dec 08, 2010 at 08:42:35AM +0800, Ng Oon-Ee wrote:
On Tue, 2010-12-07 at 14:47 -0500, keenerd wrote:
Here are some of my favorites. And some stats about what is in the AUR.
-Kyle http://kmkeen.com
Point of curiosity, how were the listed packages created? Some things (like including icons) I can see, but things like .swp etc... doesn't everybody use makepkg --source?
That was part of the point of the exercise. It'd be nice if everyone did, but it's clearly not the case. d
On 7 December 2010 10:02, Loui Chang <louipc.ist@gmail.com> wrote:
On Tue 07 Dec 2010 03:58 +0800, Ray Rashif wrote:
On 6 December 2010 22:47, Dave Reisner <d@falconindy.com> wrote:
On Mon, Dec 06, 2010 at 03:20:06PM +0100, Heiko Baums wrote:
In most cases there's a reason for having binaries, icons and the like in a package. And whether such a package actually has a bad quality or its contents are necessary can't be decided by a bot.
In _all_ cases, binaries are not permissable as stated by the AUR guidelines [1]. Your opinion doesn't change this. A proposal to amend the guidelines can.
There is no need to ammend the guidelines. We have been including desktop files, images (needed by the desktop files most of the time) and init scripts all along, because it should be a common understanding.
find /var/abs -name *.png | wc -l == 60
I might be kind of crazy here, but maybe desktop files and icons are things that should be distributed from upstream. So maintainers should work to get those included like a patch or whatnot.
Definitely. Another interesting observation I've made in packages from both our repos and AUR: * Upstream tarball includes icons but they are not provided by the install method This gives a false impression that upstream does not provide the icon, so the maintainer includes it in the package. As for desktop files, aside from the above, sometimes they violate specs, so maintainers replace them with compatible ones, including an icon image along as well (forgetting that they can get it from the tarball instead).
On Sun, Dec 5, 2010 at 10:19 PM, Slash <demodevil@gmail.com> wrote:
It may make more sense to include this functionality as part of the upload process and notify the maintainer at that point,
I like Dave's idea better. (So weird not calling him Falconindy.)
so comment areas aren't filled with bot spam every time there's a new package revision for packages with legitimate "binary" file usage.
Not possible. As it is now, it'll at most send a single message per package. Ever. -Kyle http://kmkeen.com
On 6 December 2010 03:42, keenerd <keenerd@gmail.com> wrote:
Thanks for all the input. I'm pushing the posts now and it should be done in a few hours. For now it is just doing a single pass, but in the future I'll set it up to track the RSS.
There is a lot of fun stuff in the AUR. More stats later. For now, here is my favorite:
472 packages had a single PNG. 1 package had 100 PNGs.
For more fun, try to find the bot. He's tagging 4% of packages, so it should not be too hard.
-Kyle http://kmkeen.com
I just got spammed by your bot. You should really omit the warning for png's. It should be easy to change your script only to check whether file output contains "ELF", "executable" or "shared object". To paraphrase your message "I think you can do better." Anyway, this makes me think that the current AUR guidelines should be more clear about this issue. To be more precise I mean this part: After logging in to the AUR web interface, a user can submit a gzipped tarball (.tar.gz) of a directory containing build files for a package. The directory inside the tarball should contain a PKGBUILD, any .install files, patches, etc. (ABSOLUTELY no binaries). I'm pretty sure binaries originally referred to executables and libraries (which are in fact executables). My proposal is to change this sentence accordingly. Either use "ABSOLUTELY no executables" or "ABSOLUTELY no binaries except for icons." Also it should be mentioned in Arch Packaging Standards. opinions?
On Mon, 6 Dec 2010 07:32:51 +0100 Lukáš Jirkovský <l.jirkovsky@gmail.com> wrote:
On 6 December 2010 03:42, keenerd <keenerd@gmail.com> wrote:
Thanks for all the input. I'm pushing the posts now and it should be done in a few hours. For now it is just doing a single pass, but in the future I'll set it up to track the RSS.
There is a lot of fun stuff in the AUR. More stats later. For now, here is my favorite:
472 packages had a single PNG. 1 package had 100 PNGs.
For more fun, try to find the bot. He's tagging 4% of packages, so it should not be too hard.
-Kyle http://kmkeen.com
I just got spammed by your bot. You should really omit the warning for png's. It should be easy to change your script only to check whether file output contains "ELF", "executable" or "shared object". To paraphrase your message "I think you can do better."
Anyway, this makes me think that the current AUR guidelines should be more clear about this issue. To be more precise I mean this part:
After logging in to the AUR web interface, a user can submit a gzipped tarball (.tar.gz) of a directory containing build files for a package. The directory inside the tarball should contain a PKGBUILD, any .install files, patches, etc. (ABSOLUTELY no binaries).
I'm pretty sure binaries originally referred to executables and libraries (which are in fact executables). My proposal is to change this sentence accordingly. Either use "ABSOLUTELY no executables" or "ABSOLUTELY no binaries except for icons."
Also it should be mentioned in Arch Packaging Standards.
opinions?
Hm yes that sounds far more elegant than the proposal I wrote this morning and sent just a couple of minutes ago. -- Jabber: atsutane@freethoughts.de Blog: http://atsutane.freethoughts.de/ Key: 295AFBF4 FP: 39F8 80E5 0E49 A4D1 1341 E8F9 39E4 F17F 295A FBF4
Am Mon, 6 Dec 2010 17:35:33 +0100 schrieb Thorsten Töpper <atsutane@freethoughts.de>:
I'm pretty sure binaries originally referred to executables and libraries (which are in fact executables). My proposal is to change this sentence accordingly. Either use "ABSOLUTELY no executables" or "ABSOLUTELY no binaries except for icons."
Also it should be mentioned in Arch Packaging Standards.
opinions?
Hm yes that sounds far more elegant than the proposal I wrote this morning and sent just a couple of minutes ago.
In Arch Packaging Standards? Arch binary packages without binaries? I think it's sufficient to keep it in the AUR guidelines as Dave mentioned because this affects only AUR, not the repos. Heiko
On 6 December 2010 10:42, keenerd <keenerd@gmail.com> wrote:
Thanks for all the input. I'm pushing the posts now and it should be done in a few hours. For now it is just doing a single pass, but in the future I'll set it up to track the RSS.
There is a lot of fun stuff in the AUR. More stats later. For now, here is my favorite:
472 packages had a single PNG. 1 package had 100 PNGs.
For more fun, try to find the bot. He's tagging 4% of packages, so it should not be too hard.
I'm not liking this one bit. It's definitely spam to me, since comments trigger e-mail notifications and I don't want e-mails from bots just because my package has an image/icon.
Excerpts from keenerd's message of 2010-12-03 13:46:10 -0500:
If no one can think of a better way to deal with the nonconforming packages, I'll write a bot to post insulting comments. Personally, I really like this solution. The AUR has always had a wild west frontier / insane asylum feel to it. The less regulation, the better it works. But a few well placed suggestions could help make the two thousand maintainers do a better job.
Isn't this the sort of thing namcap was designed for? Maybe namcap should be extended to do checks on .src packages, and a report could be posted automatically using namcap when someone posts a .src package to the AUR. -- David Campbell
Am 03.12.2010 22:54, schrieb David Campbell:
Excerpts from keenerd's message of 2010-12-03 13:46:10 -0500:
If no one can think of a better way to deal with the nonconforming packages, I'll write a bot to post insulting comments. Personally, I really like this solution. The AUR has always had a wild west frontier / insane asylum feel to it. The less regulation, the better it works. But a few well placed suggestions could help make the two thousand maintainers do a better job.
Isn't this the sort of thing namcap was designed for? Maybe namcap should be extended to do checks on .src packages, and a report could be posted automatically using namcap when someone posts a .src package to the AUR. -- David Campbell
No, namcap is a tool for providing clean packages, not for clean aur-tarballs. But again, if the "uncleanlyness" of the aur tarballs in discussion here only consists in the fact that they provide icon files, they are IMHO not even unclean. Regards Stefan
On Fri 03 Dec 2010 16:54 -0500, David Campbell wrote:
Excerpts from keenerd's message of 2010-12-03 13:46:10 -0500:
If no one can think of a better way to deal with the nonconforming packages, I'll write a bot to post insulting comments. Personally, I really like this solution. The AUR has always had a wild west frontier / insane asylum feel to it. The less regulation, the better it works. But a few well placed suggestions could help make the two thousand maintainers do a better job.
Isn't this the sort of thing namcap was designed for? Maybe namcap should be extended to do checks on .src packages, and a report could be posted automatically using namcap when someone posts a .src package to the AUR.
The problem is that namcap's implementation is not meant for untrusted PKGBUILDs. Sourcing those build files is a big security flaw, so we can't do that for the AUR.
On Sun, Dec 5, 2010 at 10:55 PM, Loui Chang <louipc.ist@gmail.com> wrote:
The problem is that namcap's implementation is not meant for untrusted PKGBUILDs. Sourcing those build files is a big security flaw, so we can't do that for the AUR.
Thankfully, what I'm doing here does not even look at the pkgbuild. It just looks at the directory structure, runs "file" on everything and compares this to a (tediously compiled) whitelist. Nothing fancy. Would make a lot of sense to have it built in. -Kyle http://kmkeen.com
Am Sun, 5 Dec 2010 22:58:50 -0500 schrieb keenerd <keenerd@gmail.com>:
On Sun, Dec 5, 2010 at 10:55 PM, Loui Chang <louipc.ist@gmail.com> wrote:
The problem is that namcap's implementation is not meant for untrusted PKGBUILDs. Sourcing those build files is a big security flaw, so we can't do that for the AUR.
Thankfully, what I'm doing here does not even look at the pkgbuild. It just looks at the directory structure, runs "file" on everything and compares this to a (tediously compiled) whitelist. Nothing fancy. Would make a lot of sense to have it built in.
-Kyle http://kmkeen.com
Are you R.Daneel? And are you flooding several inboxes with such useless comments regarding "wrong" or "bad" packages just because they contain some "local" files which are not provided by upstream like a single icon or a tarball with a ruleset? Not a good idea. See e.g. opcion and logcheck. The first is a Java application and contains one icon for the desktop menu which is not provided by upstream. The second has some tarballs with rules included which are also not provided by upstream. I'm not the maintainer of these packages. But writing such comments for these packages is just useless and floods needlessly several inboxes of users who are subscribed to those comments. Please, don't exaggerate the "QA". Heiko
Am Mon, 6 Dec 2010 05:50:39 +0100 schrieb Heiko Baums <lists@baums-on-web.de>:
Please, don't exaggerate the "QA".
And, please, stop that! If you - and not a stupid bot - think there's a real issue with a package, write a comment for this package manually or contact the maintainer by private e-mail. Heiko
On Sun, 5 Dec 2010 22:58:50 -0500 keenerd <keenerd@gmail.com> wrote:
On Sun, Dec 5, 2010 at 10:55 PM, Loui Chang <louipc.ist@gmail.com> wrote:
The problem is that namcap's implementation is not meant for untrusted PKGBUILDs. Sourcing those build files is a big security flaw, so we can't do that for the AUR.
Thankfully, what I'm doing here does not even look at the pkgbuild. It just looks at the directory structure, runs "file" on everything and compares this to a (tediously compiled) whitelist. Nothing fancy. Would make a lot of sense to have it built in.
-Kyle http://kmkeen.com
Hm dunno how your Bot works but is there a way to read the size from a png file for it and say everything larger than x*y pixels shall be removed? If not there's still the way to say everything > x KB shall be removed. The rules need to be modified to this anyway, however as Heiko already said, not every upstream tarball provides the icon necessary for a desktop file and there are plenty of apps which need one for DE users. Thorsten
The problem is that namcap's implementation is not meant for untrusted PKGBUILDs. Sourcing those build files is a big security flaw, so we can't do that for the AUR.
We can create minimal chroot with bash and namcap only. It would require changes to the infrastructure but it could improve the PKGBUILDs in AUR a lot. Here's how it could work: * user uploads tarball with a package to AUR, the tarball is moved to the "staging area". * uploader can see his/her (I wonder how many girls are here :-)) package in AUR interface immediately – this is mostly to prevent consecutive uploads of the same package. Other users can't see it until it's checked by namcap. * create the chroot and check the package using namcap. then of course clean the chroot * if there are errors in the package send email/other notification to the uploader. Otherwise the package is made available to public. -> it could be interesting to made namcap results available too. The package "Package Details" could include namcap log somewhere.
participants (21)
-
Christopher Brannon
-
Dave Reisner
-
David Campbell
-
Heiko Baums
-
Ike Devolder
-
Isaac Dupree
-
Justin Davis
-
Kaiting Chen
-
keenerd
-
KESHAV P.R.
-
Loui Chang
-
Lukáš Jirkovský
-
Ng Oon-Ee
-
PyroPeter
-
Ray Rashif
-
Simon Stoakley
-
Slash
-
Stefan Husmann
-
Thomas S Hatch
-
Thorsten Töpper
-
Xyne