[aur-dev] Fighting spam on the AUR
Status quo: 06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date The account suspension feature does not help here. Options: * Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments. * Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1]. * Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR. * Block IP addresses. Bye-bye, Tor users! Comments and suggestions welcome! We need to find a proper solution as soon as possible! [1] http://www.google.com/recaptcha
Em 13/03/2013 07:33, Lukas Fleischer escreveu:
* Block IP addresses. Bye-bye, Tor users!
Some ISP provide a dynamic Internet IP address. So, blocking dynamic IP address won't help much as the spammer can simply turn on and off the internet modem.
On Wed, Mar 13, 2013 at 08:08:24AM -0300, Rafael Ferreira wrote:
Em 13/03/2013 07:33, Lukas Fleischer escreveu:
* Block IP addresses. Bye-bye, Tor users!
Some ISP provide a dynamic Internet IP address. So, blocking dynamic IP address won't help much as the spammer can simply turn on and off the internet modem.
Block a whole range of IP addresses and use account-based whitelists then. Yes, this is a lot of work...
On Wed, Mar 13, 2013 at 11:33 AM, Lukas Fleischer <archlinux@cryptocrack.de> wrote:
Status quo:
06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date
The account suspension feature does not help here.
Options:
* Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments.
I suggest a flag 24–hour immunity for added/updated packages and a 60–minute immunity after a package gets unflagged.
* Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1].
MAPTCHAs can be solved easily by bots, reCAPTCHA itself is evil, and image CAPTCHAs can be solved by Indians for a dollar or two per thousand images.
* Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR.
Maybe block the ability of commenting and flagging in the first 24 hours of an user account’s existence?
* Block IP addresses. Bye-bye, Tor users!
Don’t worry, http://proxy.org is here to help our lovely spammers. Also, is email verification necessary? If yes, block 10minutemail.com and other services of this kind. If not, make it so and see “if yes”. -- Kwpolska <http://kwpolska.tk> | GPG KEY: 5EAAEA16 stop html mail | always bottom-post http://asciiribbon.org | http://caliburn.nl/topposting.html
I don't want the AUR to become a closed system where everything has to be approved by TU's or moderators. What if two users were required to mark a package out of date (next to other security measures). Maybe an alternate way (not really a solution) is implementing (better) spam detection algorithms? For reference, how many packages are usually marked out of date per day, and how many are genuine? *Florian Dejonckheere* florian@floriandejonckheere.be http://www.floriandejonckheere.be floriandejonckheere sip:florian@floriandejonckheere.be On 13 March 2013 15:36, Kwpolska <kwpolska@gmail.com> wrote:
On Wed, Mar 13, 2013 at 11:33 AM, Lukas Fleischer <archlinux@cryptocrack.de> wrote:
Status quo:
06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date
The account suspension feature does not help here.
Options:
* Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments.
I suggest a flag 24–hour immunity for added/updated packages and a 60–minute immunity after a package gets unflagged.
* Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1].
MAPTCHAs can be solved easily by bots, reCAPTCHA itself is evil, and image CAPTCHAs can be solved by Indians for a dollar or two per thousand images.
* Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR.
Maybe block the ability of commenting and flagging in the first 24 hours of an user account’s existence?
* Block IP addresses. Bye-bye, Tor users!
Don’t worry, http://proxy.org is here to help our lovely spammers.
Also, is email verification necessary? If yes, block 10minutemail.com and other services of this kind. If not, make it so and see “if yes”.
-- Kwpolska <http://kwpolska.tk> | GPG KEY: 5EAAEA16 stop html mail | always bottom-post http://asciiribbon.org | http://caliburn.nl/topposting.html
Hi, just an AUR spam "victim" here. I had a relatively recent case of one account flagging all of my 183 packages alphabetically at 90- to 120-second intervals. When I began to unflag some of them, a new account flagged those at 60-second intervals the next day. On 03/13/2013 10:36 AM, Kwpolska wrote:
On Wed, Mar 13, 2013 at 11:33 AM, Lukas Fleischer <archlinux@cryptocrack.de> wrote:
Status quo:
06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date
The account suspension feature does not help here.
Options:
* Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments. I suggest a flag 24–hour immunity for added/updated packages and a 60–minute immunity after a package gets unflagged.
* Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1]. MAPTCHAs can be solved easily by bots, reCAPTCHA itself is evil, and image CAPTCHAs can be solved by Indians for a dollar or two per thousand images.
* Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR.
Maybe block the ability of commenting and flagging in the first 24 hours of an user account’s existence? If someone is a new user, they probably want to either comment about a
* Block IP addresses. Bye-bye, Tor users! Don’t worry, http://proxy.org is here to help our lovely spammers.
Also, is email verification necessary? If yes, block 10minutemail.com and other services of this kind. If not, make it so and see “if yes”. Blocking disposable email sites is just playing catch-up. Looking at some of the few other users affected around the same time, all spam flags were done by different accounts with different disposable email sites. Just googling quickly, I can find dozens of various disposable email sites that haven't been used as of yet. Also, this catch-up game is a no-win for mods (TU's); when you take this to its logical conclusion, you get horrendously large databases of spammer's IPs, emails, etc. This is evidenced by stopforumspam.com, which, in an attempt to combat spam, has amassed almost 44 million spammer records. It's a waste to attempt to recreate this kind of thing on the AUR with
In my case, this wouldn't have helped. The spammer waited >24 hours to start reflagging my packages. And if we start extend these intervals, it just wastes the time of legitimate users. package, flag a package, or upload one or two PKGBUILDs. If they're not interested in maintaining packages, then their account is essentially useless for the first 24 hours. these stopgap measures, really. I don't remember if you need to verify your email when you create an AUR account, but that's definitely a good starting point. Still, a lot of these sites allow you to read any email sent to the disposable address (while you have the site tab open), so it's not even close to 100% effective.
-- Kwpolska <http://kwpolska.tk> | GPG KEY: 5EAAEA16 stop html mail | always bottom-post http://asciiribbon.org | http://caliburn.nl/topposting.html
All the solutions posted on this thread (besides Xyne's) are really going in the wrong direction; not only are they just rehashes of old discussions on aur-dev/aur-general, they're focusing on things like IP address and email, or setting time limits, when they should be addressing the behavior itself. These other things can be circumvented (with very minimal effort on the spammer, I will note, but manage to cause significant annoyance to most users), but when a user is, say, systematically flagging one maintainer's packages alphabetically [1], there should be a system (as Xyne has detailed) in place to address the behavior manually (i.e. with TU intervention). If TU's must intervene anyway, let's use some proactive measures, shall we? [1] This is just an example, so don't focus in too hard on this specific behavior and lose sight of the big picture of preventing spam (useless comments, incorrect flags, junk PKGBUILDs) of all kinds. For example, the spammer could have a list of maintainers and cycle through the list, or iterate over them pseudo-randomly, and that would defeat measures tailored to the specific aforementioned behavior.
On Wed, Mar 13, 2013 at 11:33:18AM +0100, Lukas Fleischer wrote:
Status quo:
06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date
The account suspension feature does not help here.
Options:
* Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments.
* Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1].
* Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR.
* Block IP addresses. Bye-bye, Tor users!
Please just do this. We aren't dealing with some "spam epidemic" so much as we are dealing with a small number of bored idiots who are hiding behind Tor exit nodes.
Comments and suggestions welcome! We need to find a proper solution as soon as possible!
On Thu, Mar 14, 2013 at 10:33:39PM -0400, Dave Reisner wrote:
On Wed, Mar 13, 2013 at 11:33:18AM +0100, Lukas Fleischer wrote:
Status quo:
06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date
The account suspension feature does not help here.
Options:
* Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments.
* Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1].
* Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR.
* Block IP addresses. Bye-bye, Tor users!
Please just do this. We aren't dealing with some "spam epidemic" so much as we are dealing with a small number of bored idiots who are hiding behind Tor exit nodes.
Comments and suggestions welcome! We need to find a proper solution as soon as possible!
PLEASE DO THIS https://aur.archlinux.org/packages/ now he is vote spamming -- Daniel Wallace Archlinux Trusted User (gtmanfred) Georgia Institute of Technology
On Mon, Mar 18, 2013 at 01:33:54PM -0400, Daniel Wallace wrote:
On Thu, Mar 14, 2013 at 10:33:39PM -0400, Dave Reisner wrote:
On Wed, Mar 13, 2013 at 11:33:18AM +0100, Lukas Fleischer wrote:
Status quo:
06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date
The account suspension feature does not help here.
Options:
* Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments.
* Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1].
* Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR.
* Block IP addresses. Bye-bye, Tor users!
Please just do this. We aren't dealing with some "spam epidemic" so much as we are dealing with a small number of bored idiots who are hiding behind Tor exit nodes.
Comments and suggestions welcome! We need to find a proper solution as soon as possible!
PLEASE DO THIS
https://aur.archlinux.org/packages/
now he is vote spamming
I hate captchas as much as the next person, but seriously, this is getting to be ridiculous. Something needs to be done, and the best route right now would be captchas. Thanks, -- William Giokas | KaiSforza GnuPG Key: 0x73CD09CF Fingerprint: F73F 50EF BBE2 9846 8306 E6B8 6902 06D8 73CD 09CF
On Mon, Mar 18, 2013 at 1:51 PM, William Giokas <1007380@gmail.com> wrote:
I hate captchas as much as the next person, but seriously, this is getting to be ridiculous. Something needs to be done, and the best route right now would be captchas.
Indeed. Maybe we could implement captchas while we discuss a better long-term solution. Sergio
Thanks, -- William Giokas | KaiSforza GnuPG Key: 0x73CD09CF Fingerprint: F73F 50EF BBE2 9846 8306 E6B8 6902 06D8 73CD 09CF
On Mon, Mar 18, 2013 at 01:33:54PM -0400, Daniel Wallace wrote:
On Thu, Mar 14, 2013 at 10:33:39PM -0400, Dave Reisner wrote:
On Wed, Mar 13, 2013 at 11:33:18AM +0100, Lukas Fleischer wrote:
Status quo:
06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date
The account suspension feature does not help here.
Options:
* Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments.
* Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1].
* Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR.
* Block IP addresses. Bye-bye, Tor users!
Please just do this. We aren't dealing with some "spam epidemic" so much as we are dealing with a small number of bored idiots who are hiding behind Tor exit nodes.
Comments and suggestions welcome! We need to find a proper solution as soon as possible!
PLEASE DO THIS
https://aur.archlinux.org/packages/
now he is vote spamming
I will take care of this today, as I already said before in another reply.
-- Daniel Wallace Archlinux Trusted User (gtmanfred) Georgia Institute of Technology
Am 13.03.2013 11:33, schrieb Lukas Fleischer:
Status quo:
06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date
The account suspension feature does not help here.
Options:
* Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments.
* Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1].
* Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR.
* Block IP addresses. Bye-bye, Tor users!
Comments and suggestions welcome! We need to find a proper solution as soon as possible!
We already tested all this years ago with the Wiki and Forums. Why reinvent the wheel instead of just using an existing solution? I could point you to the code if wanted; it's pretty simple and should be easy to integrate into the aur registration. Greetings, Pierre -- Pierre Schmitz, https://pierre-schmitz.com
On Fri, Mar 15, 2013 at 05:13:43PM +0100, Pierre Schmitz wrote:
Am 13.03.2013 11:33, schrieb Lukas Fleischer:
Status quo:
06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date
The account suspension feature does not help here.
Options:
* Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments.
* Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1].
* Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR.
* Block IP addresses. Bye-bye, Tor users!
Comments and suggestions welcome! We need to find a proper solution as soon as possible!
We already tested all this years ago with the Wiki and Forums. Why reinvent the wheel instead of just using an existing solution? I could point you to the code if wanted; it's pretty simple and should be easy to integrate into the aur registration.
Because we suspect that the bots spamming the AUR were specifically designed for this specific setup of this specific platform and might react to such a simple change. Given the effort required to implement this, I agree that it is worth trying out, though. I will look into this on Monday/Tuesday. If the captcha will not prove itself in practice I will implement a blacklist/whitelist based solution. Thank you for all the replies.
Greetings,
Pierre
-- Pierre Schmitz, https://pierre-schmitz.com
The bot could easily be adjusted to execute the command and pass the form. There seems to be only one command (on the wiki). If multiple commands were to be used (maybe including interactive ones), I think the probability of automation would be greatly decreased. *Florian Dejonckheere* florian@floriandejonckheere.be http://www.floriandejonckheere.be floriandejonckheere sip:florian@floriandejonckheere.be On 15 March 2013 17:33, Lukas Fleischer <archlinux@cryptocrack.de> wrote:
On Fri, Mar 15, 2013 at 05:13:43PM +0100, Pierre Schmitz wrote:
Am 13.03.2013 11:33, schrieb Lukas Fleischer:
Status quo:
06:54 < gtmanfred> ok, it really is time for something else 06:54 < gtmanfred> the spammer is now creating a new account for every comment and flag out of date
The account suspension feature does not help here.
Options:
* Allow package maintainers to block the "Flag package out-of-date" feature for a certain amount of time. Note that this might eventually cripple the "out-of-date" function. Also, this does not work for comments.
* Use CAPTCHAs during account registration. We could either use MAPTCHAs ("What is 1 + 1?") or something like reCAPTCHA [1].
* Moderate new accounts. Might be a lot of work. We need some TUs that review and unlock accounts. Also, it might be hard to distinguish a spam bot from a regular user. If we require a short application text, this might result in less users joining the AUR.
* Block IP addresses. Bye-bye, Tor users!
Comments and suggestions welcome! We need to find a proper solution as soon as possible!
We already tested all this years ago with the Wiki and Forums. Why reinvent the wheel instead of just using an existing solution? I could point you to the code if wanted; it's pretty simple and should be easy to integrate into the aur registration.
Because we suspect that the bots spamming the AUR were specifically designed for this specific setup of this specific platform and might react to such a simple change. Given the effort required to implement this, I agree that it is worth trying out, though.
I will look into this on Monday/Tuesday. If the captcha will not prove itself in practice I will implement a blacklist/whitelist based solution.
Thank you for all the replies.
Greetings,
Pierre
-- Pierre Schmitz, https://pierre-schmitz.com
Am 16.03.2013 17:07, schrieb Florian Dejonckheere:
The bot could easily be adjusted to execute the command and pass the form. There seems to be only one command (on the wiki). If multiple commands were to be used (maybe including interactive ones), I think the probability of automation would be greatly decreased.
The script is configurable to display different questions and answers. You can even set multiple and a random one will be picked. Till now no bot was programmed to bypass the forum or wiki registration, even though both are a way more viable target than the AUR. Therefor I would suggest to just try it; the code is already written. We can gradually extend it later if needed. It's actually important to not go with the best solution we can come up with right from the start. Greetings, Pierre -- Pierre Schmitz, https://pierre-schmitz.com
On Sat, Mar 16, 2013 at 05:26:31PM +0100, Pierre Schmitz wrote:
Am 16.03.2013 17:07, schrieb Florian Dejonckheere:
The bot could easily be adjusted to execute the command and pass the form. There seems to be only one command (on the wiki). If multiple commands were to be used (maybe including interactive ones), I think the probability of automation would be greatly decreased.
The script is configurable to display different questions and answers. You can even set multiple and a random one will be picked.
Till now no bot was programmed to bypass the forum or wiki registration, even though both are a way more viable target than the AUR. Therefor I would suggest to just try it; the code is already written. We can gradually extend it later if needed. It's actually important to not go with the best solution we can come up with right from the start.
I just added a very hacky CAPTCHA implementation to the AUR production system -- let's see what happens.
Greetings,
Pierre
-- Pierre Schmitz, https://pierre-schmitz.com
On Mon, Mar 18, 2013 at 09:36:46PM +0100, Lukas Fleischer wrote:
On Sat, Mar 16, 2013 at 05:26:31PM +0100, Pierre Schmitz wrote:
Am 16.03.2013 17:07, schrieb Florian Dejonckheere:
The bot could easily be adjusted to execute the command and pass the form. There seems to be only one command (on the wiki). If multiple commands were to be used (maybe including interactive ones), I think the probability of automation would be greatly decreased.
The script is configurable to display different questions and answers. You can even set multiple and a random one will be picked.
Till now no bot was programmed to bypass the forum or wiki registration, even though both are a way more viable target than the AUR. Therefor I would suggest to just try it; the code is already written. We can gradually extend it later if needed. It's actually important to not go with the best solution we can come up with right from the start.
I just added a very hacky CAPTCHA implementation to the AUR production system -- let's see what happens.
The spammer worked around this pretty quickly. I disabled the registration form until we come up with a better solution.
Greetings,
Pierre
-- Pierre Schmitz, https://pierre-schmitz.com
The spammer worked around this pretty quickly. I disabled the registration form until we come up with a better solution.
At Chakra we are considering to implement a "request account" form instead of directly allowing user registrations, so any TU or developer should accept the request before the account gets actually created. Is not nice, but maybe may work. Greez Manuel
participants (11)
-
Daniel Wallace
-
Dave Reisner
-
Florian Dejonckheere
-
Kwpolska
-
Limao Luo
-
Lukas Fleischer
-
Manuel Tortosa Moreno
-
Pierre Schmitz
-
Rafael Ferreira
-
Sergio Correia
-
William Giokas