[arch-general] ML is being sent to Spam by Gmail
Recently a (disproportionately) large number of emails from the various Arch Linux mailing lists are getting rerouted to my Spam folder in Gmail. It seems to have started roughly around the time of this change: [arch-devops] Finally fixed DKIM (I hope)[1] dkim header remarks indicate either failed or missing dkim sigs for those messages. Is anyone else seeing this behavior change? @Florian Pritz, whatever you did doesn't seem to be working, in fact it is worse. :( [1] https://lists.archlinux.org/pipermail/arch-devops/2016-May/000066.html -- Eli Schwartz
2016-06-08 6:26 GMT+02:00 Eli Schwartz via arch-general < arch-general@archlinux.org>:
dkim header remarks indicate either failed or missing dkim sigs for those messages.
Is anyone else seeing this behavior change?
I didn't look very long but i could only find one message from this list in my Gmail spam folder. (About a missing GPG key 5 days ago.) The DKIM result on that one was just "none", but it got a -1.3 (minimum 5 needed) spam rating so I guess that's why it was shot down. At work we have had problems with regular messages (including those from Google) ending up being tagged spam though.
I am also seeing many more emails from this mailing list go to the spam folder in GMail (a few every week). I'm don't know how to check the DKIM information though. Srikrishna Sekhar On 8 June 2016 at 11:34, Henrik Danielsson via arch-general < arch-general@archlinux.org> wrote:
2016-06-08 6:26 GMT+02:00 Eli Schwartz via arch-general < arch-general@archlinux.org>:
dkim header remarks indicate either failed or missing dkim sigs for those messages.
Is anyone else seeing this behavior change?
I didn't look very long but i could only find one message from this list in my Gmail spam folder. (About a missing GPG key 5 days ago.) The DKIM result on that one was just "none", but it got a -1.3 (minimum 5 needed) spam rating so I guess that's why it was shot down. At work we have had problems with regular messages (including those from Google) ending up being tagged spam though.
Hi, You can add a filter in gmail, set `list:"arch-general@archlinux.org"` to match the exact email address and click "Never send it to Spam". Enjoy. Best Regards, Syrone Wong On Wed, Jun 8, 2016 at 2:29 PM, Sri Krishna via arch-general <arch-general@archlinux.org> wrote:
I am also seeing many more emails from this mailing list go to the spam folder in GMail (a few every week).
I'm don't know how to check the DKIM information though.
Srikrishna Sekhar
On 8 June 2016 at 11:34, Henrik Danielsson via arch-general < arch-general@archlinux.org> wrote:
2016-06-08 6:26 GMT+02:00 Eli Schwartz via arch-general < arch-general@archlinux.org>:
dkim header remarks indicate either failed or missing dkim sigs for those messages.
Is anyone else seeing this behavior change?
I didn't look very long but i could only find one message from this list in my Gmail spam folder. (About a missing GPG key 5 days ago.) The DKIM result on that one was just "none", but it got a -1.3 (minimum 5 needed) spam rating so I guess that's why it was shot down. At work we have had problems with regular messages (including those from Google) ending up being tagged spam though.
Hi, Same issue here: For the last few weeks I got plenty of e-mails from the Arch MLs sent to spam. Since I've tweaked Gmail's filters it seems to be OK now. On 16-06-08 14:49:14, Syrone Wong via arch-general wrote:
Hi,
You can add a filter in gmail, set `list:"arch-general@archlinux.org"` to match the exact email address and click "Never send it to Spam".
Enjoy. Best Regards, Syrone Wong
On Wed, Jun 8, 2016 at 2:29 PM, Sri Krishna via arch-general <arch-general@archlinux.org> wrote:
I am also seeing many more emails from this mailing list go to the spam folder in GMail (a few every week).
I'm don't know how to check the DKIM information though.
Srikrishna Sekhar
On 8 June 2016 at 11:34, Henrik Danielsson via arch-general < arch-general@archlinux.org> wrote:
2016-06-08 6:26 GMT+02:00 Eli Schwartz via arch-general < arch-general@archlinux.org>:
dkim header remarks indicate either failed or missing dkim sigs for those messages.
Is anyone else seeing this behavior change?
I didn't look very long but i could only find one message from this list in my Gmail spam folder. (About a missing GPG key 5 days ago.) The DKIM result on that one was just "none", but it got a -1.3 (minimum 5 needed) spam rating so I guess that's why it was shot down. At work we have had problems with regular messages (including those from Google) ending up being tagged spam though.
Am 08.06.2016 um 10:34 schrieb Pierre Neidhardt via arch-general:
On 16-06-08 14:49:14, Syrone Wong via arch-general wrote:
On Wed, Jun 8, 2016 at 2:29 PM, Sri Krishna via arch-general
dear pierre, syrone and Srikrishna, please respect the new community code of conduct and stop top posting [0]. thank you! [0] https://wiki.archlinux.org/index.php/Code_of_conduct#Top_posting
On 08.06.2016 06:26, Eli Schwartz via arch-general wrote:
dkim header remarks indicate either failed or missing dkim sigs for those messages.
That's weird. For me the signatures very just fine. Could you show me the exact error you get (assuming there is one)? Maybe also run the full mail (source, including headers) through `opendkim-testmsg` (part of the opendkim package)? If there is no error, the mail verifies fine. Also missing DKIM signatures are not our fault and FWIW a missing signature should not cause mails to go to spam. An invalid signature also shouldn't unless there is a DMARC policy for that domain that states so. archlinux.org currently doesn't publish a DMARC policy so the default of letting everything through applies. We do change the From address of any mail that uses DMARC though and resign the mail with our key so that signatures for those mails are valid. Since we change the From address, the DMARC policy of the original sender no longer matters. I don't know if gmail provides any information as to why they classify a specific mail as spam, but if they do, please show me. If they do not, please send me all the headers of one mail that has been delivered to spam so I can check them for possible problems. Do the mails that go to spam have anything in common? Do they all contain links or are they all sent by users from a specific domain? For mails where we change the From address, the old address appears is added to CC so check that as well. Florian
On 8 June 2016 at 10:55, Florian Pritz via arch-general < arch-general@archlinux.org> wrote:
On 08.06.2016 06:26, Eli Schwartz via arch-general wrote:
dkim header remarks indicate either failed or missing dkim sigs for those messages.
That's weird. For me the signatures very just fine. Could you show me the exact error you get (assuming there is one)?
Maybe also run the full mail (source, including headers) through `opendkim-testmsg` (part of the opendkim package)? If there is no error, the mail verifies fine.
Also missing DKIM signatures are not our fault and FWIW a missing signature should not cause mails to go to spam. An invalid signature also shouldn't unless there is a DMARC policy for that domain that states so. archlinux.org currently doesn't publish a DMARC policy so the default of letting everything through applies.
We do change the From address of any mail that uses DMARC though and resign the mail with our key so that signatures for those mails are valid. Since we change the From address, the DMARC policy of the original sender no longer matters.
I don't know if gmail provides any information as to why they classify a specific mail as spam, but if they do, please show me. If they do not, please send me all the headers of one mail that has been delivered to spam so I can check them for possible problems.
Gmail imposes more strict checks on email coming in over IPv6, with the rationale that IPv6 enabled machines are more modern and thus should be configured properly for newer verification techniques. Of course, for a mailinglist this does not fly since it does not generate the messages itself. See also [1], specifically "Additional guidelines for IPv6". So... A stupid but possibly pragmatic approach is to use IPv4 when relaying email to gmail. [1] https://support.google.com/mail/answer/81126
also, word checking as well. 2016/06/08 18:18 "Maarten de Vries via arch-general" < arch-general@archlinux.org>:
On 8 June 2016 at 10:55, Florian Pritz via arch-general < arch-general@archlinux.org> wrote:
On 08.06.2016 06:26, Eli Schwartz via arch-general wrote:
dkim header remarks indicate either failed or missing dkim sigs for those messages.
That's weird. For me the signatures very just fine. Could you show me the exact error you get (assuming there is one)?
Maybe also run the full mail (source, including headers) through `opendkim-testmsg` (part of the opendkim package)? If there is no error, the mail verifies fine.
Also missing DKIM signatures are not our fault and FWIW a missing signature should not cause mails to go to spam. An invalid signature also shouldn't unless there is a DMARC policy for that domain that states so. archlinux.org currently doesn't publish a DMARC policy so the default of letting everything through applies.
We do change the From address of any mail that uses DMARC though and resign the mail with our key so that signatures for those mails are valid. Since we change the From address, the DMARC policy of the original sender no longer matters.
I don't know if gmail provides any information as to why they classify a specific mail as spam, but if they do, please show me. If they do not, please send me all the headers of one mail that has been delivered to spam so I can check them for possible problems.
Gmail imposes more strict checks on email coming in over IPv6, with the rationale that IPv6 enabled machines are more modern and thus should be configured properly for newer verification techniques. Of course, for a mailinglist this does not fly since it does not generate the messages itself. See also [1], specifically "Additional guidelines for IPv6".
So... A stupid but possibly pragmatic approach is to use IPv4 when relaying email to gmail.
On 08.06.2016 11:18, Maarten de Vries wrote:
Gmail imposes more strict checks on email coming in over IPv6, with the rationale that IPv6 enabled machines are more modern and thus should be configured properly for newer verification techniques.
Now that is something I totally missed when I read their guidelines, but it certainly sounds like it could be the cause of these problems. I have just switched the server to deliver mails to gmail.com (only that domain, no others) via IPv4. Please tell me if you see any improvement during the next few days.
On 06/08/2016 03:45 PM, Florian Pritz via arch-general wrote:
Now that is something I totally missed when I read their guidelines, but it certainly sounds like it could be the cause of these problems. I have just switched the server to deliver mails to gmail.com (only that domain, no others) via IPv4. Please tell me if you see any improvement during the next few days.
So far so good. A week later and I haven't seen any more Arch ML emails going to spam. Thanks! :) -- Eli Schwartz
On 16.06.2016 03:46, Eli Schwartz via arch-general wrote:
On 06/08/2016 03:45 PM, Florian Pritz via arch-general wrote:
Now that is something I totally missed when I read their guidelines, but it certainly sounds like it could be the cause of these problems. I have just switched the server to deliver mails to gmail.com (only that domain, no others) via IPv4. Please tell me if you see any improvement during the next few days.
So far so good. A week later and I haven't seen any more Arch ML emails going to spam.
Since we didn't have too much traffic on the list in that week, what's the situation like now? Have you seen any more list traffic go to spam or is this fixed now? Florian
For me it's fixed now :) On Mon, Jul 4, 2016 at 7:40 PM, Florian Pritz via arch-general <arch-general@archlinux.org> wrote:
On 16.06.2016 03:46, Eli Schwartz via arch-general wrote:
On 06/08/2016 03:45 PM, Florian Pritz via arch-general wrote:
Now that is something I totally missed when I read their guidelines, but it certainly sounds like it could be the cause of these problems. I have just switched the server to deliver mails to gmail.com (only that domain, no others) via IPv4. Please tell me if you see any improvement during the next few days.
So far so good. A week later and I haven't seen any more Arch ML emails going to spam.
Since we didn't have too much traffic on the list in that week, what's the situation like now? Have you seen any more list traffic go to spam or is this fixed now?
Florian
On 07/04/2016 01:40 PM, Florian Pritz via arch-general wrote:
Since we didn't have too much traffic on the list in that week, what's the situation like now? Have you seen any more list traffic go to spam or is this fixed now?
Still good. :) I have yet to see a single email go to spam since you fixed it. -- Eli Schwartz
On 04.07.2016 19:54, Eli Schwartz via arch-general wrote:
Still good. :) I have yet to see a single email go to spam since you fixed it.
Glad to hear that. Thanks for the feedback! Florian
On 06/08/2016 04:55 AM, Florian Pritz via arch-general wrote:
On 08.06.2016 06:26, Eli Schwartz via arch-general wrote:
dkim header remarks indicate either failed or missing dkim sigs for those messages.
That's weird. For me the signatures very just fine. Could you show me the exact error you get (assuming there is one)?
In one, I see: Authentication-Results: luna.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=dray-be.20150623.gappssmtp.com header.i=@dray-be.20150623.gappssmtp.com header.b=lZvs/tYM In another, I see the line: Authentication-Results: luna.archlinux.org; dkim=none
Maybe also run the full mail (source, including headers) through `opendkim-testmsg` (part of the opendkim package)? If there is no error, the mail verifies fine.
I did try that, by saving the email(s) from Thunderbird. It matches what I'd expect from the Authentication-Results header. [eschwartz@arch ~]$ cat /tmp/Re\:\ \[aur-general\]\ afraiddns-git\ PKGBUILD\:\ review\ and\ criticism\ welcome.eml | opendkim-testmsg opendkim-testmsg: dkim_eom(): Bad signature [eschwartz@arch ~]$ cat /tmp/\[aur-general\]\ Updating\ MATE\ Packages.eml |opendkim-testmsg opendkim-testmsg: dkim_chunk(): No signature
Also missing DKIM signatures are not our fault and FWIW a missing signature should not cause mails to go to spam. An invalid signature also shouldn't unless there is a DMARC policy for that domain that states so. archlinux.org currently doesn't publish a DMARC policy so the default of letting everything through applies.
Correlation does not imply causation. On the other hand, I know very little about mail protocols so I am just floating suggestions and hoping someone who knows more than I can find the answer. :) I did not check every email.
We do change the From address of any mail that uses DMARC though and resign the mail with our key so that signatures for those mails are valid. Since we change the From address, the DMARC policy of the original sender no longer matters.
I noticed that. It is slightly frustrating since most things seem to come from the mailing list rather than the original sender...
I don't know if gmail provides any information as to why they classify a specific mail as spam, but if they do, please show me. If they do not, please send me all the headers of one mail that has been delivered to spam so I can check them for possible problems.
"Why is this message in Spam? It's similar to messages that were detected by our spam filters. Learn more" So, no. :( Pastebins of the two emails I have referenced here: http://pastebin.com/raw/v249MDTC http://pastebin.com/raw/gW4gJtXZ
Do the mails that go to spam have anything in common? Do they all contain links or are they all sent by users from a specific domain? For mails where we change the From address, the old address appears is added to CC so check that as well.
Florian
Well, none of them have had the From address changed (although there are some which do not get changed which were not sent to Spam, so I don't know if that means anything). And some users have sent emails that go to Spam as well as emails that do not go to Spam. Two users whose email went to spam use gmail.com addresses. -- Eli Schwartz
I have been meaning to write here about this issue too, 70% of ML stuff has been going straight to spam. A way to quickly create filters is to click the little drop down arrow at the top of the message, next to the 'to Foo'. Next to the 'mailing list:' title, there's a button that says 'filter messages from this mailing list'.
On 08.06.2016 17:18, Eli Schwartz via arch-general wrote:
In one, I see: Authentication-Results: luna.archlinux.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=dray-be.20150623.gappssmtp.com header.i=@dray-be.20150623.gappssmtp.com header.b=lZvs/tYM
That's weird, but it's difficult to tell what part of the mail changed. I'm also not sure when that header is being generated. Opendkim doesn't seem to verify mails consistently for me and so far I haven't managed to figure out why because debugging features also don't seem to work properly.
In another, I see the line: Authentication-Results: luna.archlinux.org; dkim=none
Well, that's entirely fine. Not everyone supports DKIM.
I noticed that. It is slightly frustrating since most things seem to come from the mailing list rather than the original sender...
That is unfortunate indeed, but there is really no good solutions to this apart from not changing the mail in any way which is also not a good solution for us given people rely on the way our lists behave (subject prefix, reply-to). Mailman has an option to wrap messages into a new message which might fix this problem, but I'd first have to investigate client support and test that feature. As I posted in another part of this thread, I've switched gmail.com delivery to IPv4. Let's see if that changes anything. Florian
Am 08.06.2016 um 06:26 schrieb Eli Schwartz via arch-general:
Recently a (disproportionately) large number of emails from the various Arch Linux mailing lists are getting rerouted to my Spam folder in Gmail.
Is anyone else seeing this behavior change?
I don't use a gmail address for arch mailinglists, but I see two problems: 1. Problems with DNS proxy (e.g. FritzBox) ------------------------------------------ The Tunderbird DKIM Checker plugin says: Ungültig (Kein DKIM Schlüssel im DNS Server gefunden) Invalid (No DKIM key found in DNS server) AFAIK this is because the FritzBox doesn't return the TXT entry: dig luna2._domainkey.archlinux.org TXT gives ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 29290 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0 while asking my local DNS proxy server but with explicitly giving the DNS-server dig @8.8.8.8 luna2._domainkey.archlinux.org TXT the correct record is returned 2. Additional Google DKIM header -------------------------------- Mails from gmail sender addresses have an additional Google DKIM header: X-Google-DKIM-Signature: and maybe this is checked by gmail and considered when calculating the spam level. Regards Uwe
participants (12)
-
Ben Oliver
-
Dragon ryu
-
Eli Schwartz
-
Florian Pritz
-
G. Schlisio
-
Henrik Danielsson
-
Ilya Boka
-
Maarten de Vries
-
Pierre Neidhardt
-
Sri Krishna
-
Syrone Wong
-
Uwe Koloska