[arch-general] ipv6 only half working
I'm having some weird problems with ipv6 on my box. My ISP is set up for ipv6, my router and server both seem to have valid ipv6 addresses, and ipv6 connectivity seems to (nominally) be working: [darose@darsys12 ~]$ ping ipv6.google.com PING ipv6.google.com(lga25s63-in-x0e.1e100.net (2607:f8b0:4006:81b::200e)) 56 data bytes 64 bytes from lga25s63-in-x0e.1e100.net (2607:f8b0:4006:81b::200e): icmp_seq=1 ttl=55 time=13.1 ms 64 bytes from lga25s63-in-x0e.1e100.net (2607:f8b0:4006:81b::200e): icmp_seq=2 ttl=55 time=11.0 ms ^C [darose@darsys12 ~]$ curl -I ipv6.google.com HTTP/1.1 200 OK Date: Mon, 23 Jul 2018 15:35:06 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info." Server: gws X-XSS-Protection: 1; mode=block X-Frame-Options: SAMEORIGIN Set-Cookie: 1P_JAR=2018-07-23-15; expires=Wed, 22-Aug-2018 15:35:06 GMT; path=/; domain=.google.com Set-Cookie: NID=135=SV0kZxYO7fTnUYm0ZJFnRx-d17kNOyTyHgPzv-xKmj_Q-LXYz_LuiBsAGHesREIbtwbKZAKYeqU8fh0saXfnylDGsBbAwCQDAKeG9iT9PwmBc0IBntqO0xkklns-YThY; expires=Tue, 22-Jan-2019 15:35:06 GMT; path=/; domain=.google.com; HttpOnly Transfer-Encoding: chunked Accept-Ranges: none Vary: Accept-Encoding However, certain server utilities - most notably freshclam (clamav) and the w3m command line web browser - are having problems successfully making connections using ipv6. For example: w3m via ipv4 works: [darose@darsys12 ~]$ w3m -4 -dump google.com | grep -i google Google [Google Search][I'm Feeling Lucky] tools Advertising ProgramsBusiness Solutions+GoogleAbout Google but w3m via ipv6 doesn't: [darose@darsys12 ~]$ w3m -6 -dump ipv6.google.com ... hangs forever ... Even weirder: this works: [darose@darsys12 ~]$ w3m -4 -dump_head google.com HTTP/1.0 200 OK Date: Mon, 23 Jul 2018 15:21:52 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 Content-Encoding: gzip Server: gws Content-Length: 4675 X-XSS-Protection: 1; mode=block X-Frame-Options: SAMEORIGIN Set-Cookie: 1P_JAR=2018-07-23-15; expires=Wed, 22-Aug-2018 15:21:52 GMT; path=/; domain=.google.com ... but this doesn't: [darose@darsys12 ~]$ w3m -6 -dump_head ipv6.google.com ... hangs ... until I hit control C ... then succeeds after the control-c (wtf?!?!?) ^CHTTP/1.0 200 OK Date: Mon, 23 Jul 2018 15:39:44 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 Content-Encoding: gzip Server: gws Content-Length: 4680 X-XSS-Protection: 1; mode=block X-Frame-Options: SAMEORIGIN Set-Cookie: 1P_JAR=2018-07-23-15; expires=Wed, 22-Aug-2018 15:39:44 GMT; path=/; domain=.google.com And freshclam fails as well: [darose@darsys12 ~]$ sudo freshclam ClamAV update process started at Mon Jul 23 11:41:13 2018 WARNING: Your ClamAV installation is OUTDATED! WARNING: Local version: 0.100.0 Recommended version: 0.100.1 DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav main.cld is up to date (version: 58, sigs: 4566249, f-level: 60, builder: sigmgr) nonblock_recv: recv timing out (30 secs) WARNING: getfile: Download interrupted: Operation now in progress (IP: 2400:cb00:2048:1::6810:bb8a) WARNING: getpatch: Can't download daily-24740.cdiff from database.clamav.net Trying host database.clamav.net (2400:cb00:2048:1::6810:b98a)... nonblock_recv: recv timing out (30 secs) WARNING: getfile: Download interrupted: Operation now in progress (IP: 2400:cb00:2048:1::6810:b98a) WARNING: getpatch: Can't download daily-24740.cdiff from database.clamav.net Downloading daily-24740.cdiff [100%] ... I'm really stumped as to how ipv6 could partially fail like this only on specific apps. Seems like it should either completely work or completely fail. Anyone have any idea what might be happening here and/or how to fix? Thanks, DR
Em julho 23, 2018 12:46 David Rosenstrauch escreveu:
I'm really stumped as to how ipv6 could partially fail like this only on specific apps. Seems like it should either completely work or completely fail.
Anyone have any idea what might be happening here and/or how to fix?
Hi David, If ping works, but bigger packet things don't, then you might have some PMTU issue that is only affecting IPv6. You can confirm this with a tcpdump trace quite easily. Look at the packet sizes. Both things you've mentioned (opening a web page and updating antivirus) are http, and it does usually use the maximum transmission unit. Regards, Giancarlo Razzolini
On 07/23/2018 12:42 PM, Giancarlo Razzolini via arch-general wrote:
Em julho 23, 2018 12:46 David Rosenstrauch escreveu:
I'm really stumped as to how ipv6 could partially fail like this only on specific apps. Seems like it should either completely work or completely fail.
Anyone have any idea what might be happening here and/or how to fix?
Hi David,
If ping works, but bigger packet things don't, then you might have some PMTU issue that is only affecting IPv6. You can confirm this with a tcpdump trace quite easily. Look at the packet sizes. Both things you've mentioned (opening a web page and updating antivirus) are http, and it does usually use the maximum transmission unit.
Regards, Giancarlo Razzolini
Makes some sense. I'll take a look on my router and see if perhaps I'm inadvertently blocking some ICMP packets. That said, one thing doesn't quite seem to fit with this explanation: Why would "curl -6 ipv6.google.com" while "w3m -6 -dump ipv6.google.com" fails? Thanks, DR
Em julho 23, 2018 14:14 David Rosenstrauch escreveu:
Makes some sense. I'll take a look on my router and see if perhaps I'm inadvertently blocking some ICMP packets.
That said, one thing doesn't quite seem to fit with this explanation:
Why would "curl -6 ipv6.google.com" while "w3m -6 -dump ipv6.google.com" fails?
I think what you have experience with the CTRL+C thing you saw is regarding how each program treats their buffers. Sometimes programs don't flush and when you send it SIGTERM they flush the buffer. Secondly, you might be hitting different routes at times, although that would point to an even larger issue at your ISP. Regards, Giancarlo Razzolini
Hi David,
I'm having some weird problems with ipv6 on my box.
It'll never catch on. :-)
but w3m via ipv6 doesn't: [darose@darsys12 ~]$ w3m -6 -dump ipv6.google.com ... hangs forever ...
I suspect the difference between how well your network/ISP is copying with TCP and UDP; HTTP connections and DNS queries. curl(1) and w3m(1) might achieve the same end result differently. You can try strace -fe %network w3m -6 -dump ipv6.google.com and see how far it gets. With older strace(1), drop the `%'. Compare with other commands that work or don't work. As for Giancarlo's suggestion of MTU, tracepath(8) is handy for checking that. It has undocumented `-4' and `-6' options. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy
On 2018-07-23 12:42 pm, Giancarlo Razzolini via arch-general wrote:
Em julho 23, 2018 12:46 David Rosenstrauch escreveu:
I'm really stumped as to how ipv6 could partially fail like this only on specific apps. Seems like it should either completely work or completely fail.
Anyone have any idea what might be happening here and/or how to fix?
Hi David,
If ping works, but bigger packet things don't, then you might have some PMTU issue that is only affecting IPv6. You can confirm this with a tcpdump trace quite easily. Look at the packet sizes. Both things you've mentioned (opening a web page and updating antivirus) are http, and it does usually use the maximum transmission unit.
Regards, Giancarlo Razzolini
I finally had some time to dig into this issue, and ran wireshark on a "freshclam" download, although I'm not sure it's helped me get any closer to figuring out what's going on. To the best of my knowledge (I'm definitely not an expert in networking) it looks like I'm having some packets dropped - but again I have no idea why (or where)? I put up a wireshark screenshot at http://darose.net/packets-dropped.png which shows the download humming along nicely, when all of a sudden it looks like the other server seems to jump way ahead in the sequence numbering, and my server keeps re-sending duplicate acks based on where it thinks the correct sequence number is. In addition, "ifconfig" shows 17 Rx dropped packets on eth0. (Possibly coincidental, possibly not.) Any idea what I might look for / where I might look from here to figure out what's causing the issue? My server is running (an up to date) Arch installation, and is behind a Netgear WNDR3700 router. Any suggestions appreciated! Thanks, DR
Em agosto 2, 2018 22:53 David Rosenstrauch escreveu:
Any idea what I might look for / where I might look from here to figure out what's causing the issue? My server is running (an up to date) Arch installation, and is behind a Netgear WNDR3700 router. Any suggestions appreciated!
It's hard to know for sure without the full capture file. But those errors *point* in the direction of MTU issues. Try setting your MTU at your interface level to a value lower than 1500. Try with 1492 first. See if that fixes the situation. You can also try ping to know what is the maximum MTU and also segment size. Regards, Giancarlo Razzolini
Hi David,
(I'm definitely not an expert in networking)
/me too.
it looks like I'm having some packets dropped - but again I have no idea why (or where)?
You have some `Len=1220', and the others are multiples. This suggests your network card is merging packets together before they're passed up to Wireshark. That may hinder the ability to deduce what's happening `on the wire', I don't know. http://rtodto.net/generic_segmentation_offload_and_wireshark/ P.S. That's gritty font rendering. :-) I've gone back to xorg-fonts-misc package's fonts recently, and I'm really liking the lack of anti-aliasing sharpness. -- Cheers, Ralph. https://plus.google.com/+RalphCorderoy
On 08/02/2018 10:01 PM, Giancarlo Razzolini via arch-general wrote:
Em agosto 2, 2018 22:53 David Rosenstrauch escreveu:
Any idea what I might look for / where I might look from here to figure out what's causing the issue? My server is running (an up to date) Arch installation, and is behind a Netgear WNDR3700 router. Any suggestions appreciated!
It's hard to know for sure without the full capture file. But those errors *point* in the direction of MTU issues. Try setting your MTU at your interface level to a value lower than 1500. Try with 1492 first. See if that fixes the situation.
You can also try ping to know what is the maximum MTU and also segment size.
Regards, Giancarlo Razzolini
Thanks much for the suggestions. After researching this further, it appears to have been an ipv6 issue with my router. I upgraded the router to dd-wrt over the weekend, and the issue now appears to be resolved. Thanks again for the help, and sorry for taking up list bandwidth. DR
participants (3)
-
David Rosenstrauch
-
Giancarlo Razzolini
-
Ralph Corderoy