All,
we have a group of Arch desktop clients that mount their user homes and group shares through nfsv3 over udp from an OmniOS server.
After today's update (the last one was on 9 Mar) the machine will not nfs-mount, complaining about illegal udp, grpid, int mount options, and the kernel spits out a
NFSV4: Unsupported transport protocol udp
although nothing on this installation anywhere ever has used nfsv4 until now, and /etc/default/autofs explicitly specifies nfsv3. Editing /etc/nfsmount.conf (which was in default state until now) accordingly did not make a difference.
I couldn't find any nfs related Arch news items.
What did I miss?
Cheerio, Hauke
Am 27.04.20 um 16:51 schrieb Hauke Fath:
NFSV4: Unsupported transport protocol udp
Have you tried vers=3 in mount options?
On Mon, 27 Apr 2020 17:23:39 +0200, Markus Schaaf via arch-general wrote:
Am 27.04.20 um 16:51 schrieb Hauke Fath:
NFSV4: Unsupported transport protocol udp
Have you tried vers=3 in mount options?
Thanks for the reply. I'll give some of the configuration.
(1) autofs config is distributed through nis:
% grep '^[/]' /etc/automount/auto.master /home auto.home -udp,nfsvers=3,resvport,nolock,noacl,grpid,retrans=5,rsize=16384,wsize=16384,rw,hard,intr /misc auto.misc -udp,nfsvers=3,resvport,grpid,local_lock=none,retrans=5,rw,hard,intr %
(2) The client autofs setup:
% grep -v '^#' /etc/autofs/autofs.conf [ autofs ] timeout = 300 browse_mode = no [ amd ] dismount_interval = 300 %
This setup has worked for years, and continues to work after rolling back to the 2020-03-09 state. There is no nfsv4 anywhere in sight: The server setup disables it, and wherever I can specify nfsv3 on the client, I have.
From the journal:
[...] Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de automount[567]: attempting to mount entry /home/XXX Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de kernel: RPC: Registered named UNIX socket transport module. Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de kernel: RPC: Registered udp transport module. Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de kernel: RPC: Registered tcp transport module. Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de kernel: RPC: Registered tcp NFSv4.1 backchannel transport module. Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de kernel: FS-Cache: Netfs 'nfs' registered for caching Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de kernel: *** VALIDATE nfs *** Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de kernel: *** VALIDATE nfs4 *** Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de kernel: NFSv4: Unsupported transport protocol udp Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de automount[567]: >> mount.nfs: an incorrect mount option was specified Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de automount[567]: mount(nfs): nfs: mount failure NFSSERVER:/u/homes/XXX on /home/XXX Apr 27 16:27:20 Ankogel.nt.e-technik.tu-darmstadt.de automount[567]: failed to mount /home/XXX [...]
AFAICS, the kernel has no business "validating" an nfsv4 mount, because none has been asked for.
The 2020-03-09 setup:
[...] Apr 27 18:57:11 Ankogel.nt.e-technik.tu-darmstadt.de automount[546]: attempting to mount entry /home/XXX Apr 27 18:57:11 Ankogel.nt.e-technik.tu-darmstadt.de kernel: RPC: Registered named UNIX socket transport module. Apr 27 18:57:11 Ankogel.nt.e-technik.tu-darmstadt.de kernel: RPC: Registered udp transport module. Apr 27 18:57:11 Ankogel.nt.e-technik.tu-darmstadt.de kernel: RPC: Registered tcp transport module. Apr 27 18:57:11 Ankogel.nt.e-technik.tu-darmstadt.de kernel: RPC: Registered tcp NFSv4.1 backchannel transport module. Apr 27 18:57:11 Ankogel.nt.e-technik.tu-darmstadt.de kernel: FS-Cache: Netfs 'nfs' registered for caching Apr 27 18:57:11 Ankogel.nt.e-technik.tu-darmstadt.de automount[546]: mounted /home/XXX [...]
From what I can see, either I missed some obscure new configuration that overrides the automounter's explicit mount options. Or, I hit a kernel bug.
Cheerio, Hauke
On Mon, 27 Apr 2020 19:19:21 +0200, Hauke Fath wrote:
I'll give some of the configuration.
FTR, the borken system's kernel is
Linux version 5.6.7-arch1-1 (linux@archlinux) (gcc version 9.3.0 (Arch Linux 9.3.0-1)) #1 SMP PREEMPT Thu, 23 Apr 2020 09:13:56 +0000
while the working system has
Linux version 5.5.8-arch1-1 (linux@archlinux) (gcc version 9.2.1 20200130 (Arch Linux 9.2.1+20200130-2)) #1 SMP PREEMPT Fri, 06 Mar 2020 00:57:33 +0000
Cheerio, Hauke
On 27/04/20, Hauke Fath wrote:
On Mon, 27 Apr 2020 19:19:21 +0200, Hauke Fath wrote:
I'll give some of the configuration.
FTR, the borken system's kernel is
Linux version 5.6.7-arch1-1 (linux@archlinux) (gcc version 9.3.0 (Arch Linux 9.3.0-1)) #1 SMP PREEMPT Thu, 23 Apr 2020 09:13:56 +0000
while the working system has
Linux version 5.5.8-arch1-1 (linux@archlinux) (gcc version 9.2.1 20200130 (Arch Linux 9.2.1+20200130-2)) #1 SMP PREEMPT Fri, 06 Mar 2020 00:57:33 +0000
Cheerio, Hauke
Hi Hauke,
It's a kernel configuration which is introduced with 5.6 kernel. In my kernel which is 5.6.7-arch1-1
$ zcat /proc/config.gz | grep NFS_DISABLE CONFIG_NFS_DISABLE_UDP_SUPPORT=y
The change happened with commit https://git.archlinux.org/svntogit/packages.git/commit/trunk/config?h=packag...
Cheers,
Am 27.04.20 um 20:00 schrieb Leonidas Spyropoulos via arch-general:
It's a kernel configuration which is introduced with 5.6 kernel. In my CONFIG_NFS_DISABLE_UDP_SUPPORT=y
Wow! I'd say udp is used a lot with nfsvers=3. That will break many nfs3 deployments.
On Mon, 27 Apr 2020 19:00:32 +0100, Leonidas Spyropoulos via arch-general wrote:
Thanks for the note!
It's a kernel configuration which is introduced with 5.6 kernel. In my kernel which is 5.6.7-arch1-1
$ zcat /proc/config.gz | grep NFS_DISABLE CONFIG_NFS_DISABLE_UDP_SUPPORT=y
Meaning this is an upstream change? Might have been worth an entry on www.archlinux.org/news/...
So I guess the short-term fix is to switch to an LTS kernel, and mid-term I will have to get into building my own kernels?
Cheerio, Hauke
On Mon, 27 Apr 2020 19:00:32 +0100, Leonidas Spyropoulos via arch-general wrote:
$ zcat /proc/config.gz | grep NFS_DISABLE CONFIG_NFS_DISABLE_UDP_SUPPORT=y
The change happened with commit
Re-reading, this is an Arch decision -- what is the rationale? Can anybody point me to a related discussion? And why does the kernel end up talking about nfsv4, as if it hit https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6da1a034362f86e157e251e65394f0b6570e3e3a?
Cheerio, Hauke
On 27/04/20, Hauke Fath wrote:
Re-reading, this is an Arch decision -- what is the rationale? Can anybody point me to a related discussion?
It's not, it just defaults to "Y", see patch in https://github.com/archlinux/linux/commit/b24ee6c64ca785739b3ef8d95fd6becaad...
There's also a bit of explanation if it's useful
Cheers,
On Mon, 27 Apr 2020 19:25:59 +0100, Leonidas Spyropoulos via arch-general wrote:
On 27/04/20, Hauke Fath wrote:
Re-reading, this is an Arch decision -- what is the rationale? Can anybody point me to a related discussion?
It's not, it just defaults to "Y",
-- which requires building your own kernel to change, right?
see patch in
https://github.com/archlinux/linux/commit/b24ee6c64ca785739b3ef8d95fd6becaad...
There's also a bit of explanation if it's useful
Yes, it is - actually, nfs(5) has a much longer explanation, as well as a viable workaround: Shorten the udp fragment reassembly timeout on fast networks.
On Mon, 27 Apr 2020 20:12:17 +0200, Markus Schaaf via arch-general wrote:
It's a kernel configuration which is introduced with 5.6 kernel. In my CONFIG_NFS_DISABLE_UDP_SUPPORT=y
Wow! I'd say udp is used a lot with nfsvers=3. That will break many nfs3 deployments.
This.
I (and my users) certainly would appreciate if the decision could be reconsidered.
Cheerio, Hauke
On Mon, 27 Apr 2020 19:25:59 +0100, Leonidas Spyropoulos via arch-general wrote:
On 27/04/20, Hauke Fath wrote:
Re-reading, this is an Arch decision -- what is the rationale? Can anybody point me to a related discussion?
It's not, it just defaults to "Y", see patch in
https://github.com/archlinux/linux/commit/b24ee6c64ca785739b3ef8d95fd6becaad...
Defaulting to "disabled" strikes me as heavy-handed.
From my experience, of running a few dozen nfs clients over more than fifteen years, at network speeds from 100 MBit clients / 1 GBit server to 1 GBit clients / 10 GBit server, the real-world relevance of ip fragment counter overruns is debatable. From a general background of transmission errors [1], the effect does not seem to stand out. FTR, we are running nfs through a filtering router, and found udp transport to be much more robust in the face of router outages.
So, to summarize:
The kernel nfs developers have deprecated nfs over udp in late 2019, AFAICS without much debate, and without any announcement outside the developers' mailing-list [2]. So far, so unspectacular.
The interesting question is whether Linux distributions (Arch) should import this kernel change uncritically, and dump it on their unsuspecting users, without so much as a warning.
Given nfsv3 defaults to tcp, the "protect the innocent / think of the children" argument does not fly: When the admin wants to use nfs over udp (whether she chooses, or has to), she has to actively configure the mount, and take responsibility. In this light, disabling nfs over udp in the kernel seems patronizing to professional admins.
My question is: Where should I take the issue for further discussion? Is this mailing-list the right place? A forum group? Should I file a bug report?
Cheerio, Hauke
[1] http://conferences.sigcomm.org/sigcomm/2000/conf/paper/sigcomm2000-9-1.pdf [2] https://www.spinics.net/lists/linux-nfs/msg75432.html
On Wed, 29 Apr 2020 at 12:39, Hauke Fath hf@spg.tu-darmstadt.de wrote:
On Mon, 27 Apr 2020 19:25:59 +0100, Leonidas Spyropoulos via
The interesting question is whether Linux distributions (Arch) should import this kernel change uncritically, and dump it on their unsuspecting users, without so much as a warning.
I do not think it is incumbent upon Arch to validate upstream changes to packages. We're bleeding edge for a reason.
While it is relatively trivial to compile your own kernel with those options enabled using Arch's build system, I think you'd better talk to the actual people that made the change upstream.
Am 29.04.20 um 13:44 schrieb Andy Pieters:
While it is relatively trivial to compile your own kernel with those options enabled using Arch's build system, I think you'd better talk to the actual people that made the change upstream.
This change might have slipped through unnoticed. It seems idiotic: _Add_ code to the kernel, to break users systems, with no benefit whatsoever. It's clearly against Linus' agenda to not break userland. But to tell the users to discuss this upstream is bad advice. This is a situation where a distribution should take corrective action by reverting this configuration. This would add value and remove some useless code from the kernel.
BR
On 29/04/20, Markus Schaaf via arch-general wrote:
Am 29.04.20 um 13:44 schrieb Andy Pieters:
While it is relatively trivial to compile your own kernel with those options enabled using Arch's build system, I think you'd better talk to the actual people that made the change upstream.
This change might have slipped through unnoticed. It seems idiotic: _Add_ code to the kernel, to break users systems, with no benefit whatsoever. It's clearly against Linus' agenda to not break userland. But to tell the users to discuss this upstream is bad advice. This is a situation where a distribution should take corrective action by reverting this configuration. This would add value and remove some useless code from the kernel.
BR
Might worth reaching out to linux-nfs@vger.kernel.org mailing list instead of Archlinux as this is upstream default behaviour.
Cheers,
On Wed, 29 Apr 2020 14:40:26 +0100, Leonidas Spyropoulos via arch-general wrote:
Might worth reaching out to linux-nfs@vger.kernel.org mailing list instead of Archlinux as this is upstream default behaviour.
Personally, I have no standing here nor there, so I won't follow your suggestion. The linux-nfs@ members would probably reply that downstream distributions are free to ship nfs over udp enabled kernels if they choose so...
Response to the proposal of deprecating UDP transport https://www.spinics.net/lists/linux-nfs/msg74889.html was positive, as much as there was.
Cheerio, Hauke
Personally, I have no standing here nor there, so I won't follow your suggestion. The linux-nfs@ members would probably reply that downstream distributions are free to ship nfs over udp enabled kernels if they choose so...
Response to the proposal of deprecating UDP transport https://www.spinics.net/lists/linux-nfs/msg74889.html was positive, as much as there was.
Cheerio, Hauke
As a workaround you could set up a local repo and ship your own kernel, that you rebuild (semi-?)automatically from the ABS. That way you don't depend on somebody else to fix this, but it of course puts the maintenance burden on you.
Georg
Hi,
But to tell the users to discuss this upstream is bad advice. This is a situation where a distribution should take corrective action by reverting this configuration. This would add value and remove some useless code from the kernel.
No. If you want a distribution whose maintainers think it is a good think to "correct" upstream decisions, you should use not Arch.
I'm using Ubuntu at work and often I stumble upon bugs which are fixed already upstream (and so in Arch), as in Ubuntu/Debian-land these things are discussed and maybe a fix could break other things etc, while something is still unusable.
Regards Bjoern
arch-general@lists.archlinux.org