[arch-general] NFS server broke after reboot - *urgent* need help - Arch-general

[arch-general] NFS server broke after reboot - urgent need help

older
[arch-general] Fwd: Kernel panic...

Genes Lists

31 Jan 2015 31 Jan '15

9:14 p.m.

I have a problem where nfs failed. 2 units fail to start rpc-statd.service nfs-server.service. ------------------------------------- rpc-statd: systemctl status rpc-statd rpc.statd[736]: Version 1.3.2 starting rpc.statd[736]: Flags: TI-RPC rpc.statd[736]: Running as root. chown /var/lib/nfs to choose different user rpc.statd[736]: failed to create RPC listeners, exiting systemd[1]: rpc-statd.service: control process exited, code=exited status=1 systemd[1]: Failed to start NFS status monitor for NFSv2/3 locking.. systemd[1]: Unit rpc-statd.service entered failed state. systemd[1]: rpc-statd.service failed. ------------------------------------- I am unable to start this by hand either - continues to fail same way. I had seen this once a month or so back - but was able to start it by hand after machine was up. ------------------------------------- nfs-server: systemctl-status nfs-server ● nfs-server.service - NFS server and services Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Sat 2015-01-31 16:05:32 EST; 5min ago Process: 743 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=1/FAILURE) Process: 741 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS) Main PID: 743 (code=exited, status=1/FAILURE) rpc.nfsd[743]: rpc.nfsd: writing fd to kernel failed: errno 111 (Connection refused) rpc.nfsd[743]: rpc.nfsd: unable to set any sockets for nfsd systemd[1]: nfs-server.service: main process exited, code=exited, status=1/FAILURE systemd[1]: Failed to start NFS server and services. systemd[1]: Unit nfs-server.service entered failed state. systemd[1]: nfs-server.service failed. ------------------------------------- Machine is fully updated from testing repo. I did try kernel 3.19.rc6 but it does not help. nfs-utils 1.3.2-1 linux 3.18.5-1 rpcbind 0.2.2-1 Really appreciate some help - this is my main nfs server and I can no longer get it working ... Thanks for any advice. gene

Show replies by date

Bardur Arantsson

31 Jan 31 Jan

9:28 p.m.

New subject: [arch-general] NFS server broke after reboot - *urgent* need help

On 01/31/2015 10:14 PM, Genes Lists wrote:

...

I have a problem where nfs failed. 2 units fail to start

rpc-statd.service nfs-server.service.

------------------------------------- rpc-statd: systemctl status rpc-statd

rpc.statd[736]: Version 1.3.2 starting rpc.statd[736]: Flags: TI-RPC rpc.statd[736]: Running as root. chown /var/lib/nfs to choose different user rpc.statd[736]: failed to create RPC listeners, exiting systemd[1]: rpc-statd.service: control process exited, code=exited status=1 systemd[1]: Failed to start NFS status monitor for NFSv2/3 locking.. systemd[1]: Unit rpc-statd.service entered failed state. systemd[1]: rpc-statd.service failed.

------------------------------------- I am unable to start this by hand either - continues to fail same way. I had seen this once a month or so back - but was able to start it by hand after machine was up.

------------------------------------- nfs-server:

systemctl-status nfs-server

● nfs-server.service - NFS server and services Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Sat 2015-01-31 16:05:32 EST; 5min ago Process: 743 ExecStart=/usr/sbin/rpc.nfsd $RPCNFSDARGS (code=exited, status=1/FAILURE) Process: 741 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS) Main PID: 743 (code=exited, status=1/FAILURE)

rpc.nfsd[743]: rpc.nfsd: writing fd to kernel failed: errno 111 (Connection refused) rpc.nfsd[743]: rpc.nfsd: unable to set any sockets for nfsd systemd[1]: nfs-server.service: main process exited, code=exited, status=1/FAILURE systemd[1]: Failed to start NFS server and services. systemd[1]: Unit nfs-server.service entered failed state. systemd[1]: nfs-server.service failed.

-------------------------------------

Machine is fully updated from testing repo. I did try kernel 3.19.rc6 but it does not help.

nfs-utils 1.3.2-1 linux 3.18.5-1 rpcbind 0.2.2-1

Really appreciate some help - this is my main nfs server and I can no longer get it working ...

I'm unlikely to be able to help, but it might be a good idea to try journalctl -u nfs-server journalctl -u rpc.statd etc. just to see if there's some output you've missed.

Genes Lists

9:32 p.m.

New subject: [arch-general] NFS server broke after reboot - *urgent* need help

On 01/31/2015 04:28 PM, Bardur Arantsson wrote:

...

On 01/31/2015 10:14 PM, Genes Lists wrote:

This was original thread: https://lists.archlinux.org/pipermail/arch-general/2014-June/036617.html So I tried the same commands - specifically: systemctl restart proc-fs-nfsd.mount systemctl restart rpcbind systemctl restart nfs-mountd.service systemctl restart rpc-statd.service systemctl restart nfs-idmapd.service systemctl restart rpc-svcgssd.service systemctl restart rpc-statd-notify.service systemctl restart nfs-mountd systemctl restart rpc-gssd.service rpc-svcgssd.service rpc-statd still does not start but now .. it failes with rpc-statd.service start operation timed out. Terminating. After 5 mins i tried nfs-server which now starts ... rpc-statd is still not running. And the server is once again serving NFS. This is a real mess ... eeeek!! At last NFS server is up now ... but yikes I need help so this starts by itself again. thanks!

Tomasz Kramkowski

9:50 p.m.

On 31/01/15 21:32, Genes Lists wrote:

...

On 01/31/2015 04:28 PM, Bardur Arantsson wrote:

...
On 01/31/2015 10:14 PM, Genes Lists wrote:

This was original thread: https://lists.archlinux.org/pipermail/arch-general/2014-June/036617.html

So I tried the same commands - specifically:

systemctl restart proc-fs-nfsd.mount

systemctl restart rpcbind systemctl restart nfs-mountd.service systemctl restart rpc-statd.service systemctl restart nfs-idmapd.service systemctl restart rpc-svcgssd.service systemctl restart rpc-statd-notify.service systemctl restart nfs-mountd systemctl restart rpc-gssd.service rpc-svcgssd.service

rpc-statd still does not start but now .. it failes with rpc-statd.service start operation timed out. Terminating.

After 5 mins i tried nfs-server which now starts ... rpc-statd is still not running.

And the server is once again serving NFS.

This is a real mess ... eeeek!!

At last NFS server is up now ... but yikes I need help so this starts by itself again.

thanks!

I've been having a few rather strange problems recently with services requiring an upped network interface starting before the interface is up, this is using systemd-networkd-wait-online.service. In any case, it is possible you are having a similar problem, are you sure the interfaces nfs-server is set to listen on are up? -- Tomasz Kramkowski E-Mail: tk@the-tk.com PGP: 6FCE87503AAF42AB3BF4 94FE40B037BA0A5B8680

Mike Cloaked

10:02 p.m.

New subject: [arch-general] NFS server broke after reboot - *urgent* need help

On Sat, Jan 31, 2015 at 9:50 PM, Tomasz Kramkowski wrote:

...

I've been having a few rather strange problems recently with services requiring an upped network interface starting before the interface is up, this is using systemd-networkd-wait-online.service.

In any case, it is possible you are having a similar problem, are you sure the interfaces nfs-server is set to listen on are up?

This is now reported at https://bugs.archlinux.org/task/43647 so hopefully some enlightenment will appear there to help too. -- mike c

Tomasz Kramkowski

10:08 p.m.

On 31/01/15 22:02, Mike Cloaked wrote:

...

On Sat, Jan 31, 2015 at 9:50 PM, Tomasz Kramkowski wrote:

...
I've been having a few rather strange problems recently with services requiring an upped network interface starting before the interface is up, this is using systemd-networkd-wait-online.service.

In any case, it is possible you are having a similar problem, are you sure the interfaces nfs-server is set to listen on are up?

This is now reported at https://bugs.archlinux.org/task/43647 so hopefully some enlightenment will appear there to help too.

Could it be assumed then that this problem happened after an update and is happening to multiple people? I'm getting worried about updating now, very few things rely on my nfs server but it's still relatively important it stays running in case I do need to use it. I guess I'll just wait to see what comes of this bug report. -- Tomasz Kramkowski E-Mail: tk@the-tk.com PGP: 6FCE87503AAF42AB3BF4 94FE40B037BA0A5B8680

Genes Lists

10:17 p.m.

New subject: [arch-general] NFS server broke after reboot - *urgent* need help

On 01/31/2015 04:50 PM, Tomasz Kramkowski wrote:

...

In any case, it is possible you are having a similar problem, are you sure the interfaces nfs-server is set to listen on are up?

Yes server is up - i administer via ssh remotely :)

Bardur Arantsson

10:09 p.m.

New subject: [arch-general] NFS server broke after reboot - *urgent* need help

On 01/31/2015 10:32 PM, Genes Lists wrote:

...

On 01/31/2015 04:28 PM, Bardur Arantsson wrote:

...
On 01/31/2015 10:14 PM, Genes Lists wrote:

This was original thread: https://lists.archlinux.org/pipermail/arch-general/2014-June/036617.html

First, in case it's relevant: DON'T PANIC! That's the worst you can possibly do in a situation like this. Secondly, I mentioned "journalctl -u", not "systemctl". There may be relevant stuff in there that isn't mentioned by "systemctl status" (due to cutoff). Good luck,

Genes Lists

10:19 p.m.

New subject: [arch-general] NFS server broke after reboot - *urgent* need help

On 01/31/2015 05:09 PM, Bardur Arantsson wrote:

...

First, in case it's relevant: DON'T PANIC!

Ha ha good advice ... :)

...

Secondly, I mentioned "journalctl -u", not "systemctl". There may be relevant stuff in there that isn't mentioned by "systemctl status" (due to cutoff).

yes I did look thru the logs - and posted one relevant one (nfsd) in the bug report. The rest are already covered by the systemctl status one and are not additive. Thanks for suggestion.

Tobias Powalowski

1 Feb 1 Feb

9:13 a.m.

Am 31.01.2015 um 23:19 schrieb Genes Lists:

...

On 01/31/2015 05:09 PM, Bardur Arantsson wrote:

...
First, in case it's relevant: DON'T PANIC!

Ha ha good advice ... :)

...
Secondly, I mentioned "journalctl -u", not "systemctl". There may be relevant stuff in there that isn't mentioned by "systemctl status" (due to cutoff).

yes I did look thru the logs - and posted one relevant one (nfsd) in the bug report. The rest are already covered by the systemctl status one and are not additive.

Thanks for suggestion. Could it be, that all you suffer from issues are using NFS3 and not NFS4.x My simple setup just works fine with NFS4 without any hiccups.

greetings tpowa -- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) http://www.archlinux.org tpowa@archlinux.org

Gene Cohler

2:25 p.m.

New subject: [arch-general] NFS server broke after reboot - *urgent* need help

Only use V4 here. I believe the references to older versions come from the rpc-statd systemd unit file. I'm away from computers at the moment but will double check. Also I believe it's nfs-server unit file that asks for rpc-statd. It's been working fine since system was installed in December 2013 other than the occasional quirk such as mentioned in June 2014 bug report. I forgot to mention that the new nfs-utils came in after the first reboot to pick up 3.18.5. However installing that did not by itself fix anything either. Thank you for your help. On February 1, 2015 4:13:40 AM EST, Tobias Powalowski wrote:

...

Am 31.01.2015 um 23:19 schrieb Genes Lists:

...
On 01/31/2015 05:09 PM, Bardur Arantsson wrote:

...
First, in case it's relevant: DON'T PANIC!

Ha ha good advice ... :)

...
Secondly, I mentioned "journalctl -u", not "systemctl". There may be relevant stuff in there that isn't mentioned by "systemctl status"

(due

...
to cutoff).

yes I did look thru the logs - and posted one relevant one (nfsd) in the bug report. The rest are already covered by the systemctl status one and are not additive.

Thanks for suggestion. Could it be, that all you suffer from issues are using NFS3 and not NFS4.x My simple setup just works fine with NFS4 without any hiccups.

greetings tpowa

-- Tobias Powalowski Archlinux Developer & Package Maintainer (tpowa) http://www.archlinux.org tpowa@archlinux.org

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

Gene Cohler

2:28 p.m.

New subject: [arch-general] NFS server broke after reboot - *urgent* need help

Sorry for top post. Shouldn't reply from phone. Gene -- Sent from my Android device. Please excuse my brevity.

Mike Cloaked

4:18 p.m.

New subject: [arch-general] NFS server broke after reboot - *urgent* need help

On Sun, Feb 1, 2015 at 2:25 PM, Gene Cohler wrote:

...

Only use V4 here. I believe the references to older versions come from the rpc-statd systemd unit file. I'm away from computers at the moment but will double check. Also I believe it's nfs-server unit file that asks for rpc-statd.

It's been working fine since system was installed in December 2013 other than the occasional quirk such as mentioned in June 2014 bug report.

I forgot to mention that the new nfs-utils came in after the first reboot to pick up 3.18.5. However installing that did not by itself fix anything either.

Thank you for your help.

Is this Fedora bug related? https://bugzilla.redhat.com/show_bug.cgi?id=1183992 -- mike c

3371

Age (days ago)

3372

Last active (days ago)

List overview

Download

12 comments

6 participants

participants (6)

Bardur Arantsson
Gene Cohler
Genes Lists
Mike Cloaked
Tobias Powalowski
Tomasz Kramkowski

[arch-general] NFS server broke after reboot - *urgent* need help

tags

participants (6)

[arch-general] NFS server broke after reboot - urgent need help