cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1237
Views
9
Helpful
16
Replies

Verifying NFS

spellluck
Level 1
Level 1

Hello all,

Hopefully someone can help me out as I dont know linux at all to verify some information.

I have MARS configured to backup to a NFS server. I recently received a notification from MARS stating that one of the backups failed.

Because I do not have access to the NFS server (Managed Services is why), what other ways can I verify that the NFS backup is succeeding? Whenever I attempt to use the GUI to pull raw messages from the NFS server it essentially times out.

Anything on the cli available? Any and all help/ideas are appreciated.

MARS is running 4.2.6 ( 2458 ). Yea, old, but customer absolutely refuses to give us the time of day to upgrade it.

16 Replies 16

Farrukh Haroon
VIP Alumni
VIP Alumni

You can setup a second NFS server temporarity to see if that works. You can even run NFS on windows with the help of some software/add-on (as per MARS docs).

Alternatively you could go to MARS >> Admin >> System Maintenance >> Set Runtime Logging Levels

pnarchiver = DEBUG

And see if you get any more meaningful logs.

Regards

Farrukh

FYI,

It isn't until a later release that Cisco actually includes

A) more meaningful logs for NFS

B) Event corrolation for NFS failures for the MARS appliance so it will notify more then just the account associated with "pnadmin"

"MARS is running 4.2.6 ( 2458 ). Yea, old, but customer absolutely refuses to give us the time of day to upgrade it."

well, they still get to check the box when the auditor visits;-) sad isn't it. The NFS logging enhancements is one of MANY reasons to make sure MARS is kept up to date. FWIW, we implemented our own NFS monitoring years ago because it would frequently break with no alerts being generated.

Mind if I ask how you monitor the NFS?

Each day has it's own directory in the archive.

We just check the modify time on the directory. The NFS archiving has gotten MUCH better, but back in the day this worked pretty good to let us know when it was broken.

Ah. Makes sense, and was my first instinct. Unfortunately, we're a managed service provider. For liability issues, we do not have access to view the server.

Farrukh Haroon
VIP Alumni
VIP Alumni

Also there is a 'pnexp' command available on the CLI. Even tough it is meant to export 4.x database before importing it into a new 5.x box (Generation 1). It can be used for 'immediate backups' as per the Cisco engineer on this thread:

http://forum.cisco.com/eforum/servlet/NetProf?page=netprof&forum=Expert%20Archive&topic=Security&topicID=.ee7f99a&CommCmd=MB%3Fcmd%3Dpass_through%26location%3Doutline%40%5E1%40%40.2cbeaa98/18#selected_message

Once you enter the pnexp CLI

Do the following:

export config 10.1.1.1:/archive.

Then enter the following command:

pnexp> status

Data exporting process is currently running, please use command 'log {all|recent}' to view running logs and/or progress.

pnexp> log

Aug 25 09:44:47.192 2008@LM_INFO@Thread 1024:Number of events exported: 1227522, ~0.18% completed, overall speed = 30688.05 eps

Aug 25 09:44:47.192 2008@LM_INFO@Thread 1024:Estimated-Time-To-Complete: 6 hours 18 minutes

CTRL-C

As you can see I'm doing a full backup here (config+event/report data etc), don't run that just to 'test' NFS. Its better to use the 'config' option :)

Regards

Farrukh

mhellman
Level 7
Level 7

You can use tcpdump from the command line. Try this:

tcpdump -X host

I'm not sure how intelligent it is, but some level of NFS decoding is performed by tcpdump and it may give you enough of a hint to get you going in the right direction.

Thats actually a really good idea. Of course, I keep thinking about how NFS is UDP so that a packet capture wouldn't be all that great. But If i do the dump at the same time requesting a file, i can confirm that data is indeed on the server and they're talking.

Thanks for the input!

Actually, it's TCP...at least the version I'm on now. I do think it used to be UDP though. Another reason to patch;-) Even with UDP, there will be application level acknowledgments, so tcpdump should work fine.

So, interesting enough, here is that tcpdump for communication to my NFS server. As you can see, it'd not really seeing a response from the NFS server. But the amusing part is, this is actually from when the NFS - Get Archived Files worked.

[pnadmin]$ tcpdump -i eth0 ip host 10.17.1.252

tcpdump: listening on eth0

16:08:36.407598 pnmars.1059879952 > 10.17.1.252.nfs: 108 getattr [|nfs] (DF)

16:08:37.400025 pnmars.1059879952 > 10.17.1.252.nfs: 108 getattr [|nfs] (DF)

16:08:39.400024 pnmars.1059879952 > 10.17.1.252.nfs: 108 getattr [|nfs] (DF)

16:08:43.400466 pnmars.1059879952 > 10.17.1.252.nfs: 108 getattr [|nfs] (DF)

16:08:51.400025 pnmars.1059879952 > 10.17.1.252.nfs: 108 getattr [|nfs] (DF)

16:09:07.400358 pnmars.1059879952 > 10.17.1.252.nfs: 108 getattr [|nfs] (DF)

16:09:39.400034 pnmars.1059879952 > 10.17.1.252.nfs: 108 getattr [|nfs] (DF)

16:10:39.400033 pnmars.1059879952 > 10.17.1.252.nfs: 108 getattr [|nfs] (DF)

16:11:39.400041 pnmars.1059879952 > 10.17.1.252.nfs: 108 getattr [|nfs] (DF)

16:13:07.541864 10.17.1.252.netbios-dgm > 10.17.1.255.netbios-dgm: NBT UDP PACKET(138)

interesting. pulling files should definitely result in a whole lot more activity than that. You don't really know what is happening "behind the scenes". MARS may not actually fetch data it has local for example (i.e. previously requested data, or data in the DB). The getattr should result in a response too.

I've actually expanded the tcpdump capture to get everything and then looked through it. For example, when I am doing that it isn't capturing my browsing the webgui. So how broken is tcpdump?

I'm not aware of it ever being broken in MARS, but I wouldn't have paid that much attention if it was. The MARS box isn't using both interfaces is it?

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: