NDMP Failure with 99

[sri_vani]

Some thing (fishy) and strange to me appreciate Ur thoughts and help here:

when we trying to take full backup of one particular volume--backup completed successfully with status 0.

when we validated with the actual data with backedup data-we found chunk of files missing..

When verified further I found that one particular folder outboundxxxx is nt getting backedup.

So I have created a test policy tried to take a backup to that particular file--unfortunately tat was failed with 99.

Am able to take the backup till before the previous folder..Hope am clear,,,nt sure wre am goin wrng ..

Detailed Status

1/24/2014 7:50:28 AM - begin writing
1/24/2014 7:50:29 AM - Error ndmpagent(pid=2556) NDMP_LOG_ERROR 1 DATA: Backup terminated: backup configuration failure (2) on RPC error: ERROR: BAD ENV OP (for PATH)
1/24/2014 7:50:29 AM - Error ndmpagent(pid=2556) NDMP backup failed, path =PATH1/24/2014 7:50:29 AM - Error bptm(pid=12244) none of the NDMP backups for client CLIENTNAME completed successfully    
1/24/2014 7:50:34 AM - end writing; write time: 00:00:06
NDMP backup failure(99)

For this case I cant share logs publicly but I can share thru Private message if reqd..

Thanks in advance..

[Nicolai]

Consider this TN: http://www.symantec.com/docs/TECH56492

Especially the messages/syslog/eventlog on the NDMP appliance. I think is can shed light on the ERROR: BAD ENV OP message. 

Using NDMP exclude list ?  (Hint: http://www.symantec.com/docs/TECH10195 )

Similar issue: https://www-secure.symantec.com/connect/forums/ndmp-backup-terminated-backup-configuration-failure-2-rpc-derror-error-bad-env-op

[sri_vani]

some finding -nt sure how much it wld be helpful to resolve this

1)when verified this particular folder attributes from server end-read only option is grayed out

I have requested our filer team to verify Can ndmproot access this folder? I have yet to get the results

2)Found this error on the logs:

 nb_bind_on_port_addr: set socket option SO_EXCLUSIVEADDRUSE

http://msdn.microsoft.com/en-us/library/windows/desktop/cc150667%28v=vs.85%29.aspx

Is it anything related to it? may be am wrong

3)while trying to increase the debug log for more info getting below error

V-1-8-0 Invalid setting/Value is passed.

http://www.symantec.com/business/support/index?page=content&id=TECH153189
http://www.symantec.com/business/support/index?page=content&id=TECH157017

yet to resolve this

[sri_vani]

Thq Nicolai..

sure lemme check and get back to u

[sri_vani]

3)rd issue got resolved after service refresh...and able to change the debug levels

[sri_vani]

ahh wat a pain...lemme take a long breath

after so much of troubleshooting I found that all the missed chunk of files are backedup in the special folder which is never part of volume.

The actual folder name is outboundxxxx,

but when verified  restore window , found some Strange folder with the name :60RFN00

all these missed chunk were exists in this folder

Why and how it renamed needs to be investigated..

Is it a issue with Windows,FILER,/nbu(am sure nt NBU).

Then who is the culprit how the actual folder name(outboundxxxx) got renamed as :60RFN00

[sri_vani]

when backing up

vol/vol1/....../.../.../outbound folder -It is failing with 99 but intrestingly the backup gt successful if i change the path as vol/vol1.../.../:60RFN00

[RamNagalla]

interesting... keep us posted  with your investigations..

[sri_vani]

Sure Nagalla ...but unfortunately gt stuck up ...

Have raised a support call ...Stay Tuned for more updates

[Marianne]

Have you verified if this maybe share-name vs actual name?

i.e. shared as outboundxxx but actual path name on filer is :60RFN00?

[rk1074]

hmmm interesting case....

I also tried to get some some info from storage as per the Granny's suggestion and what I get to know know is that  in case of CIFS share..share name can be different from the actual name of the volume on the filer and in order to get the backup  we need to give the the actual name...

so it may be the case that outboundxxx happens to be CIFS share

[Marianne]

Seems you had a similar error a couple of months ago that is still not yet resolved?

https://www-secure.symantec.com/connect/forums/ndmp-backup-terminated-backup-configuration-failure-2... 

[sri_vani]

My filer team confirmed me that they dont see this strange name as we see it from backups :(

[Marianne]

Have they actually provided you with output of 'ls' command?

http://www.wafl.co.uk/tag/ls/ :

In “priv set advanced” you can use “ls” to look at what actually exists on your volumes - ...

[RamNagalla]

by any chance did you see the NDMPd log .. how the folder names looks like?

does it :60RFN00  or outboundxxx   ?

[sri_vani]

No, the only thing they set as share is the Work*** volume and outboard is subfolder of it..

[sri_vani]

Honestly it dint give me any clue ..I sent the NDMPD log thru private messagner plz verify..

[sri_vani · Accepted solution]

#Edit#

1)Filer team took the snapshot of the volume and able to see the outboundxxx folder as outboundxxx.

Have tried taking the backup of this snapsht ..Damn'... i gt the starnger back whn verified in the restore panel.. :(

2)I have a created a new folder and named it as test and copied the outboundxxx files to it and and tried the backup. we are able to take the backup of test folder and later I have renamed this test as outboundxxx and removed extra spaces and original outboundxxx as olderoutboundxxx.

All working good now..

[sri_vani · Accepted solution]

I think the permananet fix wld be

1)Avoid complicated folder names on filer (The folder name which i ve faced problem has very long name along with the special characters including spaces , minus etc..)

seems we cant copy these  complex names to one NTFS volume to another volume within filer

2)Upgrade the netbackup version to 7.6 or 7.5.x(ourz is 7.0.1)

[RamNagalla]

thats nice... Thank you very  much for the update...

please mark the solution, it will help the other who have the same issue.. though its a strange and rare error... :-