NDMP backup of Netapp-Filer fails - sometimes (Status 23)

[rgd]

hi all!

i have a problem backing up a netapp filer via NDMP, but only sometimes

setup: Single Masterserver/Mediaserver with W2k8R2 and Netbackup 7.5.0.5, LTO-Library and Netapp-3020 Filer

the NDMP dump itself is successful, both times, but thereafter, something goes wrong

i attach 4 logs, one the log of the job seen in the GUI, the other one being bptm reduced to the job in question & the final moments (some info had to be replaced by policy, sry), once for the failed job, once for the successful one - to be specific, its actually the second try of the same job

trouble begins at line 56 of the bptm*_end.txt where on one attempt the connection succeeds, and fails on the other..
it seems to be a connection issue, which is strange though, since none of the listed ports is explicitly allowed by the FW inbetween the two networks (10.32.238 and 10.61.192) except the ones specified by symantec (10000, 1556, 13720, 13724, 13782, 13783)

it also seems that the problem occurs on large backups more frequently than on smaller ones

should further info be required let me know

if someone could help me to understand the issue, and if its really necessary to open a wide range of ports (although i doubt that)
thanks for your time

rob

p.s. i checked all the technotes about NDMP, Error 23 and the forums
also tried setting NDMP_PROGRESS_TIMEOUT and various other (but NOT NDMP_MOVER_CLIENT_DISABLE)

[Amol_Nair]

The NDMP_MOVER_CLIENT_DISABLE will resolve the status 23 error for NDMP backups but will affect the performance of these backups. Take a look at the following technote for its impact on backups http://www.symantec.com/docs/TECH162041

[rgd]

hi!

yes, i noticed, but i do not want to use this, since it works on some occasions, and i would like to find the cause, before i use this option

if my backups would fail all the time, i would've tested this option already

but thanks for your comment :)

[mnolan]

Good port was 1863, bad one was 2594

I usually have customer's doublecheck that ports 1025-5000 are open for the data connection server that we open.

[Deb_Wilmot]

Check out the following:

http://www.symantec.com/docs/TECH205899
 

That apepars to show the same issue you're experiencing.

From your logs:

17:53:59.370 [4856.3416] <2> NdmpMoverSession[0]: ndmp_mover_connect failed, status = NDMP_CONNECT_ERR
17:53:59.370 [4856.3416] <2> NdmpMoverSession[0]: MoverConnect method failed. Reinitializing and attempting to use MoverListen method.
17:53:59.604 [4856.3416] <2> NdmpMoverSession[0]: ndmp_mover_set_window(0, 18446744073709551615)
17:54:00.025 [4856.3416] <2> NdmpMoverSession[0]: ndmp_mover_listen failed, status = NDMP_CONN_TYPE_UNCONFIGURED_ERR
17:54:00.025 [4856.3416] <16> NdmpMoverSession[0]: ERROR Start failed
17:54:00.025 [4856.3416] <16> check_and_process_mover_tasks: NDMP Mover Client Setup failed
17:54:00.243 [4856.3416] <2> NdmpMoverSession[0]: Shutdown complete
17:54:00.243 [4856.3416] <2> write_data_tir: ndmp_task = 1
17:54:00.243 [4856.3416] <2> write_data_tir: status_to_return = -1
17:54:00.243 [4856.3416] <2> check_error_history: just tpunmount: called from bptm line 21443, EXIT_Status = 23
17:54:00.243 [4856.3416] <2> io_close: closing E:\Program Files\Veritas
 

If that doesn't resolve the issue - check out the following as it has great networking troubleshooting tooks.

[rgd]

hi deb!

saw that one too, but again, the backup is not failing all the time, so i do not think its a resolver issue, as the name is correctly resolved in the logs as well

and, besides that, i have two filers to back up with ndmp, so i believe that is probably causing more problems ...

thanks, further hints welcome :)

[jim_dalton]

I have also seen this...every weekend!

The filer/Netbackup have another attempt and the second iteration succeeds.

It only affects two vols - the same two - so you could say its data dependant.

Specifically its two folders within two vols ie /vol/vol1/folderY, /vol/vol2/folderZ.

Jim

[rgd]

hi jim!

can't confirm that it's always the same volumes - and i WISH that it would work the second attempt

but unfortunately it doesn't all the time, and i get problems with my backup-window when i have to backup 6TB via NDMP and need to wait for the second attempt ;)

what have you tried so far to get rid of the issue?