cancel
Showing results for 
Search instead for 
Did you mean: 

due to huge volume size backup fails with code 13

rVikas
Level 5
Partner

Hello Everyone,

We have a windwos 2008 -  file server with 8 A,B,C,E..etc.,drives,

Netbackup client version is 7.1 / Netbackup master/media version - Solaris 10 - 7.6.0.1 / Storage Target  - DataDomain

We have multistreaming enabled.

Incrmental backup completes successfully for this file server.

However whenever full backup runs, 2 drives always fail with code 13, 24. 

These 2 drives size is around 2TB.

 I have selected file read timeout to 3600, still no use.

Kindly advise if  i need to disable "windows open file backup"

1 ACCEPTED SOLUTION

Accepted Solutions

areznik
Level 5

There should be no consequences, NTIO disabled is the new default in NB 7.5+, meaning netbackup will turn this off for you if you upgrade your client. So this is a good thing to do even if it doesnt solve your issue, but i think it might. I had a very similar case to yours and this solved it. 

View solution in original post

20 REPLIES 20

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

first i would suggest you to upgrade the client to same as master and Media servers. and try the backups.

does it ever able to make the succesfull backups?

I think accelerator could help in this case.. however first get it upgrade and try Full backup 

 

rVikas
Level 5
Partner

Yes Ram, i will try to upgrade this 2 clients.

It had successfull backup in between however will require manual intervention to resume job 2-3 time.

If no one resumes it fails. The backup runs almost 59-60 hours.

 

Will disabling, windows open file backup help.

 

 

watsons
Level 6

Large backups may have timeout due to OS TCP settings, not controlled by Netbackup.

Look at this old technote: http://www.symantec.com/docs/TECH7576 :

Possibly the NetBackup server is getting a lot of traffic from clients; therefore, sockets are not becoming free soon enough to satisfy the demand.
 
Check the number of processes waiting for a server socket:   
     netstat -a | grep TIME_WAIT | wc -l

 

Then set the timeout of OS TCP keepalive to be lower:

ndd -get /dev/tcp tcp_keepalive_interval    (check this value)

By default, that value is 7200000, lower it to 300000 by:

ndd -set /dev/tcp tcp_keepalive_interval 300000

Retry backup..

 
 
 

SujayDesai
Level 3
Employee

You can add client name in Resilient network under master server host properties below is the step:-

Admin Cosole->Master Server Host Properties->Resilient Network->Add->Enter client name whose backup is failing

Then run below command on master server:-

On Windows:-

<install path>\netbackup\bin\admincmd\bprdreq -rereadconfig

On Unix:-

/usr/openv/netbackup/bin/admincmd/bprdreq -rereadconfig

Once this is done, run the backup if still issue persists i would recommend to check with Appcritical test and you will need to open support case with Symantec Netbackup Support Team to analyse the report.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

My 2c:

Break down the large volumes into smaller chunks, e.g:

NEW_STREAM
X:\folder1
X:\folder2
X:\folder...
NEW_STREAM
X:\folder..
X:\folder..
X:\folder...
NEW_STREAM
X:\folder..
X:\folder..
X:\folder...
NEW_STREAM
X:\folder..
X:\folder..
X:\folder...

Ensure multiplexing and multistreaming is enabled and at least 4 Jobs per Client enabled in Master's Global Attributes.

rVikas
Level 5
Partner

Hi Mairanne,

We already have multistreaming enabled.

Backup fails with code 13 for only one stream i.e one drive.

Drive size is 1.3 TB.

 

Other drives with size 2.3 TB completes successfully.

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

what is the client read timeout value that is set?

could you provide  the failed job detail status ?

have you tried upgrading the clients?

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Backup fails with code 13 for only one stream i.e one drive.

 

What I was trying to say in my previous post: 
Break down this drive into multiple streams. 

I have also posted and example of breaking down X-drive into multiple streams.

areznik
Level 5

Are the drives on the client server SAN or NAS attached? Is the bpbkar32.exe process on the client crashing?

If yes, check if you're running into the NTIO issue described here (Point 4 in the solution) and here. It is fairly safe to just disable NTIO in the registry as the article says and run a backup to see if it solves the problem. The technote even mentions that NTIO is disabled by default in 7.5+, its always better to let the OS handle things with its own native tools where possible.

 

 

sdo
Moderator
Moderator
Partner    VIP    Certified

Have you run a chkdsk to check the file system consistency for errors?

Try walking the file system yourself, does it error?  i.e. try cd / to go to the top of the drive, and then dir /s /b to walk the entire drive.

rVikas
Level 5
Partner

Incremental backup completes successfully , however i am getting problem for full backup only.

Today also 3 drives backup failed with code 13.

 

Disks are coming from SAN.

Below is error:

---------------------------------------

07/24/2015 21:23:48 - Error bptm (pid=12798) system call failed - Connection reset by peer (at child.c.1306)
07/24/2015 21:23:48 - Error bptm (pid=12798) unable to perform read from client socket, connection may have been broken
07/24/2015 21:23:48 - Error bptm (pid=12795) media manager terminated by parent process
07/24/2015 21:23:54 - Error bpbrm (pid=12771) could not send server status message
07/24/2015 21:23:57 - Info bpbkar (pid=6608) done. status: 13: file read failed

-----------------------------------------

I have set client read timeout to 9600 seconds

@Marianne

by breaking down large volumes ,dows it mean  create a seperate policy for each drive and then modify the selection list as you specified

rVikas
Level 5
Partner

@Areznic

You mentioned to perfom this step:

On the Windows 2008 NetBackup Client, disable the Symantec API called NTIO.  This will cause NetBackup to use the Microsoft API called 'BackupRead' - which does not encounter the same timing error.

STEPS TO DISABLE NTIO

1. Click Start | Run, type regedit, and click OK

2. Browse toHKEY_LOCAL_MACHINE\Software\VERITAS\NetBackup\CurrentVersion\Config

3. In the Config registry key, create a new key called NTIO

4. In the NTIO registry key, create a REG_DWORD value, give it the name UseNTIO and the valuedecimal  0 (zero)

whats the consequences of it and does it actually required.

 

areznik
Level 5

There should be no consequences, NTIO disabled is the new default in NB 7.5+, meaning netbackup will turn this off for you if you upgrade your client. So this is a good thing to do even if it doesnt solve your issue, but i think it might. I had a very similar case to yours and this solved it. 

RonCaplinger
Level 6

How many files are in the one stream that is consistently failing?

cruisen
Level 6
Partner Accredited

Hello,

can you please tell how much space do you have available on the netbackup DB folder. It sounds that for the bigg Full backup lots of new entries are written to the database that will fill the folder and then will crash the connection. You have always to assure to have more than 3 GB of space for the database available.

Perhaps you are running at the same time oracle backups ( crosscheck and expiration of files) , or elseware expiration on dB images that Netbackup is not able to handle,  when running at the same time. I often see = File read failed errors = because of heavy netbackup db activities while running backups.

Best regards,

cruisen

rVikas
Level 5
Partner

Hi Cruisen,

Its windows client and acting as DFS server.

cruisen
Level 6
Partner Accredited

Do you have checked the available space on Netbackup server please, where the software is installed. How much space is left on the disk?

rVikas
Level 5
Partner

I upgraded that server to 7.6.0.1, so streams completed successfully however it tooh huge time around 60+ hours to backup 1.8 TB.

 

Still i am not sure, it it is due to upgrade, i need to monitor few more full backups.

areznik
Level 5

I dont mean to keep harping on about the same point, but I will anyway :).

I'm 99% sure your backup is successful because upgrading to 7.6.x disabled NTIO.