cancel
Showing results for 
Search instead for 
Did you mean: 

San Client backup failling with 83

Abhisheknetback
Level 6
Certified

Hello Friends ,

Some of the weekly RMAN backup which is configured as SAN client failling with status code 83 below is one of the failed job details .

 

 

8/15/2014 10:16:10 AM - Info nbjm(pid=8568) starting backup job (jobid=634117) for client q001_vip, policy BV RMAN_PRD_Weekly, schedule Weekly-Application-Backup  
8/15/2014 10:16:10 AM - Info nbjm(pid=8568) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=634117, request id:{B11FBC96-EDCB-47F5-9066-743BB21F9270})  
8/15/2014 10:16:10 AM - requesting resource FT1-FT2-HO-STU
8/15/2014 10:16:10 AM - requesting resource mst.NBU_CLIENT.MAXJOBS.q0_vi
8/15/2014 10:16:10 AM - requesting resource mst.NBU_POLICY.MAXJOBS.BV-_RMAN_QBRN_PRD_Weekly
8/15/2014 10:16:11 AM - granted resource mst.NBU_CLIENT.MAXJOBS.q0_vi
8/15/2014 10:16:11 AM - granted resource mst.NBU_POLICY.MAXJOBS.BV-_RMAN_QBRN_PRD_Weekly
8/15/2014 10:16:11 AM - granted resource MediaID=@aaaa5;DiskVolume=DD890_HO_LSU;DiskPool=OST_DD_HO_DISK_POOL;Path=DD890_HO_LSU;StorageServer=ddho;MediaServer=ftho2
8/15/2014 10:16:11 AM - granted resource ftho2_ho_stu1
8/15/2014 10:16:11 AM - granted resource TRANSPORT
8/15/2014 10:16:14 AM - estimated 0 Kbytes needed
8/15/2014 10:16:14 AM - Info nbjm(pid=8568) started backup (backupid=q0_vi_1408083373) job for client q0_vi, policy BV--_RMAN_QBRN_PRD_Weekly, schedule Weekly-Application-Backup on storage unit ftho2_ho_stu1
8/15/2014 10:16:15 AM - started process bpbrm (31159)
8/15/2014 10:16:16 AM - connecting
8/15/2014 10:16:18 AM - connected; connect time: 00:00:02
8/15/2014 10:16:42 AM - end writing
8/15/2014 10:18:50 AM - Info bpbrm(pid=31159) q0_vi is the host to backup data from     
8/15/2014 10:18:50 AM - Info bpbrm(pid=31159) reading file list from client        
8/15/2014 10:18:51 AM - Info bpbrm(pid=31159) listening for client connection         
8/15/2014 10:18:52 AM - Info bpbrm(pid=31159) INF - Client read timeout = 1200      
8/15/2014 10:18:53 AM - Info bpbrm(pid=31159) accepted connection from client         
8/15/2014 10:18:53 AM - Info bphdb(pid=12507) Backup started           
8/15/2014 10:18:53 AM - Info bpbrm(pid=31159) bptm pid: 31186          
8/15/2014 10:18:53 AM - Info bptm(pid=31186) start            
8/15/2014 10:18:53 AM - Info bptm(pid=31186) using 262144 data buffer size        
8/15/2014 10:18:53 AM - Info bptm(pid=31186) using 32 data buffers         
8/15/2014 10:18:53 AM - Info bptm(pid=31186) USING 262144 data buffer size for FT      
8/15/2014 10:19:13 AM - Error bptm(pid=31186) Could not open FT Server pipe: pipe open failed (589841)   
8/15/2014 10:19:13 AM - Info bptm(pid=31186) EXITING with status 83 <----------        
8/15/2014 10:19:16 AM - Info bphdb(pid=12507) done. status: 83: media open error       
media open error(83)

Kindly suggest .

 

Thanks You !!!

 

1 ACCEPTED SOLUTION

Accepted Solutions

Abhisheknetback
Level 6
Certified

Hi Friends,

I got below error from nbftsrvr log on FT media server .

09/02/2014 22:13:07.206 [Debug] NB 51216 nbftsrvr 199 PID:2564 TID:139838317016832 File ID:199 [No context] 1 [SOS_MemAllocPool] SC PAL DMA-able Scatter/Gather Region Vector Pool exhausted all 161 entries used.

If the FT media server is Linux, the  number of FT pipes * number of data buffers should not exceed 161 for NetBackup 6.5.5 and prior, and should not exceed 322 for 6.5.6 and above. but i am using NBU 7.5.0.5 however it's exhausting on 161 , Cause

 it would show up the value as 161 but not 322 as there are  two SGRVpool buffers

because Linux doesn't have room to allocate 322 SGRV buffers contiguously from the kernel non-paged pool.  161 are allocated for odd numbered pipes in SGRVpool[0] and another 161 are allocated for even numbered pipes in SGRVpool[1].

NUMBER_DATA_BUFFERS_FT can be used for overriding NUMBER_DATA_BUFFERS for SAN Client jobs. For a Fibre Transport media server, the recommendation is not to exceed 16.

So, it is anyways recommended to not use the value more than 16 which is 32 in our case

http://www.symantec.com/business/support/index?page=content&id=TECH62007

I reduce the No of data buffer from 32 to 16 now it is working fine .

Dear Riaan,

You were near around in your first post, thanks for your all help .

 

 

 

 

 

View solution in original post

11 REPLIES 11

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

Does it ever working fine? or its a new configuraiton?

please have a look at below T/N and see if that helps

http://www.symantec.com/business/support/index?page=content&id=TECH206660

Abhisheknetback
Level 6
Certified

Yes, it was working earlier even in second attempt it completed successfully.

i face this issue on the day of weekly backup only some random client fails with same error .

I have doubt that some thing need to be adjusted in tuning parameters of FT media server

currently buffer size is 262144 and no of buffers 32 .

any advice ????

 

 

 

Thank You !!

 

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified
Off the top of my head... the number buffers for ft should be 16. How many jobs are you running at the same time. FT has limits. Are you using an appliance?

Abhisheknetback
Level 6
Certified

no buddy it is not appliance, for how many no of jobs runing at the same time i will update you tomorrow .

and backup is happning on disk not on tape .

 

Thank You!!
 

Abhisheknetback
Level 6
Certified

Hi ,

During which time how many backup was runing you want me to check ???

like these all are RMAN backup and as it comes in multile streems .

so you want me to check how many job was runing when that particuler streem failed or how many job was rining when parent job failed .?/

 

please tell me .

 

 

Thaks You .

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

I meant, if you kick off your job(s) that write to the FT media server. There should not be more than 32 active at any one time.

Yuvi
Level 5
Certified

Hi,

No need of tuninng FT Pipes 32 is the max limit , for the I/O it accepts.

Backups to Disk may be logs written in full try to clean it up.

If it is netbackup 7.1.x environment where dedupe configured as MSDP try below commands and link

http://www.symantec.com/business/support/index?page=content&id=TECH172281

http://www.symantec.com/business/support/index?page=content&id=TECH180659

try the above links.

Abhisheknetback
Level 6
Certified

Hi Friends,

I got below error from nbftsrvr log on FT media server .

09/02/2014 22:13:07.206 [Debug] NB 51216 nbftsrvr 199 PID:2564 TID:139838317016832 File ID:199 [No context] 1 [SOS_MemAllocPool] SC PAL DMA-able Scatter/Gather Region Vector Pool exhausted all 161 entries used.

If the FT media server is Linux, the  number of FT pipes * number of data buffers should not exceed 161 for NetBackup 6.5.5 and prior, and should not exceed 322 for 6.5.6 and above. but i am using NBU 7.5.0.5 however it's exhausting on 161 , Cause

 it would show up the value as 161 but not 322 as there are  two SGRVpool buffers

because Linux doesn't have room to allocate 322 SGRV buffers contiguously from the kernel non-paged pool.  161 are allocated for odd numbered pipes in SGRVpool[0] and another 161 are allocated for even numbered pipes in SGRVpool[1].

NUMBER_DATA_BUFFERS_FT can be used for overriding NUMBER_DATA_BUFFERS for SAN Client jobs. For a Fibre Transport media server, the recommendation is not to exceed 16.

So, it is anyways recommended to not use the value more than 16 which is 32 in our case

http://www.symantec.com/business/support/index?page=content&id=TECH62007

I reduce the No of data buffer from 32 to 16 now it is working fine .

Dear Riaan,

You were near around in your first post, thanks for your all help .

 

 

 

 

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

It seems that Riaan pointed you in the right direction, right?

Please be fair and clear the solution for your pown post and mark Riaan's post as solution.

 

Thanks!

Abhisheknetback
Level 6
Certified

Hi Marianne,

Riaan was near around but in his second post he mentioned it should not be more than 32 and it was 32 in my setup.

however backup was failling , then i loged a case with symantec and i got this solution and details , so i shared that how this issue solved .

it is not about i marked my post  as solution , and i should mark to Riaan. the answer what i marked it is the details how issue been resolved and got the cause after involving symantec.

Like, If the FT media server is Linux, the  number of FT pipes * number of data buffers should not exceed 161 for NetBackup 6.5.5 and prior, and should not exceed 322 for 6.5.6 and above. but i am using NBU 7.5.0.5 however it's exhausting on 161 ,

in above line it says no data buffers should not exceed 161 for 6.5.5 and prior and should not exceed 322 for 6.5.6 and above , we are using 7.5.0.5 so it should have to support by 322 , but in log we got error exhausting on 161 .

this all been helped by Symantec TSE so shared that how issue solved and what was the cause .

if still you think i should mark solution to Riaan, i don't have any issue . but complete information is available in the post what i shared solution provided by symantec .

 

 

Thanks You !!

Abhisheknetback
Level 6
Certified

i can share symantec case ID and mail if you want .