Netbackup Error 89

[virankumar]

Hello Team,

We are facing issue with few client s in netbackup 6.5.6 version

89 error , for particular region

10 servers effected with this error

client version and OS versions are

solaris/6.5.6

Destails log for one client

-----------------

Dec 28, 2014 6:53:00 PM - requesting resource bnmlw-Disk-stage
Dec 28, 2014 6:53:00 PM - requesting resource south-nbumast-01.NBU_CLIENT.MAXJOBS.cnmlw-ivm-014-bsn
Dec 28, 2014 6:53:00 PM - requesting resource south-nbumast-01.NBU_POLICY.MAXJOBS.cnmlw-ivm-014_rman
Dec 28, 2014 7:05:56 PM - Error bpbrm (pid=26530) Could not get shared memory for bpbrm child process communication, No space left on device (28)
Dec 28, 2014 7:05:53 PM - granted resource  south-nbumast-01.NBU_CLIENT.MAXJOBS.cnmlw-ivm-014-bsn
Dec 28, 2014 7:05:53 PM - granted resource  south-nbumast-01.NBU_POLICY.MAXJOBS.cnmlw-ivm-014_rman
Dec 28, 2014 7:05:53 PM - granted resource  MediaID=@aaaam;Path=/bnmlw-medi-01-stu1;MediaServer=bnmlw-medi-01
Dec 28, 2014 7:05:53 PM - granted resource  bnmlw-medi-01-nbu_staging-01
Dec 28, 2014 7:05:54 PM - estimated 0 kbytes needed
Dec 28, 2014 7:05:55 PM - started process bpbrm (pid=26530)
Dec 28, 2014 7:05:57 PM - end writing
problems encountered during setup of shared memory  (89)

[BSircy]

Hello virankumar,

You are receiving this error because the number of shared segments is not high enough to process all the backup jobs.

To increase the memory identifiers, you can do the following:

projmod -a -K 'project.max-msg-ids=(privileged,32768,deny)' NetBackup

Additional information on this error can be found here.

http://www.symantec.com/business/support/index?page=content&id=TECH187992&profileURL=https%3A%2F%2Fs...

I hope this helps!

[Satyajit_Pal]

Hi,

First of all check if all the client backups running on which media server. If it is the same media server, check it's OS version. If it the OS of the media server is Sol 10, then follow the below steps for the remediation.

NetBackup could not allocate enough shared memory for backup buffer control on Solaris 10 media servers under heavy load. The following can solve the issue. 

shmsys:shminfo_shmmni=512 

Should be 512 or 1024, depends on how big the system is. This entry is a part of /etc/system and it's need to be tuned along with the below details also.

/usr/openv/netbackup/db/config/SIZE_DATA_BUFFERS 
/usr/openv/netbackup/db/config/NUMBER_DATA_BUFFERS

The SIZE_DATA_BUFFER value specified must not exceed the maximum tape I/O block size supported by the both the tape drive or the HBA driver for the Operating System.

For top performance for LTO-3/4/5 drives, the below setup may help you.

NUMBER_DATA_BUFFERS: 32 
NUMBER_DATA_BUFFERS_RESTORE: 32 
SIZE_DATA_BUFFERS: 262144 
SIZE_DATA_BUFFERS_DISK: 262144 

FYI:

SIZE_DATA_BUFFERS - Located in /usr/openv/netbackup/db/config/ directory. This is the size of the buffer that is used by bptm/bpdm processes (child/parent) to communicate with each other. It is also the tape block size that bptm uses to write to tape. 

NUMBER_DATA_BUFFERS - Located in the /usr/openv/netbackup/db/config/ directory is a bptm setting to set the number of Data Buffers available to the bptm process for communication between the parent and the child .

After making changes, verify in the bptm troubleshooting log that the values have taken affect. 

Example Values: 

SIZE_DATA_BUFFERS = 262144
NUMBER_DATA_BUFFERS = 64

The media server BPTM Log will show something similar to the following:

Example output:

09:42:18.141 [920.728] <2> io_init: using 262144 data buffer size
09:42:18.141 [920.728] <2> io_init: CINDEX 0, sched Kbytes for monitoring = 30000
09:42:18.141 [920.728] <2> io_init: using 64 data buffers

You can refer the below URL for your better reference.

http://www.symantec.com/business/support/index?page=content&id=TECH62633

Cheers...

[Marianne · Accepted solution]

The problem is on your media server, not with the clients.

There are too many simultaneous streams for the media server to handle.

See below TN and related links:

About shared memory (number and size of data buffers)
http://www.symantec.com/docs/HOWTO99778

e.g. 

See Amount of shared memory required by NetBackup

See How to change the number of shared data buffers

See How to change the size of shared data buffers