cancel
Showing results for 
Search instead for 
Did you mean: 

Netbackup shutdown takes a long time

AndyBB
Level 1

Hi

Appreciate any help here...

Master / Media server is SunOS liveux50 5.10 Generic_144488-09 sun4v sparc SUNW,T5240 32544 MB memory

NBU 7.6.0.3 which saves many clients (NT, Unix), doing about 540 jobs per night and about 410 TB a month to LT04

Netbackup works absolutely fine. However we have an issue when we do a "netbackup stop", it takes a long time to close -  we have traced it to when it does a "rm -rf /usr/openv/tmp/sqlany".

This may have been happening since we upgraded to 7.6, but as we hardly ever stop Netbackup or do a reboot, never really noticed the cause (and used bp.kill_all when thinking the netbackup stop is taking ages).

 

We have gone through https://www.veritas.com/support/en_US/article.000083888 and had no issues, but still find a lot of files in sqlany.

root@liveux50 # bpps -a

NB Processes

------------

 

 

MM Processes

------------

root@liveux50 # /opt/VRTSpbx/bin/vxpbx_exchanged status

Symantec Private Branch Exchange is running

root@liveux50 # opt/VRTSpbx/bin/vxpbx_exchanged stop

Stopped Symantec Private Branch Exchange

root@liveux50 # /opt/VRTSpbx/bin/vxpbx_exchanged status

Symantec Private Branch Exchange is not running

root@liveux50 # /opt/VRTSpbx/bin/vxpbx_exchanged start

Started Symantec Private Branch Exchange

root@liveux50 # /opt/VRTSpbx/bin/vxpbx_exchanged status

Symantec Private Branch Exchange is running

root@liveux50 # bpps -a

NB Processes

------------

 

 

MM Processes

------------

root@liveux50 # /usr/openv/netbackup/bin/goodies/netbackup start

NetBackup Authentication daemon started.

NetBackup network daemon started.

NetBackup client daemon started.

NetBackup SAN Client Fibre Transport daemon started.

NetBackup Discovery Framework started.

NetBackup Database Server started.

NetBackup Authorization daemon started.

NetBackup Event Manager started.

NetBackup Audit Manager started.

NetBackup Deduplication Manager started.

NetBackup Deduplication Engine started.

NetBackup Deduplication Multi-Threaded Agent started.

NetBackup Enterprise Media Manager started.

NetBackup Resource Broker started.

Media Manager daemons started.

NetBackup request daemon started.

NetBackup compatibility daemon started.

NetBackup Job Manager started.

NetBackup Policy Execution Manager started.

NetBackup Storage Lifecycle Manager started.

NetBackup Remote Monitoring Management System started.

NetBackup Key Management daemon started.

NetBackup Service Layer started.

NetBackup Indexing Manager started.

NetBackup Agent Request Server started.

NetBackup Bare Metal Restore daemon started.

NetBackup Web Management Console started.

NetBackup Vault daemon started.

NetBackup CloudStore Service Container started.

NetBackup Service Monitor started.

NetBackup Bare Metal Restore Boot Server daemon started.

root@liveux50 # echo $?

0

 

A week later  a ls -f /usr/openv/tmp/sqlany | wc -l found 71348 in there, and the stop took about 30 minutes to complete.

The following are fine

/usr/openv/db/bin/nbdb_ping

cat /usr/openv/var/global/databases.conf

"/usr/openv/db/data/NBDB.db" -n NBDB

"/usr/openv/db/data/NBAZDB.db" -n NBAZDB

"/usr/openv/db/data/BMRDB.db" -n BMRDB

cd /opt/openv/db/bin

./nbdb_admin -validate BMRDB  this was fine

./nbdb_admin -validate NBAZDB  this was fine

./nbdb_admin -validate    this was fine

 

root@liveux50 # cat /etc/system
*ident  "@(#)system     1.18    97/06/27 SMI" /* SVR4 1.5 */
*
* SYSTEM SPECIFICATION FILE
*

* moddir:
*
*       Set the search path for modules.  This has a format similar to the
*       csh path variable. If the module isn't found in the first directory
*       it tries the second and so on. The default is /kernel /usr/kernel
*
*       Example:
*               moddir: /kernel /usr/kernel /other/modules

 

* root device and root filesystem configuration:
*
*       The following may be used to override the defaults provided by
*       the boot program:
*
*       rootfs:         Set the filesystem type of the root.
*
*       rootdev:        Set the root device.  This should be a fully
*                       expanded physical pathname.  The default is the
*                       physical pathname of the device where the boot
*                       program resides.  The physical pathname is
*                       highly platform and configuration dependent.
*
*       Example:
*               rootfs:ufs
*               rootdev:/sbus@1,f8000000/esp@0,800000/sd@3,0:a
*
*       (Swap device configuration should be specified in /etc/vfstab.)

 

* exclude:
*
*       Modules appearing in the moddir path which are NOT to be loaded,
*       even if referenced. Note that `exclude' accepts either a module name,
*       or a filename which includes the directory.
*
*       Examples:
*               exclude: win
*               exclude: sys/shmsys

 

* forceload:
*
*       Cause these modules to be loaded at boot time, (just before mounting
*       the root filesystem) rather than at first reference. Note that
*       forceload expects a filename which includes the directory. Also
*       note that loading a module does not necessarily imply that it will
*       be installed.
*
*       Example:
*               forceload: drv/foo

 

* set:
*
*       Set an integer variable in the kernel or a module to a new value.
*       This facility should be used with caution.  See system(4).
*
*       Examples:
*
*       To set variables in 'unix':
*
*               set nautopush=32
*               set maxusers=40
*
*       To set a variable named 'debug' in the module named 'test_module'
*
*               set test_module:debug = 0x13

* Begin MDD root info (do not edit)
rootdev:/pseudo/md@0:0,10,blk
* End MDD root info (do not edit)
forceload: drv/st
forceload: drv/sg
forceload: drv/fp

#Added for Netbackup
set shmsys:shminfo_shmmin=100
set shmsys:shminfo_shmmax=8589934592
set shmsys:shminfo_shmmni=512
set shmsys:shminfo_shmseg=100
*
set semsys:seminfo_semmap=64
set semsys:seminfo_semmni=1024
set semsys:seminfo_semmns=1024
set semsys:seminfo_semmnu=1024
set semsys:seminfo_semmsl=300
set semsys:seminfo_semopm=50
set semsys:seminfo_semume=64
*
set msgsys:msginfo_msgmap=512
set msgsys:msginfo_msgmax=8192
set msgsys:msginfo_msgmnb=65536
set msgsys:msginfo_msgmni=256
set msgsys:msginfo_msgssz=16
set msgsys:msginfo_msgtql=512
set msgsys:msginfo_msgseg=32767

# For NBU 7.6 upgrade
set rlim_fd_cur=16384
root@liveux50 #

 

 

What is /usr/openv/tmp/sqlany used for?

 

Thank you all

Andy BB (GB)

3 REPLIES 3

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

It seems to a location for temporary database files. I've not seen any references to it causing issue, but if you maybe upgrade you'll get a newer version of sybase that would fix the issue :)

sdo
Moderator
Moderator
Partner    VIP    Certified

I think "ls -f" is not giving you a true file count.  I reckon you might have hundreds of thousands of files.  Best to open a support case.

I can find no reference to "sqlany" in the v7.6 or v7.7 EEB guides.

sdo
Moderator
Moderator
Partner    VIP    Certified

What's in your /usr/openv/var/global/server.conf file?  Any odd switches, typos?

Do you have another to compare it to?

N.B: Do not make any changes on a whim.  Purely look/inspect for now.

FYI - about:  server.conf

https://www.veritas.com/support/en_US/article.000044409