Forum Discussion

John_Wan's avatar
John_Wan
Level 4
13 years ago
Solved

Netbackup service and database down

I had raised a case with Severity 1 case and the technical gone without notice more than 2 hours so far. How a nice service which I need to paid a lot money.

 

Anyway, the problem is no matter I restart the service and even server, it can only went up for 1 or 2 minutes only, then the database will down. The last comment from your great support guy is

vxlogcfg -a -p 51216 -o 111 -s DebugLevel=6 DiagnosticLevel=6

 

after we run this command, error message had shown as below.

V-1-8-5 An unknown arguement DiagnosticLevel=6 was passed.

 

Please find someone to take a look about it!

Backup and Recovery
NetBackup
Windows Server (2003-2008)
  • mph999's avatar
    mph999
    13 years ago

    ... not sure if this is going to help much though,  If the DB is going down, then that will cause EMM to go down, EMM going down will NOT take the DB down.

    So ...

    What is the history, NBU will not just break like this on it's own.  What happened to the server just before the problem started ?

    You  say the DB is going down, how did you determine this ?

    The NB SQL database state can be seen with :

    /usr/openv/db/bin/nbdb_ping

    If the database is really going down it could be for a number of reasons, the most common

    1  Filesystem near 100% full

    2  Transaction log is corrupt

    Could be worth loking in /usr/openv/db/log/server.log

    I notice that in the first post you talk about EMM.  So it could be the case that this is the database you talk about going down, but really, NBEMM is just the front end of the NBDB I talked bout above.  However it is quite possible the nbemm is going down, but the SQL database is staying up, in which case the nbemm log would be required.

    Here is a guide to just about everything you need to know about logs.

    https://www-secure.symantec.com/connect/articles/quick-guide-setting-logs-netbackup

    The nbemm log is 'oid 111'.

    martin

7 Replies

Sort By
  • mph999's avatar
    mph999
    Level 6
    13 years ago

    Try this version.

    vxlogcfg -a -p 51216 -o 111 -s DebugLevel=6 -s DiagnosticLevel=6

  • John_Wan's avatar
    John_Wan
    Level 4
    13 years ago

    Hi Martin,

     

    Sorry to reply you late. The problem is solve somehow. The root cause of the problem is the filesystem of NBU reached almost 100% which will need to check at harddisk management. If I right click the drive of NBU, it will not show the usage of NBU.

     

    Also, NBU is count sperately with the disk drive. So even my C: still have a lot of space, NBU can still be full while NBU is set at c:.

     

    The problem now is wanna to check why it grow so fast suddunly. Do you have any idea on how to check it? Thanks.

    John

  • mph999's avatar
    mph999
    Level 6
    13 years ago

    If the space has been reclaimed, or has gone down, I know of no way of checking what has happened.

    The only real way, is to look at what is on the disk when the disk is 'full'.

    Martin

     

  • John_Wan's avatar
    John_Wan
    Level 4
    13 years ago

    OK, let me keep monitor to see what's going on.

     

    By the way, I would like to ask how can I check how many time of a cleaning tape being mount? Because we suspect that it reached the maximum time of mount already. Also, how can we determine it? From log? Thanks if you could help smiley

  • mph999's avatar
    mph999
    Level 6
    13 years ago

    tpclean shows when the drives were cleaned

    vmquery to see how many cleanings left on a tape

     

    root@nbmaster00 bin $ vmquery -m CLN101
    ================================================================================
    media ID:              CLN101
    media type:            1/2" cleaning tape (13)
    barcode:               CLN101L4
    media description:     Cleaning Tape
    volume pool:           None (0)
    robot type:            TLD - Tape Library DLT (8)
    robot number:          0
    robot slot:            22
    robot control host:    nbmedia00
    volume group:          000_00000_TLD
    vault name:            ---
    vault sent date:       ---
    vault return date:     ---
    vault slot:            ---
    vault session id:      ---
    vault container id:    -
    created:               Thu 19 Jan 2012 09:52:12 PM PST
    assigned:              ---
    last mounted:          ---
    first mount:           ---
    expiration date:       ---
    cleanings left:        25
     
    It's probably in a log somewhere (no idea ...) but as it's in the EMM DB I'll just look there and do it the easy way ...
     
    Martin
     
    PS.  Usual reason if NBU has cause a increse in disk useage is either growth of the catalog (not likely to happen overnight ...), or logs ...
     
    You would not believe how many times I've looked at cases like this and found someone dropped a big .zip file or something in the NBU filesystems ...  I'd get whatever tool you need in windows (sorry I have unix background) and search the machine for big files say anything over 500 MB, then anything over 400 MB, etc .. and see if you find anything that shouldn't be there.
     
    Martin
  • mph999's avatar
    mph999
    Level 6
    13 years ago

    ... not sure if this is going to help much though,  If the DB is going down, then that will cause EMM to go down, EMM going down will NOT take the DB down.

    So ...

    What is the history, NBU will not just break like this on it's own.  What happened to the server just before the problem started ?

    You  say the DB is going down, how did you determine this ?

    The NB SQL database state can be seen with :

    /usr/openv/db/bin/nbdb_ping

    If the database is really going down it could be for a number of reasons, the most common

    1  Filesystem near 100% full

    2  Transaction log is corrupt

    Could be worth loking in /usr/openv/db/log/server.log

    I notice that in the first post you talk about EMM.  So it could be the case that this is the database you talk about going down, but really, NBEMM is just the front end of the NBDB I talked bout above.  However it is quite possible the nbemm is going down, but the SQL database is staying up, in which case the nbemm log would be required.

    Here is a guide to just about everything you need to know about logs.

    https://www-secure.symantec.com/connect/articles/quick-guide-setting-logs-netbackup

    The nbemm log is 'oid 111'.

    martin

  • Dyneshia's avatar
    Dyneshia
    Level 6
    13 years ago

    On a side note, when you call into support and have a production down issue, you can always request to be transfered to the live queue to speak with analyst right away .