Highlighted

Need to run NBCC and fix the issue with catalog

Catalog backups are completing with partial success. Looks like we need to run Netbackup Catalog Consistency Check.

ROOT@backupmaster# pwd
/opt/openv/netbackup/bin/support
ROOT@backupmaster# ls
MoreOldNBCCRs   bin             hp11.00         nbcplogs        output
NBCC            cmd.stderr.txt  input           nbdna           rs6000_433
NBCCR           config          linux2.4        nbsu            solaris7
OldNBCCRs       doc             nbcc            nbsupport       x86
ROOT@backupmaster# NBCC -help
NBCC: not found
ROOT@backupmaster# NBCCR -help
NBCCR: not found
 

1 Solution

Accepted Solutions
Accepted Solution!

/opt/openv/netbackup/bin/supp

/opt/openv/netbackup/bin/support is not in your $PATH so just precede the command with ./

i.e.

./NBCC -help

or use the absolute path

i.e.

/opt/openv/netbackup/bin/support/NBCC -help

 

EDIT:

Nice to see the job details for the partial success though .....

View solution in original post

20 Replies
Accepted Solution!

/opt/openv/netbackup/bin/supp

/opt/openv/netbackup/bin/support is not in your $PATH so just precede the command with ./

i.e.

./NBCC -help

or use the absolute path

i.e.

/opt/openv/netbackup/bin/support/NBCC -help

 

EDIT:

Nice to see the job details for the partial success though .....

View solution in original post

Re: Need to run NBCC and fix the issue with catalog

Run ./NBCC Current directory (.) is not by default part of PATH variable.

Re: Need to run NBCC and fix the issue with catalog

Thanks Andy and Marianne for your speedy respose! I completely forgot that it's not in the $PATH.

BTW, there are few more questions have before running NBCC.

       1. Does NBU need to be stopped on master and all media servers?

       2. What sould be the switch to run NBCC with? Does this output need to go to any file?

       3. Do i need to run NBCCR after NBCC, if so, with what switch?

       4. Any additional information or guidance would be appreciated. Like steps,etc.

Thanks Again!

Re: Need to run NBCC and fix the issue with catalog

Been a **long** time since I ran NBCC (& I have **never** run NBCCR).

Think you can go without any options, although last time I believe I did the -nozip & -nocleanup just to see what the output actually was - had a bit of time on my hands obviously. The -help does explain these options quite clearly & which you use may depend upon your environment.

The -help also explains what needs to be running & if NB is 'active' then you may get false positives so best to run when quiet. I think we also got false positives when it was "between cleanups" - images had expired but tapes waiting to be unassigned.

 

Probably best to run in conjunction with Support - I presume you have a call logged for your catalog issue?

Regarding the latter - why exactly is the catalog backup partially complete? What errors are you getting in job details?

 

 

Re: Need to run NBCC and fix the issue with catalog

Run NBCC without any options. It will produce a compressed/zip file with output that needs to be submitted to Veritas Support as part of a Support call. This is not necessarily going to fix the catalog status 1. Show us all output in the parent job and all of the child jobs. And tell us NBU version and patch level.

Re: Need to run NBCC and fix the issue with catalog

Netbackup 6.5.6 and we don't have support.
Since Netbackup is now seperated and is with VERITAS, we have a plan to start a fresh discussion with Veritas for NBU upgrade with Support.

There is one parent and 3 child jobs out of which two completes and one complete with partial success. PFA the job details.

BTW, any idea how long does this NBCC take so that i can schedule the downtime accordingly.

 

Re: Need to run NBCC and fix the issue with catalog

Seems you found this TN: http://www.veritas.com/docs/000011088 ?

No point in running NBCC if you cannot log a Support call for assistance.

Rather run bpdbm -consistency and check resulting bpdbm log for errors.

See : https://www-secure.symantec.com/connect/forums/bpdbm-consistency-checks 
and https://www-secure.symantec.com/connect/forums/how-use-bpdbm-consitency 

Re: Need to run NBCC and fix the issue with catalog

You can identify corrupt image headers yourself and remove it.

See this TN:

DOCUMENTATION: How to manually identify and remove zero-byte or corrupt image header files ......
http://www.veritas.com/docs/000034008 

Scroll down to  
Alternative Method:
and follow steps suggested in this section.

Re: Need to run NBCC and fix the issue with catalog

Just for reference, you don't need to stop NBU jobs to run NBCC.

The documantation says to stop them, but it doesn't actually matter - you may get 16_4 errors with it running, but we just ignore these anyway as they will have cleared when the backup completes, and in the very unlikely event they don't, you'll pick them up on the next run of nbcc.

The other mis-conception is that you have to run NBCC for an upgrade, again, totally untrue - there are no NBCC errors that will cause an upgrade to fail, PROVIDING, you are not upgrading from 5.x

Once on 6.x or 7.x, any nbcc errors before an upgrade will just be carried across and still be there after the upgrade.

Re: Need to run NBCC and fix the issue with catalog

Thanks Marianne!
I'm going with your suggestion first with the steps mentioned under 'Alternative Method' TN.

@mph999 - Awesome and thanks for clearing the mis-conceptions about NBCC.
Second option is to go with running NBCC. I'll keep you posted with the result and updates.

I'm also planning in parallel to have a cold catalog copy to be on safe side.

Re: Need to run NBCC and fix the issue with catalog

We are unable to find any corrupt image headers from the 'find' commands

I'm going with NBCC now. We're discussing if we can run it today in afternoon but just want to know how long will it take.

Also, it will not break anything or try to repair, right?
We just need to run NBCC on master and the output will go under the support directory? Or Is this need to be directed somewhere?
 

Any other things to take care of?

Re: Need to run NBCC and fix the issue with catalog

NBCC will not break or repair anything so you don't need to worry about that. It just reads data from the database, no updates are made at all unless you tell it to.

The output will be logged under output\nbcc directory automatically so there is no need to do any file redirections.

NBCC will vary in how long it will take. Shouldn't take longer than 20 minutes on an average system.

As Martin said earlier, just run it, you don't need to "prepare" the system in anyway. As long as NetBackup is up and running it will work.

 

Re: Need to run NBCC and fix the issue with catalog

Thank you!

I'll be running NBCC shortly and have also planned to run the bpdbm consistency check [level 2] if it completes early and doesn't mess with anything in production.

I'll keep you all posted and once again thanks for all your inputs/thoughts Marianne/Martin/Revaroo

 

Re: Need to run NBCC and fix the issue with catalog

One thing - download the latest copy of NBCC before you send in the output ....

http://www.veritas.com/docs/000037262

This can be run, even if the NBU version is lower.

Can he log a call? See this

Can he log a call?

See this post: https://www-secure.symantec.com/connect/forums/need-run-nbcc-and-fix-issue-catalog#comment-11494161 

 

Is there a purpose running NBCC if user does not have support?

No, if the customer does not

No, if the customer does not pay for support then they can't open a support case unless they are going through a contract renewel process.

Re: Need to run NBCC and fix the issue with catalog

No point in running NBCC then. Use other methods as suggested above to find problematic images.

Re: Need to run NBCC and fix the issue with catalog

Two methods exist for validating the consistency of the NetBackup EMM (Enterprise Media Manager) and BMR (Bare Metal Restore) Databases. One is performed from a command line interface CLI and the other by using the NetBackup database administration utility which is included in NetBackup 6.5.2 or later.

Note: Before performing a consistency check, stop all NetBackup Services on the EMM server.

The master and EMM servers are usually the same server.

Once all services have stopped, start only the ASA Service.

Windows
6.x: <install_path>\Veritas\NetBackup\bin>bpup -e ASANYs_VERITAS_NB
7.x: <install_path>\Veritas\NetBackup\bin>bpup -e SQLANYs_VERITAS_NB

UNIX: 
/usr/openv/netbackup/bin/nbdbms_start_stop start

Method 1: Using the command line interface (CLI):
To perform a consistency check from the CLI, issue the following commands:
 
Windows:
<install_path>\NetBackup\bin>nbdb_admin <databasename> -validate -full
 
UNIX:
cd /usr/openv/db/bin
./nbdb_admin <databasename> -validate -full
 
If any inconsistencies are found, they will be reported.
 
For a consistent database, no errors reported will be returned as output.

Method 2: Using the NetBackup Database Administration utility:
To perform the consistency check using the NB DB Admin Utility (provided in NetBackup 6.5.2 or later) do the following:
 
Windows:
<install_path>\NetBackup\bin>NbDbAdmin.exe
 
The NbDbAdmin GUI will load up and ask for the database password. If the password has never been changed, enter the default password nbusql. The tool will present you with the General tab. When the tool starts up, it will implicitly run a consistency check of the database. If no errors are found, the tool will report Database NBDB is alive and well. To check the BMR database, select the BMRDB and run the validation by navigating: General tab > Validation > Validate > Full.  Confirm that no errors were reported by checking the final line of the output.

(Figure 1)
 
 
UNIX:
cd /usr/openv/db/bin
./dbadm
 
The dbadm MUI (menu user interface) will open and ask for the database password. If the password has never been changed, enter the default nbusql. As with the Windows GUI, the UNIX MUI also implicitly checks the consistency of the EMM database when it is started. If no errors are found, the database consistency will be displayed as OK. To check the BMR database, select the BMRDB, enter choice 4 and run a Full validation.

(Figure 2)
  
 

That will validate the actual

That will validate the actual Sybase database but will not validate the consistency between the media statuses in the "mediaDB and volDB" (as they used to be known) and the NetBackup images database that NBCC would check.