cancel
Showing results for 
Search instead for 
Did you mean: 

database system error

Kasra_Hashemi
Level 5

Hi,

Whenever I want to restore a mailbox that was backed up with granular recovery, I encounter this error, I tried all these steps carefully but again I get the same error.

https://www.veritas.com/support/en_US/article.100009306

DB Sys Err.jpg

Netdigest
1 ACCEPTED SOLUTION

Accepted Solutions

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@Kasra_Hashemi 

I am not an Exchange expert (or NBU for Exchange expert), but I see numerous 'unable to .... '  error messages, e.g
"unable to successfully enumerate: Microsoft Information Store\Limit Office2013\Database\<user-name> "
"Unable to open or get value for registry key \HKEY_LOCAL_MACHINE\SOFTWARE\Veritas\NetBackup\CurrentVersion\Config\Granular\CopyLocal, error 2 "

Just search for the word 'unable' and 'error'.

Maybe the user name user 'mon-admin1' does not have sufficient permissions? I cannot say.

Hopefully @Lowell_Palecek will have better insight and advice. 

 

View solution in original post

21 REPLIES 21

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

As per the TN, you need 'nbfsd log on the client on which you have launched the Backup, Archive and Restore GUI'.

Please copy this log to nbfsd.txt and upload as attachment.

@Marianne   I've just attached the log.

Netdigest

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

What time did you try the restore? And on which server did you open BAR GUI? 

There are entries in the log between 08:29 and 15:30 with no errors. 
There is no ' <16> _nbfs_connect ' error as per article 100009306.

Lowell_Palecek
Level 6
Employee

Database system error is a catch-all for "it didn't work." Under the covers, your attempt to open the "Microsoft Information Store" node becomes a bplist request to bprd, which turns it into a Q_IMAGE_LIST_FILES query to bpdbm. If bpdbm doesn't find any files matching the request, you get this error.

The fact that you are failing at the MIS level indicates that nbfsd is not in the picture - at the browse side. Your backup didn't back anything up. You should see a hierarchy as follows starting at the MIS level:

Microsoft Information Store
   <database name>
      Database
         <mailbox name>
         <mailbox name>
         ...
   <database name>
   ...

Nblbc and nbfsd are not involved until you drill into a mailbox. The databases and mailbox names should have been catalogued at backup time. Bpdbm would then retrieve them without any need to involve nblbc on the client.

Look into your backups. Start with the bpfis log and see if it says no databases were found that match the backup criteria.

Hamza_H
Moderator
Moderator
   VIP   

Hello,

 

Please check first that:

-The backup is succussfull (status=0), no warning/error message in the detailled status.

-bpclntcmd -pn from your exchange nodes & the media server (the ouput?)

 

We faced this problem before and the root cause behind it was that the configuration of hosts files on all 'Master, Media and both exchange nodes) must have all the entries because the bpclntcmd didn't respond , i.e:

Master server's IP       NBUMASTER              NBUMASTER.FQDN

Media server's IP        NBUMEDIA                  NBUMEDIA.FQDN

Node1's IP                  NODE1                         NODE1.FQDN

Node2's IP                  NODE2                        NODE2.FQDN

after these changes, run bpclntcmd -clear_host_cache on all those hosts

check NFS on media server.

Client for NFS is running

Server for NFS is stopped and disabled.

 

Rememeber ! : After editing hosts files, run bpclntcmd -clear_host_cache 

@Marianne @Hamza_H @Lowell_Palecek @Hamza_H 

Whenever I want to expand a mailbox from the list I encounter this error( look at the picture :(

And as you know this error has a solution , mentioned in this post

https://www.veritas.com/support/en_US/article.100014437 )

if this link doesn't work, search for this KB Number : 100014437

and the solution provided in this KB is : "

Add the Distributed Application Restore Mapping entries to the disaster recovery master server host properties."

which I did and I've added Mapping Entry in that dialog box .


db err.PNG

Netdigest

Hamza_H
Moderator
Moderator
   VIP   

Hello,

Links you shared are not working,

Besides, have you checked the steps I shared in my post?

Hi @Hamza_H  Yes I did all the steps which you have mentioned, and it returns the information that you said.

Netdigest

Hamza_H
Moderator
Moderator
   VIP   

can you please share:

 

Version of NBU & OS for :

MASTER, MEDIA and CLIENT and exchange version.

also please share Ncflbc and ncfgre logs:

\NetBackup\bin\vxlogcfg -a -p 51216 -o 351 -s DebugLevel=6

\NetBackup\bin\vxlogcfg -a -p 51216 -o 352 -s DebugLevel=6

 

ps : don't forget to change the verbo again :

\NetBackup\bin\vxlogcfg -a -p 51216 -o 351 -s DebugLevel=0

\NetBackup\bin\vxlogcfg -a -p 51216 -o 352 -s DebugLevel=0

 

Hamza_H
Moderator
Moderator
   VIP   

Do you also get these errors :

 

grt 2016.PNG

 

source : https://www.veritas.com/content/support/en_US/downloads/update.UPD418628

 

@Hamza_H 

no that's not my issue, and for the links which you said they are not working , I have updated them . please check them out.

I really appreciate your effort , thank you .

Netdigest

Hamza_H
Moderator
Moderator
   VIP   

nope still not working "The document you are seeking is no longer available"

 

**edited**: yes I found the technote, "Cannot select items to restore Exchange GRT from DR site. Getting database errors or nothing in the mailboxes"

 

unless mistaken, I didn't get a reply regarding versions and logs..

I'm sorry I misinterpreted your original post. You weren't clear regarding what action of yours was failing. Your broader screenshot shows that you did get mailboxes catalogued, and now you have told us that the action that failed is expanding the mailbox.

NetBackup performs many steps in this operation. There can be different reasons for it to fail. Problems with nbfsd or permissions are not the only possibilities. We need to start with the logs and see which step failed.

First, I have some preliminary questions:
1. What versions of NetBackup and Exchange do you have, and what Exchange-related EEBs do you have installed, if any?
2. If you make the left panel of the BAR GUI wide enough to see the full mailbox names, are the aliases correct? You don't need to post a screenshot, just check that if you expect "Joe Smith [jsmith]" you don't have something weird inside the brackets.

Now to the logs.

1. Do you get an ncflbc log on one of your Exchange servers? If no, check the bpdbm log as described below. If yes, continue here.

2. Turn up logging on the client where you get the ncflbc log.

If nblbc executes on the same client where you are running the Backup, Archive, and Restore (BAR) application, select File / NetBackup Client Properties from the menu. On the Troubleshooting tab of the pop-up window, set the General logging level to 2 and the Verbose level to 5.

If your ncflbc log appears on a different Exchange server, go to the NetBackup console on the master server, and set the two logging levels for that client under client host properties.

3. Start with a fresh BAR GUI session. (Close it and reopen it.) Repeat the step that fails.

4. Look in the ncflbc log for error details. It may be ending with exit status 103. Look upstream for the failure that leads to that status.

NCF components use Veritas Unified Logging, which is hard to read in its raw form. Run vxlogview to format it for reading. Since we don't need pid / tid info, you can leave out the -d option, as follows:

>cd <install location>\NetBackup\bin
>vxlogview -i 351 >..\logs\ncflbc\nblbc.txt

If in step 1, you cannot find an ncflbc log on any client, check the bpdbm log. For this you need logging at level 3 or more on the master server.

2a. Look for ImageListFiles::executeQuery.

This method is called every time you expand a node in the BAR GUI. It logs its input as "pattern =". Advance through these entries until the pattern is your mailbox name. This is the bpdbm execution that returns your "database system error" status 220. If you are not getting an ncflbc log anywhere, look here for the reason.

@Lowell_Palecek 

Thanks for your detailed response

Netbackup version : 8.0 build 165

Exchange : 2013 CU 12

and aliases are all in correct form.

I will do the rest and tell you ASAP.

Netdigest

after changing the log level to 4, I got this log file, please take a look at it.

Netdigest

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

@Kasra_Hashemi 

PLEASE have another look at @Lowell_Palecek 's last post:

".... 

NCF components use Veritas Unified Logging, which is hard to read in its raw form. Run vxlogview to format it for reading. Since we don't need pid / tid info, you can leave out the -d option, as follows:

>cd <install location>\NetBackup\bin
>vxlogview -i 351 >..\logs\ncflbc\nblbc.txt  "

Please use the vxlogview command and then post the .txt file? 

@Marianne is this command supposed to create a logfile named " nblbc.txt" ?

 

Netdigest

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Yes.
You run OID 315 through vxlogview command to produce readable output. 

If you do not redirect the output to a file (   > filename ) then the output will be displayed on the screen. 

With @Lowell_Palecek 's command, the .txt file will be created in the ncflbc log directory. 

You can read up in NBU Command Ref Guide about vxlogview. 

@Marianne

thanks, here is the log. 

Netdigest