cancel
Showing results for 
Search instead for 
Did you mean: 

SQL DB backups are failing with error - 2

lovee
Level 5

Hello

NBU vers. - 7.6.1.2

SQL DB backups were working well but got failed from last 3 days. I a policy of 8 Clients 4 are failing. Below is the error message. VM backups are working fine.

Aug 2, 2017 9:11:01 AM - Info bphdb (pid=15232) Backup started
Aug 2, 2017 9:11:01 AM - connected; connect time: 0:00:00
Aug 2, 2017 9:11:10 AM - Info dbclient (pid=15252) USER - An instance of SQL Server was not found on this client.
Aug 2, 2017 9:11:10 AM - Info dbclient (pid=15252) INF - Results of executing <Daily_Incr.bch>: 
Aug 2, 2017 9:11:10 AM - Info dbclient (pid=15252) <0> operations succeeded. <0> operations failed.
Aug 2, 2017 9:11:30 AM - Error bpbrm (pid=51574) from client usssp0036.na.novartis.net: ERR - command failed: none of the requested files were backed up (2)
Aug 2, 2017 9:11:39 AM - Info dbclient (pid=1956) USER - An instance of SQL Server was not found on this client.
Aug 2, 2017 9:11:39 AM - Info dbclient (pid=1956) INF - Results of executing <Trans.bch>: 
Aug 2, 2017 9:11:39 AM - Info dbclient (pid=1956) <0> operations succeeded. <0> operations failed.
Aug 2, 2017 9:11:59 AM - Error bpbrm (pid=51574) from client usssp0036.na.novartis.net: ERR - command failed: none of the requested files were backed up (2)
Aug 2, 2017 9:11:59 AM - Error bpbrm (pid=51574) from client usssp0036.na.novartis.net: ERR - bphdb exit status = 2: none of the requested files were backed up
Aug 2, 2017 9:12:00 AM - Info bphdb (pid=15232) done. status: 2: none of the requested files were backed up
Aug 2, 2017 9:12:00 AM - end writing
none of the requested files were backed up  (2)

and batch script:

OPERATION BACKUP
DATABASE $ALL
SQLINSTANCE $ALL
EXCLUDE "master"
SQLHOST "client name"
NBSERVER "master server name"
MAXTRANSFERSIZE 6
BLOCKSIZE 7
DUMPOPTION INCREMENTAL
NUMBUFS 2
ENDOPER TRUE 

10 REPLIES 10

lovee
Level 5

for another server getting below error:

NF - backup log "MV2Dev" to VIRTUAL_DEVICE='VNBU0-16080-15928-1501660658' with  stats = 10, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2
Aug 2, 2017 1:57:39 AM - Info dbclient (pid=16080) INF - Number of stripes: 1, Number of buffers per stripe 2.
Aug 2, 2017 1:57:40 AM - Info dbclient (pid=16080) INF - Created VDI object for SQL Server instance <ALUSS1-SD360001>. Connection timeout is <300> seconds.
Aug 2, 2017 2:12:41 AM - Info dbclient (pid=16080) ERR - Error in VxBSACreateObject: 3.
Aug 2, 2017 2:12:41 AM - Info dbclient (pid=16080)     CONTINUATION: - System detected error, operation aborted.
Aug 2, 2017 2:12:42 AM - Info dbclient (pid=16080) ERR - Error in GetCommand: 0x80770004.
Aug 2, 2017 2:12:42 AM - Info dbclient (pid=16080) DBMS MSG - ODBC return code <-1>, SQL State <37000>, SQL Message <3202><[Microsoft][SQL Server Native Client 11.0][SQL Server]Write on "VNBU0-16080-15928-1501660658" failed: 995(The I/O operation has been aborted because of either a thread exit or an application request.)>.
Aug 2, 2017 2:12:42 AM - Info dbclient (pid=16080)     CONTINUATION: - An abort request is preventing anything except termination actions.
Aug 2, 2017 2:12:42 AM - Info dbclient (pid=16080) DBMS MSG - SQL Message <3271><[Microsoft][SQL Server Native Client 11.0][SQL Server]A nonrecoverable I/O error occurred on file "VNBU0-16080-15928-1501660658:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).>
Aug 2, 2017 2:12:42 AM - Info dbclient (pid=16080) DBMS MSG - SQL Message <3013><[Microsoft][SQL Server Native Client 11.0][SQL Server]BACKUP LOG is terminating abnormally.>
Aug 2, 2017 2:12:43 AM - Info dbclient (pid=16080) ERR - Error found executing <backup log "MV2Dev" to VIRTUAL_DEVICE='VNBU0-16080-15928-1501660658' with  stats = 10, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2>.
Aug 2, 2017 2:12:43 AM - Info dbclient (pid=16080) INF - OPERATION #14 of batch Trans.bch FAILED with STATUS 1 (0 is normal). Elapsed time = 906(906) seconds.

 

As always the first question what changed from when it was to when it started failing.

Have you tried to reconnect to the SQL server through the Netbackup SQL Agent GUI on the problematic clients ?

Have seen upgrades and security patches remove this connection.

Are the relevant netbackup service user(s) member of the relevant SQL server roles like sysadmin ?

The standard questions: Have you checked: 1) What has changed. 2) The manual 3) If there are any tech notes or VOX posts regarding the issue

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Show this error to the SQL dba and ask what has changed:

An instance of SQL Server was not found on this client.

Please copy dbclient log on the client to dbclient.txt and upload here.

 

please see the attached dbclient logs.

as per DBA they didn’t seen any backup failure attempts on DB side.

 

Thiago_Ribeiro
Moderator
Moderator
Partner    VIP    Accredited

As @Marianne said before, about this message can you see the instance? You said that backup was working well, what change on last days? In the NetBackup services, is there an account with privileges necessary?

Aug 2, 2017 9:11:10 AM - Info dbclient (pid=15252) USER - An instance of SQL Server was not found on this client.

From client usssp0036.na.novartis.net > Veritas NetBackup > NetBackup Agents > NetBackup MS SQL Client, are you able to see the instance and databases??

About this error, have you had opportunity to check these TNs?

Aug 2, 2017 2:12:42 AM - Info dbclient (pid=16080) ERR - Error in GetCommand: 0x80770004.

https://www.veritas.com/support/en_US/article.000004893
https://www.veritas.com/support/en_US/article.TECH42958

 

Regards,

 

Thiago

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
You forgot to attach dbclient log.

please see attached dbclient logs.

i browse to attached log . i select the log file and then post. but seems its not attaching.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
VOX developers have fixed the issue with attachments.

I am confused by all the different hostnames seen in dbclient log.

Extract:

08:42:24.818 [13188.13788] <4> CGlobalInformation::SQLEnumerator: INF - Hosts and instances retrieved from host list string
08:42:24.818 [13188.13788] <4> CGlobalInformation::SQLEnumerator: INF - host: ALUSS1-SD360001
08:42:24.819 [13188.13788] <4> CGlobalInformation::SQLEnumerator: INF - instance: <default>
08:42:24.819 [13188.13788] <4> CGlobalInformation::SQLEnumerator: INF - host: USSSP0022
08:42:24.819 [13188.13788] <4> CGlobalInformation::SQLEnumerator: INF - instance: BKUPEXEC
08:42:24.819 [13188.13788] <4> CGlobalInformation::SQLEnumerator: INF - host: USSSP0023
08:42:24.819 [13188.13788] <4> CGlobalInformation::SQLEnumerator: INF - instance: BKUPEXEC
08:42:24.819 [13188.13788] <2> CDBbackmain::UpdateBatchFileForAllInstances: DBG - Instance not found for host usssp0036.alconnet.com, trying batch SQLHOST USSSP0036
08:42:24.819 [13188.13788] <2> CDBbackmain::UpdateBatchFileForAllInstances: DBG - Instance not found for host [USSSP0036], trying the hostname returned by windows api [USSSP0036]
08:42:24.820 [13188.13788] <16> CDBbackmain::UpdateBatchFileForAllInstances: USER - An instance of SQL Server was not found on this client.

Please help us understand where all of these hostnames fit in the picture w.r.t this SQL client:
ALUSS1-SD360001
USSSP0022
USSSP0023

Is this a Backup Exec server?

NBU35
Level 6

IN addition of Marianne's Reply, I can see following error in logs.

Aug 2, 2017 2:12:42 AM - Info dbclient (pid=16080) DBMS MSG - SQL Message <3271><[Microsoft][SQL Server Native Client 11.0][SQL Server]A nonrecoverable I/O error occurred on file "VNBU0-16080-15928-1501660658:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).

 

Generally this is related to share memory and get resolved after reboot of client.

Also i believe this client has both BackupExec and NBU components installed on it, if yes, then uninstall BAckupExec.