SQL backup exits status 1 - all child streams successful
After finally getting my SQL backup to connect and start writing data, I noticed that all of the child streams completed successfully, yet the parent job exited with status 1. I've checked the dbclient log, as well as the SQL ERRLOG, and they are not very specific as to why they are not being backed up. The master/model, and several other databases that have been created on the system backed up fine.
dbclient logs (note, I've scrubbed the db names, host names, and IP's and replaced with substitutions)
dbclient logs (note, I've scrubbed the db names, host names, and IP's and replaced with substitutions)
16:16:38.593 [7792.160] <16> VxBSACreateObject: ERR - Could not create new image with file /physical_hostname.MSSQL7.SQL_INSTANCE.db.DB_NAME.~.7.001of001.20100323160122..C.
16:16:38.593 [7792.160] <4> getOwnerName: entering getOwnerName.
16:16:38.593 [7792.160] <4> dbc_GetServerName: entering dbc_GetServerName.
16:16:38.593 [7792.160] <4> getServerName: entering getServerName.
16:16:38.593 [7792.160] <4> getServerName: Read server name from nb_master_config: master_server
16:16:38.593 [7792.160] <4> dbc_GetClientName: entering dbc_GetClientName.
16:16:38.593 [7792.160] <4> dbc_GetClientName: Returning NB client name: physical_hostname-bur
16:16:38.593 [7792.160] <4> getOwnerName: entering getOwnerName.
16:16:38.593 [7792.160] <4> dbc_GetServerName: entering dbc_GetServerName.
16:16:38.593 [7792.160] <4> getServerName: entering getServerName.
16:16:38.593 [7792.160] <4> getServerName: Read server name from nb_master_config: master_server
16:16:38.593 [7792.160] <4> dbc_GetClientName: entering dbc_GetClientName.
16:16:38.593 [7792.160] <4> dbc_GetClientName: Returning NB client name: physical_hostname-bur
16:16:38.593 [7792.160] <4> dbc_RemoteWriteFile: master_server root 37 logs/mssql_backup_failures/032310.rpt a 1
16:16:38.687 [7792.160] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2046: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
16:16:38.687 [7792.160] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2060: service: bprd
16:16:38.750 [7792.160] <2> logconnections: BPRD CONNECT FROM physical_IP.1389 TO master_IP.13724
16:16:39.234 [7792.160] <4> dbc_RemoteWriteFile: INF - RemoteWriteFile status = 12
16:16:39.234 [7792.160] <4> getOwnerName: entering getOwnerName.
16:16:39.234 [7792.160] <4> dbc_GetServerName: entering dbc_GetServerName.
16:16:39.234 [7792.160] <4> getServerName: entering getServerName.
16:16:39.234 [7792.160] <4> getServerName: Read server name from nb_master_config: master_server
16:16:39.234 [7792.160] <4> dbc_GetClientName: entering dbc_GetClientName.
16:16:39.234 [7792.160] <4> dbc_GetClientName: Returning NB client name: physical_hostname-bur
16:16:38.687 [7792.160] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2060: service: bprd
16:16:38.750 [7792.160] <2> logconnections: BPRD CONNECT FROM physical_IP.1389 TO master_IP.13724
16:16:39.234 [7792.160] <4> dbc_RemoteWriteFile: INF - RemoteWriteFile status = 12
16:16:39.234 [7792.160] <4> getOwnerName: entering getOwnerName.
16:16:39.234 [7792.160] <4> dbc_GetServerName: entering dbc_GetServerName.
16:16:39.234 [7792.160] <4> getServerName: entering getServerName.
16:16:39.234 [7792.160] <4> getServerName: Read server name from nb_master_config: master_server
16:16:39.234 [7792.160] <4> dbc_GetClientName: entering dbc_GetClientName.
16:16:39.234 [7792.160] <4> dbc_GetClientName: Returning NB client name: physical_hostname-bur
16:16:39.234 [7792.160] <4> dbc_RemoteWriteFile: physical_hostname-bur root 37 logs/mssql_backup_failures/032310.rpt a 1
16:16:39.328 [7792.160] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2046: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
16:16:39.328 [7792.160] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2060: service: bprd
16:16:39.390 [7792.160] <2> logconnections: BPRD CONNECT FROM physical_IP.1391 TO master_IP.13724
16:16:40.171 [7792.160] <4> dbc_RemoteWriteFile: INF - RemoteWriteFile status = 0
16:16:40.171 [7792.160] <16> DBthreads::dbclient: ERR - Error in VxBSACreateObject: 3.
16:16:40.171 [7792.160] <1> DBthreads::dbclient: CONTINUATION: - System detected error, operation aborted.
16:16:40.171 [7792.160] <4> VxBSAEndTxn: INF - entering VxBSAEndTxn.
16:16:40.171 [7792.160] <4> VxBSAEndTxn: INF - Transaction being ABORTED.
16:16:40.171 [7792.160] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_LOG_DIRECTORY
16:16:40.171 [7792.160] <4> VxBSAGetEnv: INF - returning - DBCLIENT
16:16:40.171 [7792.160] <4> VxBSAEndTxn: INF - Cleaning directory: <C:\Program Files\Veritas\NetBackup\Logs\DBCLIENT>
16:16:40.171 [7792.160] <4> delete_old_files: entering delete_old_files.
16:16:40.171 [7792.160] <8> close_image: Session being terminated abnormally, cleaning up
16:16:40.171 [7792.160] <4> closeApi: entering closeApi.
16:16:40.171 [7792.160] <4> closeApi: INF - EXIT STATUS 6: the backup failed to back up the requested files
16:16:39.328 [7792.160] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2060: service: bprd
16:16:39.390 [7792.160] <2> logconnections: BPRD CONNECT FROM physical_IP.1391 TO master_IP.13724
16:16:40.171 [7792.160] <4> dbc_RemoteWriteFile: INF - RemoteWriteFile status = 0
16:16:40.171 [7792.160] <16> DBthreads::dbclient: ERR - Error in VxBSACreateObject: 3.
16:16:40.171 [7792.160] <1> DBthreads::dbclient: CONTINUATION: - System detected error, operation aborted.
16:16:40.171 [7792.160] <4> VxBSAEndTxn: INF - entering VxBSAEndTxn.
16:16:40.171 [7792.160] <4> VxBSAEndTxn: INF - Transaction being ABORTED.
16:16:40.171 [7792.160] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_LOG_DIRECTORY
16:16:40.171 [7792.160] <4> VxBSAGetEnv: INF - returning - DBCLIENT
16:16:40.171 [7792.160] <4> VxBSAEndTxn: INF - Cleaning directory: <C:\Program Files\Veritas\NetBackup\Logs\DBCLIENT>
16:16:40.171 [7792.160] <4> delete_old_files: entering delete_old_files.
16:16:40.171 [7792.160] <8> close_image: Session being terminated abnormally, cleaning up
16:16:40.171 [7792.160] <4> closeApi: entering closeApi.
16:16:40.171 [7792.160] <4> closeApi: INF - EXIT STATUS 6: the backup failed to back up the requested files
16:16:40.171 [7792.160] <4> close_image: INF - backup FAILED
16:16:40.171 [7792.160] <4> close_image: INF ---- end of Backup ---
16:16:40.171 [7792.160] <4> close_image: INF ---- end of Backup ---
16:16:40.171 [7792.160] <4> VxBSATerminate: INF - entering VxBSATerminate.
16:16:40.171 [7792.160] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_DEBUGFD
16:16:40.171 [7792.160] <4> VxBSAGetEnv: INF - returning -
16:16:40.171 [7792.160] <4> KillAllThreads: INF - Killing group #0
16:16:40.171 [7792.160] <4> KillAllThreads: INF - Issuing SignalAbort to MS SQL Server VDI
16:16:40.171 [7792.5676] <16> ReadFromVirtualDevice: ERR - Error in GetCommand: 0x80770004.
16:16:40.171 [7792.5676] <1> ReadFromVirtualDevice: CONTINUATION: - An abort request is preventing anything except termination actions.
16:16:40.171 [7792.5676] <4> KillAllThreads: INF - Killing group #0
16:16:40.187 [7792.7312] <4> KillAllThreads: INF - Killing group #0
16:16:40.187 [7792.5740] <16> CODBCaccess::LogODBCerr: DBMS MSG - ODBC return code <-1>, SQL State <37000>, SQL Message <3202><[Microsoft][SQL Native Client][SQL Server]Write on "VNBU0-7792-5740-1269360083" failed: 995(The I/O operation has been aborted because of either a thread exit or an application request.)>.
16:16:40.187 [7792.5740] <16> CODBCaccess::LogODBCerr: DBMS MSG - SQL Message <3271><[Microsoft][SQL Native Client][SQL Server]A nonrecoverable I/O error occurred on file "VNBU0-7792-5740-1269360083:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).>
16:16:40.187 [7792.5740] <16> CODBCaccess::LogODBCerr: DBMS MSG - SQL Message <3013><[Microsoft][SQL Native Client][SQL Server]BACKUP DATABASE is terminating abnormally.>
16:16:40.187 [7792.5740] <16> Dbbackrec::PerformNBOperation: ERR - Error found executing <backup database "DATABASE_NAME" to VIRTUAL_DEVICE='VNBU0-7792-5740-1269360083' with stats = 10, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2>.
16:16:40.187 [7792.5740] <4> DBDisconnect: INF - Logging out of SQL Server with handle <0x01a772f0>
16:16:40.187 [7792.5740] <4> getOwnerName: entering getOwnerName.
16:16:40.187 [7792.5740] <4> dbc_GetServerName: entering dbc_GetServerName.
16:16:40.187 [7792.5740] <4> getServerName: entering getServerName.
16:16:40.187 [7792.5740] <4> getServerName: Read server name from nb_master_config: master_server
16:16:40.187 [7792.5740] <4> dbc_GetClientName: entering dbc_GetClientName.
16:16:40.187 [7792.5740] <4> dbc_GetClientName: Returning NB client name: physical_hostname-bur
16:16:40.171 [7792.160] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_DEBUGFD
16:16:40.171 [7792.160] <4> VxBSAGetEnv: INF - returning -
16:16:40.171 [7792.160] <4> KillAllThreads: INF - Killing group #0
16:16:40.171 [7792.160] <4> KillAllThreads: INF - Issuing SignalAbort to MS SQL Server VDI
16:16:40.171 [7792.5676] <16> ReadFromVirtualDevice: ERR - Error in GetCommand: 0x80770004.
16:16:40.171 [7792.5676] <1> ReadFromVirtualDevice: CONTINUATION: - An abort request is preventing anything except termination actions.
16:16:40.171 [7792.5676] <4> KillAllThreads: INF - Killing group #0
16:16:40.187 [7792.7312] <4> KillAllThreads: INF - Killing group #0
16:16:40.187 [7792.5740] <16> CODBCaccess::LogODBCerr: DBMS MSG - ODBC return code <-1>, SQL State <37000>, SQL Message <3202><[Microsoft][SQL Native Client][SQL Server]Write on "VNBU0-7792-5740-1269360083" failed: 995(The I/O operation has been aborted because of either a thread exit or an application request.)>.
16:16:40.187 [7792.5740] <16> CODBCaccess::LogODBCerr: DBMS MSG - SQL Message <3271><[Microsoft][SQL Native Client][SQL Server]A nonrecoverable I/O error occurred on file "VNBU0-7792-5740-1269360083:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).>
16:16:40.187 [7792.5740] <16> CODBCaccess::LogODBCerr: DBMS MSG - SQL Message <3013><[Microsoft][SQL Native Client][SQL Server]BACKUP DATABASE is terminating abnormally.>
16:16:40.187 [7792.5740] <16> Dbbackrec::PerformNBOperation: ERR - Error found executing <backup database "DATABASE_NAME" to VIRTUAL_DEVICE='VNBU0-7792-5740-1269360083' with stats = 10, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2>.
16:16:40.187 [7792.5740] <4> DBDisconnect: INF - Logging out of SQL Server with handle <0x01a772f0>
16:16:40.187 [7792.5740] <4> getOwnerName: entering getOwnerName.
16:16:40.187 [7792.5740] <4> dbc_GetServerName: entering dbc_GetServerName.
16:16:40.187 [7792.5740] <4> getServerName: entering getServerName.
16:16:40.187 [7792.5740] <4> getServerName: Read server name from nb_master_config: master_server
16:16:40.187 [7792.5740] <4> dbc_GetClientName: entering dbc_GetClientName.
16:16:40.187 [7792.5740] <4> dbc_GetClientName: Returning NB client name: physical_hostname-bur
16:16:40.187 [7792.5740] <4> dbc_RemoteWriteFile: physical_hostname-bur root 37 logs/mssql_backup_failures/032310.rpt a 1023 TIMESTAMP 20100323161640
MASTERSERVER master_server
CLIENT physical_hostname
INSTANCE SQL_INSTANCE
DATABASE DATABASE_NAME
OPERATION BACKUP
OBJECTTYPE DATABASE
STATUS -1
INFO Server Status: Communication with the server has not been initiated or the server status has not been retrieved from the server.
INFO Error in VxBSACreateObject: 3.
INFO System detected error, operation aborted.
INFO Error in GetCommand: 0x80770004.
INFO An abort request is preventing anything except termination actions.
INFO ODBC return code <-1>, SQL State <37000>, SQL Message <3202><[Microsoft][SQL Native Client][SQL Server]Write on "VNBU0-7792-5740-1269360083" failed: 995(The I/O operation has been aborted because of either a thread exit or an application request.)>.
INFO SQL Message <3271><[Microsoft][SQL Native Client][SQL Server]A nonrecoverable I/O error occurred on file "VNBU0-7792-5740-1269360083:" 995(The...
16:16:40.265 [7792.5740] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2046: VN_REQUEST_SERVICE_SOCKET: 6 0x0000000616:01:23.593 [7792.160] <2> logconnections: BPRD CONNECT FROM physical_IP.4733 TO master_IP.13724
16:01:23.593 [7792.160] <4> serverResponse: entering serverResponse.
16:01:23.593 [7792.160] <4> serverResponse: initial client_read_timeout = <900>
16:01:23.593 [7792.160] <4> readCommMessages: Entering readCommMessages
16:01:33.593 [7792.160] <4> serverResponse: 16:01:26 Initiating backup
16:01:33.593 [7792.160] <4> readCommMessages: Entering readCommMessages
16:16:38.593 [7792.160] <16> readCommFile: ERR - timed out after 900 seconds while reading from C:\Program Files\Veritas\NetBackup\Logs\user_ops\mssql\logs\0323110160120-7792-5964-000-000-prg
16:16:38.593 [7792.160] <32> serverResponse: ERR - could not read from comm file <C:\Program Files\Veritas\NetBackup\Logs\user_ops\mssql\logs\0323110160120-7792-5964-000-000-prg>
16:16:38.593 [7792.160] <16> CreateNewImage: ERR - serverResponse() failed
16:16:38.593 [7792.160] <4> closeApi: entering closeApi.
16:16:38.593 [7792.160] <4> closeApi: INF - EXIT STATUS 6: the backup failed to back up the requested files
And this is where the errors begin in the SQL ERRLOG:
2010-03-22 18:47:27.84 Logon Error: 18452, Severity: 14, State: 1.
2010-03-22 18:47:27.84 Logon Login failed for user 'sa'. The user is not associated with a trusted SQL Server connection. [CLIENT: CLIENT_IP]
2010-03-22 18:58:48.50 Logon Error: 18452, Severity: 14, State: 1.
2010-03-22 18:58:48.50 Logon Login failed for user 'sa'. The user is not associated with a trusted SQL Server connection. [CLIENT: CLIENT_IP]
2010-03-22 20:06:47.25 spid54 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:06:47.25 spid54 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-8108-1269287491'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:06:47.25 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 20:06:47.25 Backup BACKUP failed to complete the command BACKUP DATABASE DATABASE_1. Check the backup application log for detailed messages.
2010-03-22 20:06:47.25 spid54 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:06:47.25 spid54 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-8108-1269287491'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:22:09.28 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 20:22:09.28 Backup BACKUP failed to complete the command BACKUP DATABASE DATABASE_2. Check the backup application log for detailed messages.
2010-03-22 20:22:09.28 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:22:09.28 spid53 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-2748-1269288413'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:22:09.28 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:22:09.28 spid53 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-2748-1269288413'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:37:31.29 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 20:37:31.29 Backup BACKUP failed to complete the command BACKUP DATABASE Failed_DATABASE. Check the backup application log for detailed messages.
2010-03-22 20:37:31.29 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:37:31.29 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-6608-1269289335'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:37:31.29 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:37:31.29 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-6608-1269289335'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:52:50.42 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:52:50.42 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-1052-1269290255'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:52:50.42 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 20:52:50.42 Backup BACKUP failed to complete the command BACKUP DATABASE master. Check the backup application log for detailed messages.
2010-03-22 20:52:50.42 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:52:50.42 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-1052-1269290255'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:08:09.45 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 21:08:09.45 Backup BACKUP failed to complete the command BACKUP DATABASE model. Check the backup application log for detailed messages.
2010-03-22 21:08:09.45 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:08:09.45 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-2892-1269291174'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:08:09.45 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:08:09.45 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-2892-1269291174'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:23:28.53 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 21:23:28.53 Backup BACKUP failed to complete the command BACKUP DATABASE msdb. Check the backup application log for detailed messages.
2010-03-22 21:23:28.53 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:23:28.53 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-7644-1269292093'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:23:28.53 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:23:28.53 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-7644-1269292093'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:38:47.67 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 21:38:47.67 Backup BACKUP failed to complete the command BACKUP DATABASE RP_DB. Check the backup application log for detailed messages.
2010-03-22 21:38:47.67 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:38:47.67 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-6068-1269293012'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:38:47.67 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:38:47.67 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-6068-1269293012'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:54:06.71 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 21:54:06.71 Backup BACKUP failed to complete the command BACKUP DATABASE RQM_DB. Check the backup application log for detailed messages.
2010-03-22 21:54:06.71 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:54:06.71 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-7992-1269293931'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:54:06.71 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:54:06.71 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-7992-1269293931'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 22:09:25.75 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 22:09:25.75 Backup BACKUP failed to complete the command BACKUP DATABASE RTC_DB. Check the backup application log for detailed messages.
2010-03-22 22:09:25.75 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 22:09:25.75 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-6060-1269294850'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 22:09:25.75 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 22:09:25.75 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-6060-1269294850'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 00:00:49.90 spid17s This instance of SQL Server has been using a process ID of 6792 since 2/22/2010 10:46:36 PM (local) 2/22/2010 10:46:36 PM (UTC). This is an informational message only; no user action is required.
2010-03-23 16:16:40.17 Backup Error: 3041, Severity: 16, State: 1.
2010-03-23 16:16:40.17 Backup BACKUP failed to complete the command BACKUP DATABASE DATABASE_1. Check the backup application log for detailed messages.
2010-03-23 16:16:40.17 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:16:40.17 spid53 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-7792-5740-1269360083'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:16:40.17 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:16:40.17 spid53 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-7792-5740-1269360083'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:32:01.35 Backup Error: 3041, Severity: 16, State: 1.
2010-03-23 16:32:01.35 Backup BACKUP failed to complete the command BACKUP DATABASE DATABASE_2. Check the backup application log for detailed messages.
2010-03-23 16:32:01.35 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:32:01.35 spid53 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-7792-6804-1269361005'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:32:01.36 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:32:01.36 spid53 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-7792-6804-1269361005'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:37:10.09 Backup Database backed up. Database: DATABASE_1, creation date(time): 2010/03/18(15:37:30), pages dumped: 600, first LSN: 18:221:130, last LSN: 18:277:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'VNBU0-7092-660-1269362074'}). This is an informational message only. No user action is required.
2010-03-23 16:38:04.91 Backup Database backed up. Database: DATABASE_2, creation date(time): 2010/03/18(15:43:05), pages dumped: 600, first LSN: 18:209:124, last LSN: 18:263:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'VNBU0-7092-5408-1269362254'}). This is an informational message only. No user action is required.
2010-03-23 16:46:09.00 Backup Database backed up. Database: master, creation date(time): 2010/02/22(22:46:35), pages dumped: 600, first LSN: 207:16:93, last LSN: 207:64:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'VNBU0-7092-7172-1269362618'}). This is an informational message only. No user action is required.
2010-03-23 16:47:03.83 Backup Database backed up. Database: model, creation date(time): 2003/04/08(09:13:36), pages dumped: 600, first LSN: 18:16:70, last LSN: 18:56:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'VNBU0-7092-5136-1269362793'}). This is an informational message only. No user action is required.
2010-03-23 16:47:22.53 Backup Error: 3041, Severity: 16, State: 1.
2010-03-23 16:47:22.53 Backup BACKUP failed to complete the command BACKUP DATABASE Failed_DATABASE. Check the backup application log for detailed messages.
2010-03-23 16:47:22.53 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:47:22.53 spid53 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-7792-7428-1269361927'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:47:22.53 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:47:22.53 spid53 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-7792-7428-1269361927'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:47:22.53 Backup Error: 3201, Severity: 16, State: 7.
2010-03-23 16:47:22.53 Backup Cannot open backup device 'VNBU0-7092-2176-1269362309'. Operating system error 0x80070002(The system cannot find the file specified.).
2010-03-23 16:47:22.53 Backup Error: 3041, Severity: 16, State: 1.
2010-03-23 16:47:22.53 Backup BACKUP failed to complete the command BACKUP DATABASE Failed_DATABASE. Check the backup application log for detailed messages.
One thing I'm wondering is what are the different types of modes a database can be in that could cause issues with the backup?
MASTERSERVER master_server
CLIENT physical_hostname
INSTANCE SQL_INSTANCE
DATABASE DATABASE_NAME
OPERATION BACKUP
OBJECTTYPE DATABASE
STATUS -1
INFO Server Status: Communication with the server has not been initiated or the server status has not been retrieved from the server.
INFO Error in VxBSACreateObject: 3.
INFO System detected error, operation aborted.
INFO Error in GetCommand: 0x80770004.
INFO An abort request is preventing anything except termination actions.
INFO ODBC return code <-1>, SQL State <37000>, SQL Message <3202><[Microsoft][SQL Native Client][SQL Server]Write on "VNBU0-7792-5740-1269360083" failed: 995(The I/O operation has been aborted because of either a thread exit or an application request.)>.
INFO SQL Message <3271><[Microsoft][SQL Native Client][SQL Server]A nonrecoverable I/O error occurred on file "VNBU0-7792-5740-1269360083:" 995(The...
16:16:40.265 [7792.5740] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2046: VN_REQUEST_SERVICE_SOCKET: 6 0x0000000616:01:23.593 [7792.160] <2> logconnections: BPRD CONNECT FROM physical_IP.4733 TO master_IP.13724
16:01:23.593 [7792.160] <4> serverResponse: entering serverResponse.
16:01:23.593 [7792.160] <4> serverResponse: initial client_read_timeout = <900>
16:01:23.593 [7792.160] <4> readCommMessages: Entering readCommMessages
16:01:33.593 [7792.160] <4> serverResponse: 16:01:26 Initiating backup
16:01:33.593 [7792.160] <4> readCommMessages: Entering readCommMessages
16:16:38.593 [7792.160] <16> readCommFile: ERR - timed out after 900 seconds while reading from C:\Program Files\Veritas\NetBackup\Logs\user_ops\mssql\logs\0323110160120-7792-5964-000-000-prg
16:16:38.593 [7792.160] <32> serverResponse: ERR - could not read from comm file <C:\Program Files\Veritas\NetBackup\Logs\user_ops\mssql\logs\0323110160120-7792-5964-000-000-prg>
16:16:38.593 [7792.160] <16> CreateNewImage: ERR - serverResponse() failed
16:16:38.593 [7792.160] <4> closeApi: entering closeApi.
16:16:38.593 [7792.160] <4> closeApi: INF - EXIT STATUS 6: the backup failed to back up the requested files
And this is where the errors begin in the SQL ERRLOG:
2010-03-22 18:47:27.84 Logon Error: 18452, Severity: 14, State: 1.
2010-03-22 18:47:27.84 Logon Login failed for user 'sa'. The user is not associated with a trusted SQL Server connection. [CLIENT: CLIENT_IP]
2010-03-22 18:58:48.50 Logon Error: 18452, Severity: 14, State: 1.
2010-03-22 18:58:48.50 Logon Login failed for user 'sa'. The user is not associated with a trusted SQL Server connection. [CLIENT: CLIENT_IP]
2010-03-22 20:06:47.25 spid54 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:06:47.25 spid54 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-8108-1269287491'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:06:47.25 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 20:06:47.25 Backup BACKUP failed to complete the command BACKUP DATABASE DATABASE_1. Check the backup application log for detailed messages.
2010-03-22 20:06:47.25 spid54 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:06:47.25 spid54 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-8108-1269287491'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:22:09.28 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 20:22:09.28 Backup BACKUP failed to complete the command BACKUP DATABASE DATABASE_2. Check the backup application log for detailed messages.
2010-03-22 20:22:09.28 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:22:09.28 spid53 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-2748-1269288413'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:22:09.28 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:22:09.28 spid53 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-2748-1269288413'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:37:31.29 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 20:37:31.29 Backup BACKUP failed to complete the command BACKUP DATABASE Failed_DATABASE. Check the backup application log for detailed messages.
2010-03-22 20:37:31.29 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:37:31.29 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-6608-1269289335'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:37:31.29 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:37:31.29 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-6608-1269289335'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:52:50.42 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:52:50.42 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-1052-1269290255'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 20:52:50.42 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 20:52:50.42 Backup BACKUP failed to complete the command BACKUP DATABASE master. Check the backup application log for detailed messages.
2010-03-22 20:52:50.42 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 20:52:50.42 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-1052-1269290255'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:08:09.45 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 21:08:09.45 Backup BACKUP failed to complete the command BACKUP DATABASE model. Check the backup application log for detailed messages.
2010-03-22 21:08:09.45 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:08:09.45 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-2892-1269291174'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:08:09.45 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:08:09.45 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-2892-1269291174'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:23:28.53 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 21:23:28.53 Backup BACKUP failed to complete the command BACKUP DATABASE msdb. Check the backup application log for detailed messages.
2010-03-22 21:23:28.53 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:23:28.53 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-7644-1269292093'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:23:28.53 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:23:28.53 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-7644-1269292093'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:38:47.67 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 21:38:47.67 Backup BACKUP failed to complete the command BACKUP DATABASE RP_DB. Check the backup application log for detailed messages.
2010-03-22 21:38:47.67 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:38:47.67 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-6068-1269293012'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:38:47.67 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:38:47.67 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-6068-1269293012'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:54:06.71 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 21:54:06.71 Backup BACKUP failed to complete the command BACKUP DATABASE RQM_DB. Check the backup application log for detailed messages.
2010-03-22 21:54:06.71 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:54:06.71 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-7992-1269293931'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 21:54:06.71 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 21:54:06.71 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-7992-1269293931'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 22:09:25.75 Backup Error: 3041, Severity: 16, State: 1.
2010-03-22 22:09:25.75 Backup BACKUP failed to complete the command BACKUP DATABASE RTC_DB. Check the backup application log for detailed messages.
2010-03-22 22:09:25.75 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 22:09:25.75 spid55 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-8024-6060-1269294850'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-22 22:09:25.75 spid55 Error: 18210, Severity: 16, State: 1.
2010-03-22 22:09:25.75 spid55 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-8024-6060-1269294850'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 00:00:49.90 spid17s This instance of SQL Server has been using a process ID of 6792 since 2/22/2010 10:46:36 PM (local) 2/22/2010 10:46:36 PM (UTC). This is an informational message only; no user action is required.
2010-03-23 16:16:40.17 Backup Error: 3041, Severity: 16, State: 1.
2010-03-23 16:16:40.17 Backup BACKUP failed to complete the command BACKUP DATABASE DATABASE_1. Check the backup application log for detailed messages.
2010-03-23 16:16:40.17 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:16:40.17 spid53 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-7792-5740-1269360083'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:16:40.17 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:16:40.17 spid53 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-7792-5740-1269360083'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:32:01.35 Backup Error: 3041, Severity: 16, State: 1.
2010-03-23 16:32:01.35 Backup BACKUP failed to complete the command BACKUP DATABASE DATABASE_2. Check the backup application log for detailed messages.
2010-03-23 16:32:01.35 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:32:01.35 spid53 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-7792-6804-1269361005'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:32:01.36 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:32:01.36 spid53 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-7792-6804-1269361005'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:37:10.09 Backup Database backed up. Database: DATABASE_1, creation date(time): 2010/03/18(15:37:30), pages dumped: 600, first LSN: 18:221:130, last LSN: 18:277:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'VNBU0-7092-660-1269362074'}). This is an informational message only. No user action is required.
2010-03-23 16:38:04.91 Backup Database backed up. Database: DATABASE_2, creation date(time): 2010/03/18(15:43:05), pages dumped: 600, first LSN: 18:209:124, last LSN: 18:263:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'VNBU0-7092-5408-1269362254'}). This is an informational message only. No user action is required.
2010-03-23 16:46:09.00 Backup Database backed up. Database: master, creation date(time): 2010/02/22(22:46:35), pages dumped: 600, first LSN: 207:16:93, last LSN: 207:64:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'VNBU0-7092-7172-1269362618'}). This is an informational message only. No user action is required.
2010-03-23 16:47:03.83 Backup Database backed up. Database: model, creation date(time): 2003/04/08(09:13:36), pages dumped: 600, first LSN: 18:16:70, last LSN: 18:56:1, number of dump devices: 1, device information: (FILE=1, TYPE=VIRTUAL_DEVICE: {'VNBU0-7092-5136-1269362793'}). This is an informational message only. No user action is required.
2010-03-23 16:47:22.53 Backup Error: 3041, Severity: 16, State: 1.
2010-03-23 16:47:22.53 Backup BACKUP failed to complete the command BACKUP DATABASE Failed_DATABASE. Check the backup application log for detailed messages.
2010-03-23 16:47:22.53 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:47:22.53 spid53 BackupMedium::ReportIoError: write failure on backup device 'VNBU0-7792-7428-1269361927'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:47:22.53 spid53 Error: 18210, Severity: 16, State: 1.
2010-03-23 16:47:22.53 spid53 BackupVirtualDeviceFile::RequestDurableMedia: Flush failure on backup device 'VNBU0-7792-7428-1269361927'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
2010-03-23 16:47:22.53 Backup Error: 3201, Severity: 16, State: 7.
2010-03-23 16:47:22.53 Backup Cannot open backup device 'VNBU0-7092-2176-1269362309'. Operating system error 0x80070002(The system cannot find the file specified.).
2010-03-23 16:47:22.53 Backup Error: 3041, Severity: 16, State: 1.
2010-03-23 16:47:22.53 Backup BACKUP failed to complete the command BACKUP DATABASE Failed_DATABASE. Check the backup application log for detailed messages.
One thing I'm wondering is what are the different types of modes a database can be in that could cause issues with the backup?
- The SQL-Server automatic backup job initiates the execution of the batch file which queues one or more application backup jobs. If any application jobs fail and the NUMRETRIES keyword is specified in the batch file, then the failed jobs will retry and may be successful. When this condition occurs, the automatic job will complete with a status 1 even if all of the application jobs have retried and completed with status 0.This is normal behavior. The purpose of an exit status 1 under these circumstances is to alert the administrator that there was a potential issue with the overall backup, even though the end result was success. The administrator may choose to ignore the exit status 1, if the cause of the retry is acceptable in that particular backup environment.
http://seer.entsupport.symantec.com/docs/316629.htm
