MS-SQL 2000 Backups fail with Status code 2, operation type "Unknown"

[justmirsk]

Hello Everyone!
         I hope that someone can assist me with my SQL problem.  I have a server that I have never been able to get SQL backups to run on.  I have given my netbackup service account DB_BackupOperator role in SQL for all DB's and have started the Netbackup service as this account.  I have specified the SQL authentication in the client and generated my BCH files via the GUI interface.  I am running Windows Server 2000 with SQL 2000.  My Master/media servers are running on RHEL 5.  Netbackup is version 6.5.3.

When I run my backups on this server the job fails after getting to "connected" in the detailed status.  If I launch the GUI on the server and go to file-->View Status, the job shows up as operation type "unknown" and eventually fails with operation status "Error (-1)"  The verbose output from the progress window shows:

BATCHSIZE 4
OPERATION BACKUP
DATABASE "adpeet"
SQLHOST "ETIMEDB.mydomain.com"
NBSERVER "USLOUNETBACK1.mydomain.com"
MAXTRANSFERSIZE 6
BLOCKSIZE 7
NUMBUFS 2
ENDOPER TRUE
OPERATION BACKUP
DATABASE "master"
SQLHOST "ETIMEDB.mydomain.com"
NBSERVER "USLOUNETBACK1.mydomain.com"
MAXTRANSFERSIZE 6
BLOCKSIZE 7
NUMBUFS 2
ENDOPER TRUE
OPERATION BACKUP
DATABASE "model"
SQLHOST "ETIMEDB.mydomain.com"
NBSERVER "USLOUNETBACK1.mydomain.com"
MAXTRANSFERSIZE 6
BLOCKSIZE 7
NUMBUFS 2
ENDOPER TRUE
OPERATION BACKUP
DATABASE "msdb"
SQLHOST "ETIMEDB.mydomain.com"
NBSERVER "USLOUNETBACK1.mydomain.com"
MAXTRANSFERSIZE 6
BLOCKSIZE 7
NUMBUFS 2
ENDOPER TRUE

ERR - Error in CDBbackrec::CDBbackrec: 6.
CONTINUATION: - The system cannot find the file specified.
ERR - Error in VxBSASetEnv(BSA_SERVICE_HOST): 6.
CONTINUATION: - The handle used to associate this call with a previous VxBSAInit() call is invalid.
ERR - Error in CDBbackrec::CDBbackrec: 6.
CONTINUATION: - The system cannot find the file specified.
ERR - Error in VxBSASetEnv(BSA_SERVICE_HOST): 6.
CONTINUATION: - The handle used to associate this call with a previous VxBSAInit() call is invalid.
ERR - Error in CDBbackrec::CDBbackrec: 6.
CONTINUATION: - The system cannot find the file specified.
ERR - Error in VxBSASetEnv(BSA_SERVICE_HOST): 6.
CONTINUATION: - The handle used to associate this call with a previous VxBSAInit() call is invalid.
ERR - Error in CDBbackrec::CDBbackrec: 6.
CONTINUATION: - The system cannot find the file specified.
ERR - Error in VxBSASetEnv(BSA_SERVICE_HOST): 6.
CONTINUATION: - The handle used to associate this call with a previous VxBSAInit() call is invalid.
INF - Results of executing <etimedb_default.bch>:
<0> operations succeeded. <4> operations failed.
INF - The following object(s) were not backed up successfully.
INF - adpeet
INF - master
INF - model
INF - msdb



I have read online that the message "The system cannot find the file specified" might be due to the service account not having access to the directory, however I know it has access to the directory as it is calling up this file and pulling information from it.  Please help!  I really need to get these backups going in Netbackup as soon as I can.  Please let me know if any further information is required.  

Thanks!

[Marianne]

Please share dbclient log on client. Create log directory if it does not exist and retry backup.
Also see the NetBackup for SQL Admin Guide for recommendations regarding NetBackup Client Service logon account.

[justmirsk]

Hello Marianne,
       Thanks for the response.  I have posted the log file info below.  I have followed the instructions for service accounts from the SQL admin guide, however I am going through it one more time to see if I missed anything.  I have SQL backups working on 4 other SQL 2005 boxes using the same methods as I have used on this server.  

15:36:21.575 [2516.1200] <2> init_cache: ..\libvlibs\vnet_hosts.c.990: host_cache_size: 200 0x000000c8
15:36:21.575 [2516.1200] <2> init_cache: ..\libvlibs\vnet_hosts.c.991: cache_time: 3600 0x00000e10
15:36:21.575 [2516.1200] <2> init_cache: ..\libvlibs\vnet_hosts.c.1003: host_failed_cache_size: 40 0x00000028
15:36:21.575 [2516.1200] <2> init_cache: ..\libvlibs\vnet_hosts.c.1004: cache_time: 3600 0x00000e10
15:36:21.575 [2516.1200] <2> init_cache: ..\libvlibs\vnet_hosts.c.990: host_cache_size: 200 0x000000c8
15:36:21.575 [2516.1200] <2> init_cache: ..\libvlibs\vnet_hosts.c.991: cache_time: 3600 0x00000e10
15:36:21.575 [2516.1200] <2> init_cache: ..\libvlibs\vnet_hosts.c.1003: host_failed_cache_size: 40 0x00000028
15:36:21.575 [2516.1200] <2> init_cache: ..\libvlibs\vnet_hosts.c.1004: cache_time: 3600 0x00000e10
15:36:21.685 [2516.1200] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2043: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
15:36:21.685 [2516.1200] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2057: service: bprd
15:36:21.778 [2516.1200] <2> logconnections: BPRD CONNECT FROM 10.232.10.18.1387 TO 10.232.55.201.13724
15:36:21.888 [2516.1200] <4> CGlobalInformation::VCSVirtualNameList: INF - Veritas Cluster Server is not installed.
15:36:21.888 [2516.1200] <1> CGlobalInformation::VCSVirtualNameList: CONTINUATION: - The system cannot find the path specified.
15:36:21.888 [2516.1200] <4> getServerName: Read server name from nb_master_config: uscucnetback1
15:36:21.888 [2516.1200] <4> CDBIniParms::CDBIniParms: INF - NT User is netbackup
15:36:21.950 [2516.1200] <4> DBConnect: INF - Logging into SQL Server with DSN <NBMSSQL_2516_1200>, SQL userid <sa> handle <0x00e51a20>.
15:36:21.997 [2516.1200] <4> CGlobalInformation::CreateDSN: INF - A successful connection to SQL Server <ETIMEDB.safetran.railad.com\> has been made using standard security with DSN <NBMSSQL_2516_1200>
15:36:21.997 [2516.1200] <4> DBDisconnect: INF - Logging out of SQL Server with handle <0x00e51a20>
15:36:21.997 [2516.1200] <4> DBConnect: INF - Logging into SQL Server with DSN <NBMSSQL_2516_1200>, SQL userid <sa> handle <0x00e51a20>.
15:36:22.028 [2516.1200] <4> DBDisconnect: INF - Logging out of SQL Server with handle <0x00e51a20>
15:36:22.028 [2516.1200] <4> DBConnect: INF - Logging into SQL Server with DSN <NBMSSQL_2516_1200>, SQL userid <sa> handle <0x00e51a20>.
15:36:22.075 [2516.1200] <4> DBDisconnect: INF - Logging out of SQL Server with handle <0x00e51a20>
15:36:22.075 [2516.1200] <4> CDBbackcat::GetVirtualServer: INF - Accessing database instance <ETIMEDB.safetran.railad.com> on server <ETIMEDB.safetran.railad.com>.
15:36:22.075 [2516.1200] <4> DBConnect: INF - Logging into SQL Server with DSN <NBMSSQL_2516_1200>, SQL userid <sa> handle <0x00e51a20>.
15:36:22.263 [2516.1200] <4> DBDisconnect: INF - Logging out of SQL Server with handle <0x00e51a20>
15:36:22.278 [2516.1200] <16> CProgress::CreateProgress: ERR - Internal error. See the dbclient log for more information.
15:36:22.278 [2516.1200] <32> CProgress::CreateProgress: ERR - Could not create directory: C:\Program Files\Veritas\NetBackup\Logs\user_ops\mssql\jobs\
15:36:22.278 [2516.1200] <1> CProgress::CreateProgress: CONTINUATION: - The system cannot find the path specified.
15:36:22.278 [2516.1200] <4> CDBbackmain::dbbackup: INF - Results of executing <etimedb.bch>: <0> operations succeeded. <0> operations failed.
15:36:54.903 [652.2648] <2> init_cache: ..\libvlibs\vnet_hosts.c.990: host_cache_size: 200 0x000000c8
15:36:54.903 [652.2648] <2> init_cache: ..\libvlibs\vnet_hosts.c.991: cache_time: 3600 0x00000e10
15:36:54.903 [652.2648] <2> init_cache: ..\libvlibs\vnet_hosts.c.1003: host_failed_cache_size: 40 0x00000028
15:36:54.903 [652.2648] <2> init_cache: ..\libvlibs\vnet_hosts.c.1004: cache_time: 3600 0x00000e10
15:36:54.903 [652.2648] <2> init_cache: ..\libvlibs\vnet_hosts.c.990: host_cache_size: 200 0x000000c8
15:36:54.903 [652.2648] <2> init_cache: ..\libvlibs\vnet_hosts.c.991: cache_time: 3600 0x00000e10
15:36:54.903 [652.2648] <2> init_cache: ..\libvlibs\vnet_hosts.c.1003: host_failed_cache_size: 40 0x00000028
15:36:54.903 [652.2648] <2> init_cache: ..\libvlibs\vnet_hosts.c.1004: cache_time: 3600 0x00000e10
15:36:55.012 [652.2648] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2043: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
15:36:55.012 [652.2648] <2> vnet_vnetd_service_socket: ..\libvlibs\vnet_vnetd.c.2057: service: bprd
15:36:55.106 [652.2648] <2> logconnections: BPRD CONNECT FROM 10.232.10.18.1395 TO 10.232.55.201.13724
15:36:55.215 [652.2648] <4> CGlobalInformation::VCSVirtualNameList: INF - Veritas Cluster Server is not installed.
15:36:55.215 [652.2648] <1> CGlobalInformation::VCSVirtualNameList: CONTINUATION: - The system cannot find the path specified.
15:36:55.215 [652.2648] <4> getServerName: Read server name from nb_master_config: uscucnetback1
15:36:55.215 [652.2648] <4> CDBIniParms::CDBIniParms: INF - NT User is serviceadmin
15:36:55.293 [652.2648] <4> DBConnect: INF - Logging into SQL Server with DSN <NBMSSQL_temp_652_2648>, SQL userid <safetran\netbackup> handle <0x01161a20>.
15:36:55.325 [652.2648] <4> CGlobalInformation::CreateDSN: INF - A successful connection to SQL Server <ETIMEDB\> has been made using standard security with DSN <NBMSSQL_temp_652_2648>
15:36:55.325 [652.2648] <4> DBDisconnect: INF - Logging out of SQL Server with handle <0x01161a20>
15:36:55.325 [652.2648] <4> DBConnect: INF - Logging into SQL Server with DSN <NBMSSQL_temp_652_2648>, SQL userid <safetran\netbackup> handle <0x01161a20>.
15:36:55.372 [652.2648] <4> DBDisconnect: INF - Logging out of SQL Server with handle <0x01161a20>
15:37:37.152 [652.2648] <4> StartupProcess: INF - Starting: <"C:\Program Files\Veritas\NetBackup\bin\progress" MSSQL>

[Marianne]

I assume you have verified that user 'serviceadmin' can create files/folders in  C:\Program Files\Veritas\NetBackup\Logs\user_ops\mssql\jobs?

The only errors in dbclient seem to be these 2 lines (not very informative!):
15:36:22.278 [2516.1200] <16> CProgress::CreateProgress: ERR - Internal error. See the dbclient log for more information.
15:36:22.278 [2516.1200] <32> CProgress::CreateProgress: ERR - Could not create directory: C:\Program Files\Veritas\NetBackup\Logs\user_ops\mssql\jobs\

Anything useful in SQL Errorlog or Event viewer Application log?

[justmirsk]

Hello,
     I have verified that "Serviceadmin" is able to create files and folders in the mentioned directory.  I have been going through the event logs and the only error I am seeing is in the System event log.  The event ID is 7031 and the error message is "Symantec Private Branch Exchange service terminated unexpectedly." 

    I tried the backup again and viewed the SQL logs however there are no errors or warnings in the logs for the time of the backups. 

[rj_nbu]

BSA_SERVICE_HOST error


Chekc the hosts file entries, batch file to make sure that the hostname of the master server is correct, and hostname to ip resolution is doen correctly

[justmirsk]

Hello Rajeev,
       Thanks for the post.  I don't believe that name resolution is the issue as file level backups on this server work just fine.  Is there something different for name resolution with SQL backups than file level backups?  I am going through the name resolution troubleshooting guide just to verify again.

[rj_nbu · Accepted solution]

Hi


In the dbclient logs, this is the master servername returned

Read server name from nb_master_config: uscucnetback1


IN your batch file, this is the master server name configured

NBSERVER "USLOUNETBACK1.mydomain.com"

What does C:\Program Files\veritas\Netbackup\bin\bpclntcmd -pn      return ?

[justmirsk]

Rajeev,
     Thanks for the information.  My problem was that I had created the backup script file while the SQL client was set to use my media server not the master server.  I had since changed it to use the master server, however I did not update the script file.  After updating the script file to use the master server name the backups are now running.  Thank you very much for you assistance. 

JustMirsk