MS SQL Server DB backups failing with "Error: Failed to query database metadata for..."
Our MS SQL Server DB backups scheduled in NetBackup were running successfully till few days back when they suddenly started failing. FYI... We're using the legacy/traditional "bch" script based method for taking DB backups as many of our SQL Servers are configured as Availability Group Clusters. I checked the dbclient logs and found the below errors, 13:38:12.797 [20988.19888] <2> check_vnetd_process_dup_permission: OpenProcessToken() failed: 5 13:52:13.069 [20988.19888] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <ODBC Driver 13 for SQL Server> 13:52:13.073 [20988.19888] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <SQL Server> 13:56:20.245 [15484.17208] <16> CODBCaccess::LogODBCerr: DBMS MSG - ODBC return code <-1>, SQL State <28000>, SQL Message <18456><[Microsoft][SQL Server Native Client 11.0][SQL Server]Login failed for user 'sa'.>. 13:56:22.133 [15484.17208] <4> CGlobalInformation::CreateDSN: INF - Failed to connect. Attempting operation with next installed ODBC Driver <ODBC Driver 13 for SQL Server> 13:56:22.267 [15484.17208] <16> CODBCaccess::LogODBCerr: DBMS MSG - ODBC return code <-1>, SQL State <28000>, SQL Message <18456><[Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Login failed for user 'sa'.>. 13:56:22.403 [15484.17208] <4> CGlobalInformation::CreateDSN: INF - Failed to connect. Attempting operation with next installed ODBC Driver <SQL Server> 13:56:54.486 [15484.17208] <16> CODBCaccess::LogODBCerr: DBMS MSG - ODBC return code <-1>, SQL State <08001>, SQL Message <18><[Microsoft][ODBC SQL Server Driver][DBNETLIB]SSL Security error>. 13:56:56.072 [15484.17208] <16> CDBbackmain::LoadUserLogin: ERR - Internal error. See the dbclient log for more information. 13:56:56.131 [15484.17208] <32> CDBbackmain::LoadUserLogin: ERR - Error 64 has been returned from GetCurrentDSN 13:56:56.132 [15484.17208] <4> CDBbackmain::dbbackup: INF - Results of executing <C:\Program Files\Veritas\NetBackup\DbExt\MsSql\All_DB_Full.bch>: <0> operations succeeded. <0> operations failed. 13:57:16.202 [15484.17208] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <ODBC Driver 13 for SQL Server> 13:57:16.202 [15484.17208] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <SQL Server> When I shared the error "Login failed for user 'sa'" with the SQL team, they said they've done hardening for user "sa" and so now all backups have to be scheduled by user "sysbackup". So I updated the connection properties of NetBackup SQL Client with "sysbackup" user credentials. After that the "Login failed for user 'sa'" error stopped but was still getting the below errors, 17:08:48.370 [1936.19556] <2> check_vnetd_process_dup_permission: OpenProcessToken() failed: 5 17:14:11.001 [1936.19556] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <ODBC Driver 13 for SQL Server> 17:14:11.005 [1936.19556] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <SQL Server> 17:14:59.264 [14700.7380] <2> check_vnetd_process_dup_permission: OpenProcessToken() failed: 5 17:15:04.732 [14700.7380] <8> CDBbackupView::ClassDropDownList: WARN - Error: bppllist not found in C:\Program Files\Veritas\NetBackup\bin\admincmd\bppllist.exe. 17:15:16.035 [14700.7380] <8> CDBbackupView::ClassDropDownList: WARN - Error: bppllist not found in C:\Program Files\Veritas\NetBackup\bin\admincmd\bppllist.exe. 17:15:50.710 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <master> 17:15:52.731 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <model> 17:15:54.745 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <msdb> 17:15:56.759 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <ReportDB> 17:15:58.774 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <ReportDBTTSL> 17:16:00.787 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <SQLPerfMon> 17:16:02.788 [9780.15764] <4> CDBbackmain::dbbackup: INF - Results of executing <C:\Program Files\Veritas\NetBackup\dbext\mssql\temp\__17_15_50_219_00.bch>: <0> operations succeeded. <6> operations failed. 17:16:51.977 [14700.7380] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <ODBC Driver 13 for SQL Server> 17:16:51.980 [14700.7380] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <SQL Server> I explored a lot and came to know that the errors were related to permission issues and found that the "NetBackup Client" and "NetBackup Legacy Client" services on the client system have to be run with a user having sufficient privileges. So I changed both the services' "Log On As" user from "Local System" to the Domain user configured as local Administrator, which we're using for SQL administration purpose. After that most of the errors ceased but no matter how much I try the below errors still remains. 16:15:23.689 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <master> 16:15:27.750 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <model> 16:15:33.811 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <msdb> 16:15:37.870 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <ReportDB> 16:15:41.926 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <ReportDBTTSL> 16:15:45.974 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <SQLPerfMon> 16:15:50.008 [8860.20884] <4> CDBbackmain::dbbackup: INF - Results of executing <C:\Program Files\Veritas\NetBackup\DbExt\MsSql\All_DB_Full.bch>: <0> operations succeeded. <6> operations failed. When I run the backup locally on the client system usingNetBackup SQL Client utility, I still get the same set of errors as below, INF - Error: Failed to query database metadata for <master> INF - Error: Failed to query database metadata for <model> INF - Error: Failed to query database metadata for <msdb> INF - Error: Failed to query database metadata for <ReportDB> INF - Error: Failed to query database metadata for <ReportDBTTSL> INF - Error: Failed to query database metadata for <SQLPerfMon> INF - Results of executing <C:\Program Files\Veritas\NetBackup\dbext\mssql\temp\__22_24_31_032_00.bch>: <0> operations succeeded. <6> operations failed.Solved9.2KViews0likes2Comments