Forum Discussion

Muzaffar_Ansari's avatar
6 years ago

MS SQL Server DB backups failing with "Error: Failed to query database metadata for..."

Our MS SQL Server DB backups scheduled in NetBackup were running successfully till few days back when they suddenly started failing.

FYI... We're using the legacy/traditional "bch" script based method for taking DB backups as many of our SQL Servers are configured as Availability Group Clusters.

I checked the dbclient logs and found the below errors,

13:38:12.797 [20988.19888] <2> check_vnetd_process_dup_permission: OpenProcessToken() failed: 5
13:52:13.069 [20988.19888] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <ODBC Driver 13 for SQL Server>
13:52:13.073 [20988.19888] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <SQL Server>
13:56:20.245 [15484.17208] <16> CODBCaccess::LogODBCerr: DBMS MSG - ODBC return code <-1>, SQL State <28000>, SQL Message <18456><[Microsoft][SQL Server Native Client 11.0][SQL Server]Login failed for user 'sa'.>.
13:56:22.133 [15484.17208] <4> CGlobalInformation::CreateDSN: INF - Failed to connect. Attempting operation with next installed ODBC Driver <ODBC Driver 13 for SQL Server>
13:56:22.267 [15484.17208] <16> CODBCaccess::LogODBCerr: DBMS MSG - ODBC return code <-1>, SQL State <28000>, SQL Message <18456><[Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Login failed for user 'sa'.>.
13:56:22.403 [15484.17208] <4> CGlobalInformation::CreateDSN: INF - Failed to connect. Attempting operation with next installed ODBC Driver <SQL Server>
13:56:54.486 [15484.17208] <16> CODBCaccess::LogODBCerr: DBMS MSG - ODBC return code <-1>, SQL State <08001>, SQL Message <18><[Microsoft][ODBC SQL Server Driver][DBNETLIB]SSL Security error>.
13:56:56.072 [15484.17208] <16> CDBbackmain::LoadUserLogin: ERR - Internal error. See the dbclient log for more information.
13:56:56.131 [15484.17208] <32> CDBbackmain::LoadUserLogin: ERR - Error 64 has been returned from GetCurrentDSN
13:56:56.132 [15484.17208] <4> CDBbackmain::dbbackup: INF - Results of executing <C:\Program Files\Veritas\NetBackup\DbExt\MsSql\All_DB_Full.bch>: <0> operations succeeded. <0> operations failed.
13:57:16.202 [15484.17208] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <ODBC Driver 13 for SQL Server>
13:57:16.202 [15484.17208] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <SQL Server>

When I shared the error "Login failed for user 'sa'" with the SQL team, they said they've done hardening for user "sa" and so now all backups have to be scheduled by user "sysbackup". So I updated the connection properties of NetBackup SQL Client with "sysbackup" user credentials.

After that the "Login failed for user 'sa'" error stopped but was still getting the below errors,

17:08:48.370 [1936.19556] <2> check_vnetd_process_dup_permission: OpenProcessToken() failed: 5
17:14:11.001 [1936.19556] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <ODBC Driver 13 for SQL Server>
17:14:11.005 [1936.19556] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <SQL Server>
17:14:59.264 [14700.7380] <2> check_vnetd_process_dup_permission: OpenProcessToken() failed: 5
17:15:04.732 [14700.7380] <8> CDBbackupView::ClassDropDownList: WARN - Error: bppllist not found in C:\Program Files\Veritas\NetBackup\bin\admincmd\bppllist.exe.
17:15:16.035 [14700.7380] <8> CDBbackupView::ClassDropDownList: WARN - Error: bppllist not found in C:\Program Files\Veritas\NetBackup\bin\admincmd\bppllist.exe.
17:15:50.710 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <master>
17:15:52.731 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <model>
17:15:54.745 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <msdb>
17:15:56.759 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <ReportDB>
17:15:58.774 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <ReportDBTTSL>
17:16:00.787 [9780.15764] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <SQLPerfMon>
17:16:02.788 [9780.15764] <4> CDBbackmain::dbbackup: INF - Results of executing <C:\Program Files\Veritas\NetBackup\dbext\mssql\temp\__17_15_50_219_00.bch>: <0> operations succeeded. <6> operations failed.
17:16:51.977 [14700.7380] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <ODBC Driver 13 for SQL Server>
17:16:51.980 [14700.7380] <4> CGlobalInformation::DeleteDSN: INF - Failed to remove DSN. Attempting operation with next installed ODBC Driver <SQL Server>

I explored a lot and came to know that the errors were related to permission issues and found that the "NetBackup Client" and "NetBackup Legacy Client" services on the client system have to be run with a user having sufficient privileges.

So I changed both the services' "Log On As" user from "Local System" to the Domain user configured as local Administrator, which we're using for SQL administration purpose.

After that most of the errors ceased but no matter how much I try the below errors still remains.

16:15:23.689 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <master>
16:15:27.750 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <model>
16:15:33.811 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <msdb>
16:15:37.870 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <ReportDB>
16:15:41.926 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <ReportDBTTSL>
16:15:45.974 [8860.20884] <4> CDBbackrec::CDBbackrec: INF - Error: Failed to query database metadata for <SQLPerfMon>
16:15:50.008 [8860.20884] <4> CDBbackmain::dbbackup: INF - Results of executing <C:\Program Files\Veritas\NetBackup\DbExt\MsSql\All_DB_Full.bch>: <0> operations succeeded. <6> operations failed.

When I run the backup locally on the client system using NetBackup SQL Client utility, I still get the same set of errors as below,

INF - Error: Failed to query database metadata for <master>
INF - Error: Failed to query database metadata for <model>
INF - Error: Failed to query database metadata for <msdb>
INF - Error: Failed to query database metadata for <ReportDB>
INF - Error: Failed to query database metadata for <ReportDBTTSL>
INF - Error: Failed to query database metadata for <SQLPerfMon>
INF - Results of executing <C:\Program Files\Veritas\NetBackup\dbext\mssql\temp\__22_24_31_032_00.bch>: <0> operations succeeded. <6> operations failed.

  • Hi,

    The error got resolved... :-)

    The issue was that earlier we were using 'sa' user for taking backups which were running successfully.

    Then as a part of hardening the SQL team created a new user 'sysbackup' and asked to replace the 'sa' user with it. Since then all the backups started failing.

    I told the SQL team that there's some permission short for 'sysbackup' user but they said they're able to fire backups successfully from their end and hence it's not a permission issue.

    I opened a ticket with Veritas support and after more than an hour of struggling the support engineer changed the user in NetBackup client's connection properties to the domain user which we use for logging and recreated the backup scrpt and the backup executed successfully.

    After comparing permissions of both the domain and 'sysbackup' user and it was discovered that the domain user has the role 'sysadmin' assigned to it.

    Now the SQL team is saying they cannot assign 'sysadmin' role to 'sysbackup' user and also cannot allow backups to run with the domain user which is a security violation and also the domain users' password changes every 30 days in which case we'll have to update the connection properties ever month.

    Is there any way where we can schedule backups with the 'sysbackup' user without assigning 'sysadmin' role to it?

  • Hi,

    The error got resolved... :-)

    The issue was that earlier we were using 'sa' user for taking backups which were running successfully.

    Then as a part of hardening the SQL team created a new user 'sysbackup' and asked to replace the 'sa' user with it. Since then all the backups started failing.

    I told the SQL team that there's some permission short for 'sysbackup' user but they said they're able to fire backups successfully from their end and hence it's not a permission issue.

    I opened a ticket with Veritas support and after more than an hour of struggling the support engineer changed the user in NetBackup client's connection properties to the domain user which we use for logging and recreated the backup scrpt and the backup executed successfully.

    After comparing permissions of both the domain and 'sysbackup' user and it was discovered that the domain user has the role 'sysadmin' assigned to it.

    Now the SQL team is saying they cannot assign 'sysadmin' role to 'sysbackup' user and also cannot allow backups to run with the domain user which is a security violation and also the domain users' password changes every 30 days in which case we'll have to update the connection properties ever month.

    Is there any way where we can schedule backups with the 'sysbackup' user without assigning 'sysadmin' role to it?