cancel
Showing results for 
Search instead for 
Did you mean: 

RDSM has encountered an issue with STS

jukebox11
Level 3
Certified

Hi

Getting error when connecting to Storage Server under Credentials

 STS error.PNG

 

Environment

Master Server: 8.1 Windows 2012

Media Server: 8.1 Windows 2016

The 4 TB drive is attached to Media Server and windows acan see it and we can access it from Master as well as Media Server using \\

The NetBackup would not connect to the device and would throw the above error

The communication between Master and Media is fine as Basic STU disk created on that Media Server works fine

The nbrmms log extract shows scket error:

Line 3327: 0,51216,395,222,154,1612524917266,15988,3696,0:,116:[ERROR] PDSTS: tag_nbu_dsid: PdvfsRead() failed for </xyz.com#1/2//NBU_PD_SERVER> (108:Unknown error),33:PureDisk:xyz.com:1,1
Line 3329: 0,51216,395,222,156,1612524917266,15988,3696,0:,97:[ERROR] PDSTS: pd_get_dsid_path: tag_nbu_dsid() failed (2060019:error occurred on network socket),33:PureDisk:xyz.com:1,1

The disk folder is accessible on Media Server

Folder.PNG

Any help is appreciated........

 

 

 

 

9 REPLIES 9

pats_729
Level 6
Employee
What has changed on Media Server Or how did you landed in this situation please provide more details...

Did you checked if spoold and spad are running on media server?

If possible Can you post sppold and spad logs here?

There was no change carried out on Media server or rather we not sure exactly what changed 

The SLP started failing and then we checked that the Backups to Disk too are failing

The disks are one raid attached to controller directly

Admin log is showing:

Line 720: 08:35:59.744 [18280.12812] <16> emmlib_QueryMembershipForMachine: (0) request failed, emmError = 2007031, nbError = 0
Line 721: 08:35:59.744 [18280.12812] <4> nbsvrgrp: EMM returned EMM_ERROR_SQLNoDataFound, continuing

nbdevconfig -getconfig -storage_server 

RDSM has encountered an issue with STS where the server was not found: getStorageServerConfig
failed to get storage server configuration information, invalid command parameter

nbdevquery.exe -listdp -U command shows the Storage Server as UP

nbdevquery.exe -liststs also lists the entry for the STS

Spoold nbrmms log:

08:35:17 INFO [7] [000001BD03D399C0]: Task Type : Receive
08:35:17 INFO [7] [000001BD03D399C0]: Started at : Fri Feb 5 08:35:17 2021
08:35:17 INFO [7] [000001BD03D399C0]: Task ID : 7
08:35:17 INFO [7] [000001BD03D399C0]: Thread ID : 000001BD03D399C0
08:35:17 INFO [7] [000001BD03D399C0]: Session ID : 2968502745
08:35:17 INFO [7] [000001BD03D399C0]: Data Selection : 2
08:35:17 INFO [7] [000001BD03D399C0]: Agent Name : nbrmms.exe
08:35:17 INFO [7] [000001BD03D399C0]: Agent Address : xyz.com:60745
08:35:17 INFO [7] [000001BD03D399C0]: OS Family : WIN32
08:35:17 INFO [7] [000001BD03D399C0]: Software Version : 11.0002.0017.09
08:35:17 INFO [7] [000001BD03D399C0]: Protocol Version : 6.6.1.1.1
08:35:17 INFO [7] [000001BD03D399C0]: Crypto engine : agent
08:35:17 INFO [7] [000001BD03D399C0]: Data Encryption : disabled
08:35:17 INFO [7] [000001BD03D399C0]: Data Compression : enabled
08:35:17 INFO [7] [000001BD03D399C0]: Accelerated Access: disabled
08:35:17 INFO [7] [000001BD03D399C0]: DC RefMode Support: YES
08:35:17 INFO [7] [000001BD03D399C0]: task 7 [thread 000001BD03D399C0] for xyz.com:60745: Receive(MSG_B_SESSION_66_START) handler running
08:35:17 INFO [8] [000001BD03D399C0]: Task Type : Receive
08:35:17 INFO [8] [000001BD03D399C0]: Started at : Fri Feb 5 08:35:17 2021
08:35:17 INFO [8] [000001BD03D399C0]: Task ID : 8
08:35:17 INFO [8] [000001BD03D399C0]: Thread ID : 000001BD03D399C0
08:35:17 INFO [8] [000001BD03D399C0]: Session ID : 1663253693
08:35:17 INFO [8] [000001BD03D399C0]: Data Selection : 2
08:35:17 INFO [8] [000001BD03D399C0]: Agent Name : nbrmms.exe
08:35:17 INFO [8] [000001BD03D399C0]: Agent Address : xyz.com:60749
08:35:17 INFO [8] [000001BD03D399C0]: OS Family : WIN32
08:35:17 INFO [8] [000001BD03D399C0]: Software Version : 11.0002.0017.09
08:35:17 INFO [8] [000001BD03D399C0]: Protocol Version : 6.6.1.1.1
08:35:17 INFO [8] [000001BD03D399C0]: Crypto engine : agent
08:35:17 INFO [8] [000001BD03D399C0]: Data Encryption : disabled
08:35:17 INFO [8] [000001BD03D399C0]: Data Compression : enabled
08:35:17 INFO [8] [000001BD03D399C0]: Accelerated Access: disabled
08:35:17 INFO [8] [000001BD03D399C0]: DC RefMode Support: YES
08:35:17 INFO [8] [000001BD03D399C0]: task 8 [thread 000001BD03D399C0] for xyz.com:60749: Receive(MSG_B_SESSION_66_START) handler running
08:35:17 ERR [8] [000001BD03D399C0]: -1: dcRecordGetEx: failed to get entry fcd85b32a29ef1a2201c007f72d2bf95 from container 64, it may be caused by race condition,invalidate data container index file cache.
08:35:17 INFO [7] [000001BD03D399C0]: task 7 [thread 000001BD03D399C0] for xyz.com:60745: Receive(MSG_B_SESSION_END) handler returns
08:35:17 INFO [7] [000001BD03D399C0]: Receive session shutdown
08:35:17 INFO [7] [000001BD03D399C0]: Uptime : 0.03 sec
08:35:17 INFO [7] [000001BD03D399C0]: Messages Received : 4
08:35:17 INFO [7] [000001BD03D399C0]: Bytes Received : 391
08:35:17 INFO [7] [000001BD03D399C0]: Seconds spent receiving : 0.00
08:35:17 INFO [7] [000001BD03D399C0]: Messages Sent : 3
08:35:17 INFO [7] [000001BD03D399C0]: Bytes Sent : 286
08:35:17 INFO [7] [000001BD03D399C0]: Seconds spent sending : 0.00
08:35:17 INFO [7] [000001BD03D399C0]: 0.00 MB sent in 0.03 sec
08:35:17 INFO [7] [000001BD03D399C0]: Receive completed at: Fri Feb 5 08:35:17 2021

 

 

 

SPAD log

08:35:17 INFO [10] [000001C1C0A5F2C0]: Task Type : spad
08:35:17 INFO [10] [000001C1C0A5F2C0]: Started at : Fri Feb 5 08:35:17 2021
08:35:17 INFO [10] [000001C1C0A5F2C0]: Task ID : 10
08:35:17 INFO [10] [000001C1C0A5F2C0]: Thread ID : 000001C1C0A5F2C0
08:35:17 INFO [10] [000001C1C0A5F2C0]: Session ID : 224466003
08:35:17 INFO [10] [000001C1C0A5F2C0]: Data Selection : 1
08:35:17 INFO [10] [000001C1C0A5F2C0]: Agent Name : nbrmms.exe
08:35:17 INFO [10] [000001C1C0A5F2C0]: Agent Address : xyz.com:60732
08:35:17 INFO [10] [000001C1C0A5F2C0]: OS Family : WIN32
08:35:17 INFO [10] [000001C1C0A5F2C0]: Software Version : 11.0002.0017.09
08:35:17 INFO [10] [000001C1C0A5F2C0]: Protocol Version : 6.6.1.1.1
08:35:17 INFO [10] [000001C1C0A5F2C0]: Crypto engine : server
08:35:17 INFO [10] [000001C1C0A5F2C0]: Data Encryption : enabled
08:35:17 INFO [10] [000001C1C0A5F2C0]: Encryption method : AES-256-CTR
08:35:17 INFO [10] [000001C1C0A5F2C0]: Data Compression : enabled
08:35:17 INFO [10] [000001C1C0A5F2C0]: Accelerated Access: disabled
08:35:17 INFO [10] [000001C1C0A5F2C0]: DC RefMode Support: YES
08:35:17 INFO [10] [000001C1C0A5F2C0]: task 10 [thread 000001C1C0A5F2C0] for xyz.com:60732: spad(MSG_B_SESSION_66_START) handler running
08:35:17 INFO [10] [000001C1C0A5F2C0]: Got request: &request=6&login=deduproot&passwd=*********&action=gettime
08:35:17 INFO [10] [000001C1C0A5F2C0]: Enter handler req_realtime
08:35:17 INFO [10] [000001C1C0A5F2C0]: Exit handler req_realtime
08:35:17 INFO [11] [000001C1C0A5F2C0]: Task Type : spad
08:35:17 INFO [11] [000001C1C0A5F2C0]: Started at : Fri Feb 5 08:35:17 2021
08:35:17 INFO [11] [000001C1C0A5F2C0]: Task ID : 11
08:35:17 INFO [11] [000001C1C0A5F2C0]: Thread ID : 000001C1C0A5F2C0
08:35:17 INFO [11] [000001C1C0A5F2C0]: Session ID : 3642956760
08:35:17 INFO [11] [000001C1C0A5F2C0]: Data Selection : 1
08:35:17 INFO [11] [000001C1C0A5F2C0]: Agent Name : nbrmms.exe
08:35:17 INFO [11] [000001C1C0A5F2C0]: Agent Address : xyz.com:60736
08:35:17 INFO [11] [000001C1C0A5F2C0]: OS Family : WIN32
08:35:17 INFO [11] [000001C1C0A5F2C0]: Software Version : 11.0002.0017.09
08:35:17 INFO [11] [000001C1C0A5F2C0]: Protocol Version : 6.6.1.1.1
08:35:17 INFO [11] [000001C1C0A5F2C0]: Crypto engine : server
08:35:17 INFO [11] [000001C1C0A5F2C0]: Data Encryption : enabled
08:35:17 INFO [11] [000001C1C0A5F2C0]: Encryption method : AES-256-CTR
08:35:17 INFO [11] [000001C1C0A5F2C0]: Data Compression : enabled
08:35:17 INFO [11] [000001C1C0A5F2C0]: Accelerated Access: disabled
08:35:17 INFO [11] [000001C1C0A5F2C0]: DC RefMode Support: YES
08:35:17 INFO [11] [000001C1C0A5F2C0]: task 11 [thread 000001C1C0A5F2C0] for xyz.com:60736: spad(MSG_B_SESSION_66_START) handler running
08:35:17 INFO [11] [000001C1C0A5F2C0]: Got request: &request=9&login=deduproot&passwd=*********&action=getSDKVersion
08:35:17 INFO [11] [000001C1C0A5F2C0]: Enter handler req_storagepool
08:35:17 INFO [11] [000001C1C0A5F2C0]: Exit handler req_storagepool
08:35:17 INFO [11] [000001C1C0A5F2C0]: Got request: &request=8&login=deduproot&passwd=*********&action=getAllDataselections&id=2
08:35:17 INFO [11] [000001C1C0A5F2C0]: Enter handler req_agent
08:35:17 INFO [11] [000001C1C0A5F2C0]: Exit handler req_agent
08:35:17 INFO [10] [000001C1C0A5F2C0]: Got request: &request=6&login=deduproot&passwd=*********&action=find&dataselection=2&filter=1|0||-1|-1||*|*|*|*|*|*|*|*|*|-1|-1|-1|-1|-1|-1|-1|-1|-1|-1|*
08:35:17 INFO [10] [000001C1C0A5F2C0]: Enter handler req_realtime
08:35:17 INFO [10] [000001C1C0A5F2C0]: [_handle_find] filter [1|0||-1|-1||*|*|*|*|*|*|*|*|*|-1|-1|-1|-1|-1|-1|-1|-1|-1|-1|*]
08:35:17 INFO [10] [000001C1C0A5F2C0]: locking dir [\2]
08:35:17 INFO [10] [000001C1C0A5F2C0]: DataSelection::query: start find_by_pattern path:N:\MSDP\databases\catalog\2 basename:*
08:35:17 INFO [10] [000001C1C0A5F2C0]: created PODB instance for N:\MSDP\databases\catalog\2
08:35:17 INFO [10] [000001C1C0A5F2C0]: Open SQLite database N:\MSDP\databases\catalog\2\po.db
08:35:17 INFO [10] [000001C1C0A5F2C0]: DataSelection::query: find_by_pattern return pos.size=4, status: success
08:35:17 INFO [10] [000001C1C0A5F2C0]: unlock dir [\2]
08:35:17 INFO [10] [000001C1C0A5F2C0]: Exit handler req_realtime
08:35:17 INFO [10] [000001C1C0A5F2C0]: Got request: &request=6&login=deduproot&passwd=*********&action=addlist&dataselection=2&data=@POformatVersion=2.0
2||#client_1332|1|||||0|N|0|040777|0||260|1612524917|1612524917|1612524917|1024||||0|||0|0|0|0|
08:35:17 INFO [10] [000001C1C0A5F2C0]: Enter handler req_realtime
08:35:17 INFO [10] [000001C1C0A5F2C0]: locking dirs [\2]
08:35:17 INFO [10] [000001C1C0A5F2C0]: unlock dirs [\2]
08:35:17 INFO [10] [000001C1C0A5F2C0]: Exit handler req_realtime
08:35:17 INFO [10] [000001C1C0A5F2C0]: Got request: &request=6&login=deduproot&passwd=*********&action=find&dataselection=2&filter=1|0||-1|-1|/|NBU_PD_SERVER|*|*|*|*|*|*|*|*|-1|-1|-1|-1|-1|-1|-1|-1|-1|-1|*
08:35:17 INFO [10] [000001C1C0A5F2C0]: Enter handler req_realtime
08:35:17 INFO [10] [000001C1C0A5F2C0]: [_handle_find] filter [1|0||-1|-1|/|NBU_PD_SERVER|*|*|*|*|*|*|*|*|-1|-1|-1|-1|-1|-1|-1|-1|-1|-1|*]
08:35:17 INFO [10] [000001C1C0A5F2C0]: locking dir [\2]
08:35:17 INFO [10] [000001C1C0A5F2C0]: unlock dir [\2]
08:35:17 INFO [10] [000001C1C0A5F2C0]: Exit handler req_realtime
08:35:17 INFO [10] [000001C1C0A5F2C0]: task 10 [thread 000001C1C0A5F2C0] for xyz.com:60732: spad(MSG_B_SESSION_END) handler returns
08:35:17 INFO [10] [000001C1C0A5F2C0]: Receive session shutdown
08:35:17 INFO [10] [000001C1C0A5F2C0]: Uptime : 0.22 sec
08:35:17 INFO [10] [000001C1C0A5F2C0]: Messages Received : 8
08:35:17 INFO [10] [000001C1C0A5F2C0]: Bytes Received : 1325
08:35:17 INFO [10] [000001C1C0A5F2C0]: Seconds spent receiving : 0.00
08:35:17 INFO [10] [000001C1C0A5F2C0]: Messages Sent : 7
08:35:17 INFO [10] [000001C1C0A5F2C0]: Bytes Sent : 2526
08:35:17 INFO [10] [000001C1C0A5F2C0]: Seconds spent sending : 0.00
08:35:17 INFO [10] [000001C1C0A5F2C0]: 0.00 MB sent in 0.22 sec
08:35:17 INFO [10] [000001C1C0A5F2C0]: spad completed at: Fri Feb 5 08:35:17 2021

08:35:17 INFO [11] [000001C1C0A5F2C0]: task 11 [thread 000001C1C0A5F2C0] for xyz.com:60736: spad(MSG_B_SESSION_END) handler returns
08:35:17 INFO [11] [000001C1C0A5F2C0]: Receive session shutdown
08:35:17 INFO [11] [000001C1C0A5F2C0]: Uptime : 0.20 sec
08:35:17 INFO [11] [000001C1C0A5F2C0]: Messages Received : 6
08:35:17 INFO [11] [000001C1C0A5F2C0]: Bytes Received : 621
08:35:17 INFO [11] [000001C1C0A5F2C0]: Seconds spent receiving : 0.00
08:35:17 INFO [11] [000001C1C0A5F2C0]: Messages Sent : 5
08:35:17 INFO [11] [000001C1C0A5F2C0]: Bytes Sent : 617
08:35:17 INFO [11] [000001C1C0A5F2C0]: Seconds spent sending : 0.00
08:35:17 INFO [11] [000001C1C0A5F2C0]: 0.00 MB sent in 0.20 sec
08:35:17 INFO [11] [000001C1C0A5F2C0]: spad completed at: Fri Feb 5 08:35:17 2021

 

Would there be problem with plugin? How to check that., all .dll files are present.

 

Thanks 

 

 

 

 

 

Before you noticing problems did this server went for an abrupt reboot ?

I see following error in spoold

08:35:17 ERR [8] [000001BD03D399C0]: -1: dcRecordGetEx: failed to get entry fcd85b32a29ef1a2201c007f72d2bf95 from container 64, it may be caused by race condition,invalidate data container index file cache.

Looks like some corruption. Hope you created a support case and working with them already. If not then create a high priority case and upload following evidence for tech support for a fast track resolution
Spoold
Storaged
Spad
NBSU

Besides this can you verify in event viewer that there are no errors for file system. If possible run chkdsk on file system (under tech support supervision)

Hope it helps.

Got no errors for disk in event viewer 

Still getting the disk checked

Any reference to prove that the below log is showing disk corruption

08:35:17 ERR [8] [000001BD03D399C0]: -1: dcRecordGetEx: failed to get entry fcd85b32a29ef1a2201c007f72d2bf95 from container 64, it may be caused by race condition,invalidate data container index file cache.

Hamza_H
Moderator
Moderator
   VIP   

Hello,

 

Don't know if you checked this new technote :https://www.veritas.com/support/en_US/article.100048225

it is about cloud catalyst but the error you got in the logs is the same

also you have to verify all sort of coomunications & name resolution using the command bptestnetcomm (it is a very usefull command to troubleshoot name resolution problems)

 

good luck

jukebox11
Level 3
Certified

The name resolution works fine and the disk is locally connected to Media Server.

As mentioned earlier the local disk backup works fine

We followed the Steps to verify Data Integrity on MSDP and it didnt create any Affectedbackup.lst file 

This is the result, does it make any sense

MSDP Integrity check cmd.PNG

msdp folder.PNG

 

 

Hi
Even after this exercise if issue isn’t resolved then please involve Tech Support and allow them to run msdp check. It’s a standalone utility which does a comprehensive check on all msdp containers. It will also list you any bad containers and tries to self heal (conditional).

Hi Pats,

Is this tool available in public? can i download it or copy it from SORT?

 

Hi Jukebox11
It’s not available public. Only Tech Support has the access to it.