Forum Discussion

Martin_Barringe's avatar
13 years ago

SSL Handshakes fails

I am having problems with Backup Exec 2010 R3 SP1 with all updated as of (18th December) being able to view Enterprise Vault v9 SP2.

If I remove and reinstall backupexec the SSL hand shake works for a week and then fails again and no backups.

This is the only server affected from 2 Media Servers both running Windows 2008 R2 SP1.  If I use a 2003 SP2 X86 I can see the data partitions and all is working correctly.

Here is a extract from the BE Debug Logs

BENETNS:  [12/18/11 14:09:12] [0000]     12/18/11 14:09:12 [nrds]               - Accepted new connection.
BENETNS:  [12/18/11 14:09:12] [0000]     12/18/11 14:09:12 [nrds]               - AcceptConnection: SSL was requested
BENETNS:  [12/18/11 14:09:12] [0000]     12/18/11 14:09:12 [nrds]               - AcceptConnection: Failed Server Side SSL handshake.
BENETNS:  [12/18/11 14:09:12] [0000]     12/18/11 14:09:12 [BESocket]           - @@@@@@@MyCloseSocket called with sockfd = 692(0x2b4) retval = 0
BENETNS:  [12/18/11 14:09:12] [0000]     12/18/11 14:09:12 [nrds]               - Accepted new connection.
BENETNS:  [12/18/11 14:09:12] [0000]     12/18/11 14:09:12 [nrds]               - AcceptConnection: SSL was requested
BENETNS:  [12/18/11 14:09:13] [0000]     12/18/11 14:09:12 [nrds]               - AcceptConnection: Failed Server Side SSL handshake.
BENETNS:  [12/18/11 14:09:13] [0000]     12/18/11 14:09:12 [BESocket]           - @@@@@@@MyCloseSocket called with sockfd = 692(0x2b4) retval = 0
 

I have confirmed the trusted certificate is on the vault server and name resolution is working correctly.

Any suggestions would be helpful

  • These certificates were introduced in Backup Exec 2010 R3 after 3rd party specialists in IT security analysis notifed Symantec of a potential for a type of security breach between a media server and a remote agent that is known as a "Man in the Middle" attack.

    You can disable this functionality with a registry change, however if you do this you will open up a potential security flaw so will need to take other steps to ensure that your security is not compromised. As such it becomes a "use at your own risk" option and should really only be used as a short term workaround for an issue that Symantec are already investigating. If you use it for an issue we are unaware of then obviously we will never fix the issue. We are aware of current problems with the TLS Handshaking that is affecting publishing and other functionality with Backup Exec, as part of

    http://www.symantec.com/docs/TECH168154

    As such if any customer uses the details provided below as a workaround,  the changes should be undone once notification of a full solution of the issue has been made public. EDIT:  In fact we have received some feedback that using this registry change for one backup and then removing it again allows the systems to contuinue working correctly with the security enabled.

     

    Warning: Incorrect use of the Windows registry editor may prevent the operating system from functioning properly. Great care should be taken when making changes to a Windows registry. Registry modifications should only be carried-out by persons experienced in the use of the registry editor application. It is recommended that a complete backup of the registry and workstation be made prior to making any registry changes

     

    Create a DWORD value in

    HKLM\SOFTWARE\Symantec\Backup Exec For Windows\Backup Exec\Engine\Agents

    called

    Security Disabled

    Set the value to 1

     

    This must be done on media server and remote server and the Backup Exec services on the servers will need restarting after the change.

     

    Final Update....

    Just for info the Security Handshaking issues should now be resolved by Hotfix 180429 As such you should not need to disable security as a workaround if this Hotfix is installed to the Media Server and the remote agents have been updated since the Hotfix was installed.

    Also if you are using the Security Disabled workaround you should be able to re-enable it after applying the Hotfix.

12 Replies