Forum Discussion

Frank_Eriksen's avatar
Frank_Eriksen
Level 3
14 years ago
Solved

Using Backup Exec 2010 R2 across differenet subnets through a firewall

I have a very odd problem with my newly installed backup Exec 2010 R2 SP1 (all fixes applied) on a 2008 STD R2 SP1 64-bit media server. I'll try to explain it below:

My network is segmented into different subnets (differet IP-ranges and gateways, same subnet mask) for security reasons and all traffic between them are routed through a firewall. The backup server is located in a subnet of its own. The firewall rules allows the media server to contact any servers that need backup, and the appropriate ports are open. This has been verified with a firewall specialist as well.

The problem: "a file backup will work 1 out of 6 times on small amount of data, up to 4 GB. When trying to backup system state, exchange database (50GB) it never completes. And the point of failure or reason is never the same. Unless you put the media server on the same subnet as the server being backed up."

The backup process normally starts fine (Through the firewall), and at a good rate of approx 2GB/min. But after a few minutes it will in 70-80% cases drop speed and finally hit 0MB/min. On the same subnet it completes the job without issues with speed upto 8GB/min. The backup job can not be cancelled either, have to restart media server services to cancel all jobs.

Some further findings/information:

Furthermore, my network is IPv4 exclusive and don’t support IPv6. Tried to run the backup job exclusively with IPv4 in the job setup, this however fails.

Mailserver: Exchange 2010 SP1 Rollup2 64-bit

Agent: Updated 64-bit (Updated and same version as media server)
 
In reference to a technical reference guide regarding firewalls, I’ve also tried to limit the number of ports to 25 (1025 to 1050).

Filebackup via Backup Exec sometimes work, but run right after it fails. Failure at different stages in the backup. It’s not consistent at all.

  • Ranging from before data is transferred (snapshots) to right upon completion.
  • Port 1025 on mail server closes during backup but are still open on backup server under a running backup job.( when running netstat on both servers)
  • Microsoft filecopy between the servers (diff subnets) work with a sustained load of 33 MB/sec (file share)
  • Double checked FW rules: Everything opened.
  • Backupserver and mailserver on same subnet still works
  • Port 6101 closes immediately during backup, is this correct? We’ve only seen it once during our test runs (approx 20)
  • Using netstat –b -n

This is the case with any kid of backup job; fil, system state, mail db etc.

This case has also been submitted to Symantec Backup Exec Advanced support team.

Any ideas or solutions?

Frank

2010
Agents
Backing Up
Backup and Recovery
Backup Exec
Backup Exec (media server only)
Error messages
Remote Agent for Windows Servers
  • Frank_Eriksen's avatar
    Frank_Eriksen
    14 years ago

    The matter has now been resolved:

    Newest driver on the mailserver, and followed instructions on this page: http://support.microsoft.com/kb/951037

    And I believe the disabling NetDMA in Windows Server 2008 was the solution. I did this on both the mailserver and media server. This because I believe the media server ran full in memory and by disabling this it were resolved. Furthermore I’ve added some registry parameters under HKLM/System/CurrentControlSet/Services/TCPIP/Parameters on the media server and mailserver;

    DisableTaskOffload=1 (DWORD32)

    EnableTCPA=0 (DWORD32)

    This article also helped a lot, to determine what settings to enable and disable: http://www.speedguide.net/articles/windows-7-vista-2008-tweaks-2574

    Frank

     

  • pkh's avatar
    pkh
    Moderator
    14 years ago

    There is a known problem that the backup of Server 2008 system state takes a long time.  This could result in your backup timing out.

    Have you tried to backup your Exchange on its own, i.e., without backing up the system state in the same job?

  • Frank_Eriksen's avatar
    Frank_Eriksen
    Level 3
    14 years ago

    The Exchange and System state jobs are two different jobs. And not running at the same time. The only thing i try to back up is Exchange 2010 Sp1 Rollup2 information store.

    I'm really curious though, to why the port (1025) being used for backup suddenly times out on the server being backed up and not the media server.

  • pkh's avatar
    pkh
    Moderator
    14 years ago

    Just a long shot, check the file system of the disks where your B2D folders reside.  If it is FAT32, then each file is limited to 4GB.  To have bigger file sizes, you need to convert them to NTFS.  

  • Frank_Eriksen's avatar
    Frank_Eriksen
    Level 3
    14 years ago

    The drives are NTFS. But thanks for the tip.

    Backup works fine in the same subnet, and files are split into 20GB a piece.

  • pkh's avatar
    pkh
    Moderator
    14 years ago

    Are you using GRT to backup your Exchange?  If so, try turning off the GRT.

  • Colin_Weaver's avatar
    Colin_Weaver
    Moderator
    14 years ago

    OK we have to open a control port and a data port (NDMP Protocol)

    The control port is port 10000 - the data port can be in a configurable range - but it must be a range and must have some scope so we recommend at least 50 in the range. (Configurable in Tools --> Options --> Network and Security "Enable Remote Agent TCP dynamic port range. Obviously whatever you set here has to match the firewall.

    Also (and particular for a slow backup that takes a long time) if for any reason there is no need to send control information for an extended period of time - then the keep Alive Settings against the firewall might shut down the control connection while the data connection remains active. When this happens one end of the BE process is unaware that the connection has been dropped and can then generate comms/timeout errors in the environment when it tries to communicate on the control connection.

     

  • Frank_Eriksen's avatar
    Frank_Eriksen
    Level 3
    14 years ago

    GRT has been run both on and of, same result.

    The firewall only acts as a router, with all ports opened. Still no luck.

    For examle:

    The snapshot provider used by VSS for volume F: - Microsoft Software Shadow Copy provider 1.0 (Version 1.0.0.7).
    Network control connection is established between 192.168.150.xx:50540 <--> 192.168.153.xx:10000
    Network data connection is established between    192.168.150.xx:50548 <--> 192.168.153.xx:1025

    This one failed with port 1025 port dropped on mailserver on a diff subnet.

    The next one completed without errors:

    Processed 4 294 903 966 bytes in  2 minutes and  7 seconds.
    Throughput rate: 1935 MB/min
    Compression Type: None
     

    One more thing: Backup work nicely with earlier versions of Backup exec (ie. 12.5d) with the same network setup.
     

     


     

  • Frank_Eriksen's avatar
    Frank_Eriksen
    Level 3
    14 years ago

    The matter has now been resolved:

    Newest driver on the mailserver, and followed instructions on this page: http://support.microsoft.com/kb/951037

    And I believe the disabling NetDMA in Windows Server 2008 was the solution. I did this on both the mailserver and media server. This because I believe the media server ran full in memory and by disabling this it were resolved. Furthermore I’ve added some registry parameters under HKLM/System/CurrentControlSet/Services/TCPIP/Parameters on the media server and mailserver;

    DisableTaskOffload=1 (DWORD32)

    EnableTCPA=0 (DWORD32)

    This article also helped a lot, to determine what settings to enable and disable: http://www.speedguide.net/articles/windows-7-vista-2008-tweaks-2574

    Frank