Using Alternate Network with the new "Preferred Network" feature

[rcdauria]

Hi guys,

I have the following scenario:

-Windows Master Server
Hostname VTXMASTER01
IP 10.78.0.193 (Production Network) 

-Windows Media Server 
Hostname VTXMEDIA01
IP1 10.78.0.194 (Production Network)
IP2 66.66.66.66 (Backup Network)
IP3 67.67.67.67 (Another Backup Network I will need to use eventualy - not important right now)

-Windows 2008 Client
Hostname VTXFPS02
IP 1 10.78.0.24 (Production Network)
IP 2 66.66.66.67 (Backup Network)

Requisite is quite obvious: would like the backup to pass via the 66.66.66.0/24 network.Tricky part is: customer would not like to use hosts file solutions & etc, so I went down and took a look at the new Preferred Network settings (replaces the REQUIRED_INTERFACE and REQUIRED_NETWORK ones).

Very confusing guide, but undersood I could basically insert the 66.66.66.0/24 network as a MATCH in both media server and client, and everything would work (the specified network would be matched/used as the preferred path between the media server and the client). However, no luck: the entry was skipped and the backup went via Prodution Network. Looks like bptestnetconn does not know the 66.66.66.0/24 path:

bptestnetconn -f --prefnet -H vtxfps02 

------------------------------------------------------------------------
NBU IP_ADDRESS_FAMILY configured to use Remote Addresses:  IPv4(yes) IPv6(no)
FL:                VTXFPS02 -> 10.78.0.24                              :      0 ms SRC: ANY
------------------------------------------------------------------------

Tried changing the preferred network entry from MATCH to ONLY, but from this moment on the media server cannot connect to the client.

Any sugestions? I am using this feature the wrong way?

Thanks a lot in advance

Rafael

[RamNagalla]

you might have verified it already..however just wanted to make sure before we check something else...

does the Media server is able to ping IP address 66.66.66.67, from its backup IP. its bit tricky to find in windows , however first confirm the normal LAN communication between the media and client over the backup network is fine .( i guees this is not the case)

when you do the ping from media server to IP 66.66.66.67, make sure it is using the IP  of 66.66.66.66, and not the production IP first.

[mph999]

On media server:

PREFERRED_NETWORK = <net add target>/<CIDR> MATCH <IP to go out on>

On client

PREFERRED_NETWORK = <net add target>/<CIDR>  MATCH  <IP to go out on>

[rcdauria]

Hi guys,

Thanks for your replies.

Yes, then can ping each other normally.

@MPH999: thats exactly my setup (see below - media server 2nd IP address is actuall 66.66.66.69, not 66.66.66.67):

Rafael

[mph999]

Hmm, what does bptestbpcd show, with the rule in place.

[rcdauria]

Only production network IPs:

C:\Program Files\Veritas\NetBackup\bin\admincmd>bptestbpcd -client vtxfps02 -verbose
1 1 1
10.78.0.194:63973 -> 10.78.0.24:1556
10.78.0.194:63974 -> 10.78.0.24:1556
PEER_NAME = vtxmedia01.vantix.local
HOST_NAME = vtxfps02
CLIENT_NAME = vtxfps02
VERSION = 0x07610000
PLATFORM = win_x64
PATCH_VERSION = 7.6.1.0
SERVER_PATCH_VERSION = 7.6.1.0
MASTER_SERVER = azurenbumt01.vantix.local
EMM_SERVER = azurenbumt01.vantix.local
NB_MACHINE_TYPE = CLIENT
10.78.0.194:63975 -> 10.78.0.24:1556

C:\Program Files\Veritas\NetBackup\bin\admincmd>

[rcdauria]

Actually I've been thinking: does the Preferred Network feature relates to all communications (including "control" traffic) or only backup stream.

Because if you think about it: how will the media server know the backup network IP of the client? It's not in the DNS, it's not inserted anythere in the configuration.

Only logic would be:

-Media connects into the client using production IP network
-Media obtains the list of available IP addresses
-Media "detects" there is a second path (the 66.66.66.0/24 network)
-Media decides, via preferred network rules, that the backup will come using this network.

But that assumes the control traffic would still work on the prodution (not a problem for me).

Howeber, if the rule specifies that EVERY traffic should go via the backup network, it would be like:

-Media tries to communicate with the client using the 66.66.66.0/24 network
-Media does not know the IP address of the client in this network
-Error

Makes sense?

The guide does not make it clear, very strange.

[RamNagalla]

i understand that customer does not want to use hosts file tricks... however i would like to use it once for testing if the issue is only limited to the perfred network setting or somewhere else..

put below entry in media server host file 

66.66.66.67 VTXFPS02

and below entry in client hosts file

66.66.66.66 VTXMEDIA01

and then see how it goes with bptestbpcd and regular backup,

if it works fine.. then issue is with perfered network setting.. and if it does not work that is the area you need to look for.

[Andrew_Madsen]

In order to use the preferred network setting you need the Master server to be on the backup network as well. Also you need the client to be identified as the DNS name for the client's backup area network name in the policies and the media server as well as the master server need a DNS entry for the backup network. Otherwise the master server will initiate all traffic over the production network and that is where it will stay.

[mph999]

I'm not sure if that is correct, certainly it's not my understanding ...  If it is it is bad design ...

You may have media servers with unique backup networks to a set of clients that are complete separate from any backup network the master may be on.

However, I may well be wrong, I'll see what I can find out.   One of the guys in the US is very knowledgeable on this feature, I'll try and get hold of him.

[mph999]

This could be worth trying ...

PREFERRED_NETWORK = <target ip>  MATCH <outbound ip>

 ... use bptestbpcd to check if it goes out on <outbund ip>

It's not ideal, as it is specific to each client - but it's about as simple as this option gets, so I just want to see if it has any effect.

[rcdauria]

Hey guys,

Comments about the last replies:

@RamNagalla: yes, did that test. Everything works fine with the hosts solution. Command line shows the rigth path (via 66.66.66.0/24 network).

@Andrew: I agree with mph999 that this is weird. Demanding to have the Master Server in the backup network is totally against the product layered topology. Imagine I have 5 backup networks. What would I do?

@Mph999: Understood your point in testing like "PREFERRED_NETWORK = <target ip>  MATCH <outbound ip>"

Howeber it is still unclear to me how the Media would know the client IP address. We would have matches to every IP address on the BAN, but it actually don't know what IP is from what client. Makes sense?

Will test anyway and let you know in minutes.

Thanks,

Rafael

[rcdauria]

Ok, no luck. Here is what I did:

Media Server:

(the second entry would be an example of a seconf client)

Client:

And recycled the services in both servers.

bptestbpcd still shows the LAN connection:

C:\Program Files\Veritas\NetBackup\bin\admincmd>bptestbpcd -client vtxfps02 -verbose
1 1 1
10.78.0.194:51233 -> 10.78.0.24:1556
10.78.0.194:51234 -> 10.78.0.24:1556
PEER_NAME = vtxmedia01.vantix.local
HOST_NAME = vtxfps02
CLIENT_NAME = vtxfps02
VERSION = 0x07610000
PLATFORM = win_x64
PATCH_VERSION = 7.6.1.0
SERVER_PATCH_VERSION = 7.6.1.0
MASTER_SERVER = azurenbumt01.vantix.local
EMM_SERVER = azurenbumt01.vantix.local
NB_MACHINE_TYPE = CLIENT
10.78.0.194:51235 -> 10.78.0.24:1556

[CB_NBU]

Looking at this, what are the default routes on the client , master , media?

when using the 66.66.66.x network ,do you have a route in place on the systems to default out an alternative gateway or use the default? if default, thats your issue. I would take a look into the routing on all boxes as the potential source issue for this. 

[rcdauria]

Hi CB_NBU,

Actually they don't use a gateway, as it is a local network (both MEDIA and CLIENT are in the same layer-2 network segment).

Therefore, the only route involved is the default route the operation system creates to the local attached network (the "on-link" state - 5th line below).

Tks,
Rafael

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0        10.78.0.1      10.78.0.194    261
        10.78.0.0    255.255.255.0         On-link       10.78.0.194    261
      10.78.0.194  255.255.255.255         On-link       10.78.0.194    261
      10.78.0.255  255.255.255.255         On-link       10.78.0.194    261
       66.66.66.0    255.255.255.0         On-link       66.66.66.69    261
      66.66.66.69  255.255.255.255         On-link       66.66.66.69    261
     66.66.66.255  255.255.255.255         On-link       66.66.66.69    261
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link       10.78.0.194    261
        224.0.0.0        240.0.0.0         On-link       66.66.66.69    261
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
  255.255.255.255  255.255.255.255         On-link       10.78.0.194    261
  255.255.255.255  255.255.255.255         On-link       66.66.66.69    261
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
          0.0.0.0          0.0.0.0        10.78.0.1  Default

[mph999]

Hmm, I'll have a play - I have to build a cluster for some testing, once finished, I'll install a client on one node, and I'll have a couple of interfaces for testing.

[mph999]

... another option of course, is static routes ...

[rcdauria]

You mean like

route add 10.78.0.24/32 gw 66.66.66.67 and vice versa?

Tried that also, but it didn't work - the destination S.O. discards the packets at it "is not a router". For it to work I would have to bring up a RRAS in every windows (ugly solution) 

Thanks a lot for your help, buddy. Please let me know if you find something in your lab.

Rafael

[mph999]

Yes, like that ...

Wondering if this is going to come down to something environmental - I can;t find any eTracks where PREFERRED_NETWORK has a 'bug', though will look again.  Believe it or not, the actual workings involve quite a bit of OS .... for example ...

You can get issues (seen in Linux) under certain conditions where using a PROHIBITED rule makes bptestbpcd show a connection is made between two networks that have no connectivity.

I found that by accident, it was a right one to get your head round (took three of us if I recall correctly), but ultimately, came down to the OS.

http://www.symantec.com/docs/TECH203313

Anyways, I have dropped a note to my collegue in the US, not sure he is in today as he's not showing up on IM, but I sent him mail.

Wondering if we are going to need to look at something like wireshark, though traceroute may help.

Looks like you are on Windows ???

[rcdauria]

Yes, all Windows (thats a lab).

On production, media server will be a Netbackup 5230 Appliance.

But the simptoms are all the same (we've tried first in the production and decided to go back to the lab when noticed it wasn't clear how it works).

Rafael