bptestbpcd not showing expected results

[LT2000]

running NB 6.5.6, with a Solaris 10 master and Linux media servers. We recently had an issue communicating to several clients on a backup network.

The linux media server is connected to 4 networks, a "public" or management network, and 3 isolated backup networks. Netstat on the media server is as follows:

# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
10.10.10.0      0.0.0.0         255.255.254.0   U         0 0          0 bond0
172.31.188.0    0.0.0.0         255.255.252.0   U         0 0          0 bond1.550
172.31.192.0    0.0.0.0         255.255.252.0   U         0 0          0 bond1.551
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 bond1.551
0.0.0.0         10.10.10.254    0.0.0.0         UG        0 0          0 bond0

The client ip is 172.31.192.210.

Upon running the bptestbpcd command on to that linux client from the media server, I get:

# bptestbpcd -client 172.31.192.210
1 1 1
172.31.189.197:60892 -> 172.31.192.210:13724
172.31.189.197:50448 -> 172.31.192.210:13724
172.31.189.197:40525 -> 172.31.192.210:13724
 

Why does it appear that NB is connecting to that client from the wrong NIC? Based on the routing table, it should communicate via the 172.31.192 ip address (which is 172.31.192.24).

[mph999 · Accepted solution]

However, it gets way more complex, and I'm not going to try and explain it, when we have perfectly good technotes ...

http://www.symantec.com/docs/HOWTO60779

As you will see, the network used is a combination of the order the network interfaces are detected when the system boots, and what the OS routing tables are.

You will also see that there is a PREFERRED_NETWORK setting also.

You may be aware of 'REQUIRED_INTERFACE'  this should NOT be used if PREFERRED_NETWORK is available, and I think it is at 6.5.6

Martin

[mph999]

 I could be wrong, but you have no gateway set ...

I would suggest the netstat -rn should look something like this

172.31.192.0    172.31.192.24       255.255.252.0   U         0 0          0 bond1.550

This is an excellent explnation and shows some example routing tables.

http://solaris-x86.org/documents/guides/routing.mhtml

What happens if you run a traceroute ?

Martin

[revarooo]

REQUIRED_INTERFACE is perfectly acceptable IF you want don't want your NetBackup traffic using other network interfaces AT ALL.

But I agree with mph999, if you are not to bothered and just need the backups done, PREFERRED_NETWORK is more suitable in that instance.

[mph999]

.. true, but REQUIRED_NETWORK 'might' be dropped in future versions, so good idea to use PREFERRED which is more flexable.

M

[LT2000]

Thanks for your replies.

I found that the REQUIRED_INTERFACE was being used to route the traffic via the 172.31.189.197 IP.

Doesn't look like PREFERRED_NETWORK is available under 6.5.

Subsequent to this post, I figured why we're seeing certain output from the bptestbpcd commands we've been issuing (REQUIRED_INTERFACE, some static routes)...except for one...

On a Solaris master, the routing table is as follows:

# netstat -rn

Routing Table: IPv4
  Destination           Gateway           Flags  Ref     Use     Interface
-------------------- -------------------- ----- ----- ---------- ---------
default              10.10.10.254         UG        1     170081
10.10.10.0           10.10.10.23          U         1      33026 bge0
10.10.10.0           10.10.10.22          U         1          0 nxge3
10.10.10.0           10.10.10.28          U         1          0 bge0:1
10.10.10.0           10.10.10.27          U         1          0 bge0:2
172.31.188.0         172.31.188.22        U         1     149283 bge550002:1
172.31.188.0         172.31.188.12        U         1          0 bge550002:2
172.31.192.0         172.31.192.15        U         1      43645 bge551002:1
172.31.196.0         172.31.191.251       UG        1      45628
224.0.0.0            10.10.10.23          U         1          0 bge0
127.0.0.1            127.0.0.1            UH       86    1294321 lo0

bptestbpcd to a host with IP: 172.31.192.210, yields the following:

# bptestbpcd -client athenu_bu
1 1 1
172.31.188.12:56501 -> 172.31.192.210:13724
172.31.188.12:56504 -> 172.31.192.210:13724
172.31.188.12:56507 -> 172.31.192.210:13724

It would expect the connection to come from 172.31.192.15. I checked the bp.conf on the master, and there is no REQUIRED_INTERFACE line item.

Am I missing something? This one is killing me.

[revarooo]

The host you're running bptestbpcd from, what is the REQUIRED_INTERFACE set to?

[LT2000]

I'm running it from the solaris master server which has no REQUIRED_INTERFACE in bp.conf.

The initial bptestbpcd command was from the linux media server to the same client (172.31.192.210), and the anomaly there was due to the REQUIRED_INTERFACE having been defined on the media server.

Thanks to our security staff, we have all these isolated backup vlans which make troubleshooting connectivity issues a challenge.