5220 appliance IPMI confusion and mystery

[Lynne_Seamans]

Have a 5220 netbackup appliance which acts as master and media server.

Recently upgraded to latest software, 2.6.1.1.  Also, purchased additional memory and another storage shelf, which was successfully installed.

Running along just fine, when i get a call from one of our "Networking Guys',  tracing packets for some other problems, but see an awful lot of traffic for a certain MAC addr.   And it's one - off from what's listed as the main interface for my 5220.  Not mine, I say.

Well, he assigned the mac address a name and IP number and now, when I https to it, VOILA, IPMI interface for my 5220.

How did this happen?  I looked at the back, and there is an ethernet-looking connection labelled IPMI, but there's no cable in it. 

Not causing any trouble, but I would like to know if this is something new, or somethings that's been broadcasting a while and someone just noticed it.  Besides the "solving the mystery" aspect, I do have another appliance which I might like to get IPMI going on, and would really like to know what happened here.

[chashock]

This is very, very odd behavior.  The closest thing I can think of is that the chipset provides a BMC and RMM controller, and possibly the BMC is operating on a shared NIC configuration, but I don't have a 5220 immediately available to even see if this is a possibility for the 5220 implementation (it is on other motherboards).

I'd take a look at the BIOS configuration the next time you have an opportunity to reboot it, and make sure that the BMC controller is unconfigured.  

[Andrew_Madsen]

Are you certain that you are at the IPMI instead of the appliance web page? They are different and use different users to access them.

[Lynne_Seamans]

Yup, positive.  The one you need to log in as "sysadmin" rather than "admin".  To be sure, though, here's what it looks like:

[Lynne_Seamans]

thanks for that suggestion.  think i saw the instructions for that in the 5220 admin guide...   will check it out.

[Andrew_Madsen]

That is perplexing. The IPMI is on a daughter board so there is no sharing of the Ethernet connection. Here are some things to like at:

From the CLISH: Support > IPMI Network Show

This should give us SOME idea as to how this is happening. 

On the back of the box there should be Eth1, Eth2, Eth3, Eth4, IPMI, and Serial see which ones have cables installed. I know you said IPMI did not but which ones do?

[Lynne_Seamans]

bladerunner.Support> IPMI Network show
IP Address Source    : Static
IP Address           : 192.168.0.10
Subnet Mask          : 255.255.255.0
Gateway IP Address   : 0.0.0.0

pretty boring... and not the ip number i can access it on. 

[sdo]

To confirm the MAC address of the IPMI:

> Support > Maintenance >
P@ssw0rd
/opt/Symantec/scspagent/IPS/sisipsoverride.sh
P@ssw0rd
...then select option 1. Overrride prevention except for Self Protection
...then option:   6. 8 hours
...enter a comment, or your name, or a case number, anything...
maintenance-!> elevate
nb-appliance: # /usr/bin/ipmitool lan print 3

HTH

[Lynne_Seamans]

[chashock]

That is the default address for the IPMI.

On the Configuration tab, is there a drop-down for BMC?  See if that has the IP address you can access it on.  If not the Configuration tab, look at the System Information tab (I'm going off the top of my head here).

[Lynne_Seamans]

Here's what my "Configuration" tab looks like, the drop down I see is for "LAN Channel", and the only option besides the "Baseboard Mgmt" that's showing is Intel(R) RMM3

[sdo]

You seem to be missing a "Default Gateway IP" on the IPMI network config.

You need to identify what the default gateway is for your subnet, which itself appears to 192.168.0.0/24, because your netmask is 255.255.255.0.

Once you know the gateway IP, you can set the gateway, by 'exit'ing from OS root shell, back to CLIsh shell, and then:

> Support > IPMI Network Configure 192.168.0.10 255.255.255.0 192.168.0.xxx

...where ".xxx" is the trailing part of the IP address of the default gateway for that subnet.

.

And then try pinging the IPMI address again.

.

Here's a tip:  If there's no-one around to confirm the gateway IP, then it is usually pretty harmless to just try and guess it.  You could try using .254 as the trailing part of the gateway address, and then try to ping the actual IPMI address again.  You also try pinging the gateway itself.  But, the thing is, is that 192.168.x.x are classic DMZ addresses, and so maybe your network admin has disabled ping (ICMP) type traffic to/from a DMZ.  Maybe also blocked most traffic to that subnet.

Even if after setting the gateway IP to .254, if the IPMI does not ping, you should try a telnet to port 80 and try another telnet to port 443 of the IPMI address, if these get through - then it's able to listen to you - so then try browsing (internet explorer, firefox, etc) to the IPMI address. 

What I'm trying to say is that in a great many cases, a failed ping does not necessarily mean that you cannot reach a device.  The only way to actually confim whether one can reach a device is to try to telnet to a port that is known, or 100% expected, to be open (and listenning) on the target device.

[sdo]

That's weird, the IPMI lan print shows 192.168.0.10, but the IPMI sysadmin GUI shows 166.66.87.190.

Hmmm.  Now I'm confused.

[sdo]

(just as a side note - on my own test rig that uses an Asus motherboard, I have installed an IPMI type daughterboard too, and this has the ability to intentionally configure a kind of shared foot-print, i.e. the IPMI module/software configuration can dictate which of the physical on-board 1Gb NICs to actually use as the IPMI.).

.

So, on your appliance, when selecting "Baseboard Mgmt" I think you have effectively told the IPMI module/software to use one of the four Gb NICs.  Whereas setting the channel "Intel(R) RMM" selects the actual physical port labelled on the back of the appliance as "IPMI".

[sdo]

Are you sure you're logged in to a v2.6.1.1 appliance.  My appliance IPMI sysadmin GUI looks quite different, several extra options (e.g. IP v6).

.

And I'm fairly sure you've configured the wrong channel of the IPMI.  On my appliances, I have configured the LAN Channel named 'Intel(R) RMM' as my IPMI, and on my appliance the LAN Channel named 'Baseboard Mgmt' is effectively clear.

.

I bet if you ran these commands you'll see it in text form too:

/usr/bin/ipmitool lan print 1
/usr/bin/ipmitool lan print 2
/usr/bin/ipmitool lan print 3

[Lynne_Seamans]

Well, like I said, I didn't overtly configure this..  It just sort of start chattering...    and someone used our DHCP server to assign an IP number to the MAC addr they saw in the "chattering"..  and I went in and put that IP number in the IPMI gui. 

I do have a 5200 at 2.5.3 also.  I identified this IPMI that just sprouted up and as 5220 based on serial number.  Is there a way I can verify that?

I do understand about "baseboard Mgmt" making a cable in the actual IPMI connection not required.  I do have a photo showing that port empty.. 

[sdo]

Did the Network admin give you the MAC address of the chatty port?

Does it match any of the MAC address shown from the IPMI lan print #chan-num    i.e. channels 1, 2 and 3?

The commands I gave you were for v2.6.1.1.

I guess you could google and find the equivalent commands for appliance v2.5.3 to show the detail re the IPMI channels on a v2.5.3 appliance.

.

IMO, I would move the IPMI cable to the IPMI port, and clear the 'Baseboard Mgmt' config and configure the 'Intel(R) RMM' channel instead.   As this is the Symantec recommended (required?) configuration.

[Lynne_Seamans]

Yes, he said it was 00-1E-67-2F-48-26  and suspected my 5220 (name: bladerunner) because 00:1E:67:2F:48:24 and 00:1E:67:2F:48:25 belonged to it. 

here's the ipmitool lan print for 1,2,3.   2 is not a LAN channel, it says.

bladerunner:/home/maintenance # /usr/bin/ipmitool lan print 1
Set in Progress         : Set Complete
Auth Type Support       : NONE MD5 PASSWORD
Auth Type Enable        : Callback : NONE MD5 PASSWORD
                        : User     : NONE MD5 PASSWORD
                        : Operator : NONE MD5 PASSWORD
                        : Admin    : NONE MD5 PASSWORD
                        : OEM      :
IP Address Source       : Static Address
IP Address              : 166.66.87.190
Subnet Mask             : 255.255.252.0
MAC Address             : 00:1e:67:2f:48:26
SNMP Community String   : INTEL
IP Header               : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00
BMC ARP Control         : ARP Responses Enabled, Gratuitous ARP Disabled
Gratituous ARP Intrvl   : 0.0 seconds
Default Gateway IP      : 166.66.86.254
Default Gateway MAC     : 00:00:00:00:00:00
Backup Gateway IP       : 0.0.0.0
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : Disabled
802.1q VLAN Priority    : 0
RMCP+ Cipher Suites     : 1,2,3,6,7,8,11,12,0
Cipher Suite Priv Max   : caaaXXaaaXXaaXX
                        :     X=Cipher Suite Unused
                        :     c=CALLBACK
                        :     u=USER
                        :     o=OPERATOR
                        :     a=ADMIN
                        :     O=OEM
bladerunner:/home/maintenance # /usr/bin/ipmitool lan print 2
Channel 2 is not a LAN channel
bladerunner:/home/maintenance # /usr/bin/ipmitool lan print 3
Set in Progress         : Set Complete
Auth Type Support       : NONE MD5 PASSWORD
Auth Type Enable        : Callback : NONE MD5 PASSWORD
                        : User     : NONE MD5 PASSWORD
                        : Operator : NONE MD5 PASSWORD
                        : Admin    : NONE MD5 PASSWORD
                        : OEM      :
IP Address Source       : Static Address
IP Address              : 192.168.0.10
Subnet Mask             : 255.255.255.0
MAC Address             : 00:1e:67:2f:48:27
SNMP Community String   : INTEL
IP Header               : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00
BMC ARP Control         : ARP Responses Enabled, Gratuitous ARP Disabled
Gratituous ARP Intrvl   : 0.0 seconds
Default Gateway IP      : 0.0.0.0
Default Gateway MAC     : 00:00:00:00:00:00
Backup Gateway IP       : 0.0.0.0
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : Disabled
802.1q VLAN Priority    : 0
RMCP+ Cipher Suites     : 1,2,3,6,7,8,11,12,0
Cipher Suite Priv Max   : caaaXXaaaXXaaXX
                        :     X=Cipher Suite Unused
                        :     c=CALLBACK
                        :     u=USER
                        :     o=OPERATOR
                        :     a=ADMIN
                        :     O=OEM
bladerunner:/home/maintenance #

[sdo]

There it is, on IPMI channel 1 - the 'chatty' config is the 'Baseboard Mgmt' config.

I suggest that you move the IP config of the Baseboard Mgmt (channel 1) to become the IP config of the Intel RMM (chennel 3) - and then move the cable end to the labelled IPMI port - and then clear the config of the Baseboard Mgmt (channel 1).

[chashock]

That's what I thought had happened.  As that screen shot above shows, you don't have IPMI configured, you have the BMC, and the BMC uses the onboard 1GbE NICs (eth0-eth3) not the IPMI port.  What may have happened is a mistake was made at install time and the BMC was accidentally configured to use DHCP instead of the IPMI being properly setup.  I've seen that happen before.

By default, the BMC should have a static address of 0.0.0.0.  That should prevent it from chatting on the network at all.  If you change that to 0.0.0.0 and delete the information in the subnet and gateway fields you shouldn't have any more traffic traversing your network from the BMC.  Out of the factory these are not set to DHCP for the BMC port, so as I say it must have been accidentally changed probably during configuration.