UCS-FI-6332 unable to ping each other (FI-A and FI-B) even the cluster is showing UP

[Wajeeh Ghouri]

Hi team, 

i am doing the initial configs for the UCS FI - cluster is showing as UP however i can't ping from FI-A to FI-B and vice versa, Configuration is correct i have validated. (IP, mask and default GW) 

Default GW device is not yet setup, Does it goes to default gw to communicated each other ? 

Both FI are directly connected to L1-L1 and L2-L2 ports.

(local-mgmt)# show cluster extended-state
Cluster Id: xxxxxxx

Start time: xxxx
Last election time: xxxxxx

B: UP, SUBORDINATE
A: UP, PRIMARY

B: memb state UP, lead state SUBORDINATE, mgmt services state: UP
A: memb state UP, lead state PRIMARY, mgmt services state: UP
heartbeat state PRIMARY_OK

INTERNAL NETWORK INTERFACES:
eth1, UP
eth2, UP

HA NOT READY
No device connected to this Fabric Interconnect
(local-mgmt)#

Appreciate your help.

thank you

[Wes Austin · Accepted solution]

You need layer 3 connectivity out of the management interface in order for the FI to communicate. L1-L2 is only for heartbeats for HA.

Yes, you need to setup your default gateway first.

[pranavgu · Accepted solution]

How are you actually pinging the FI from you laptop or from the local-mgmt. Have we configured the gateway on the switch where the FI are connected. Please ensure that gateway is configured properly as Wes is mentioning above.

[Wajeeh Ghouri]

Thanks for your reply, 

I was pinging from FI-A to FI-B via local-mgmt -correct.

Default GW device is not configured yet.

So, its mean that FI-A to FI-B can only ping each other mgmt IP addresses via the default GW device..?? I was thinking that they are connected directly with one another, so they should ping each other. 

Also, 

I further dig into it and connected to nxos prompt on FI, found that the mgmt ip is down, not sure why its force down? And i was trying to go into the 'configure terminal' prompt to go into the interface and bring this interface UP, however i could not get into the Config t, as there is no option to config it... OR it could be that reason that i can not ping the other FI mgmt IP address. 

-B(nxos)#

!Command: show running-config interface mgmt0
!Time: Fri Apr xxx.xx.xxx.

version 5.0(3)N2(3.21b)

interface mgmt0
shutdown force
ip address xx.xx.xx.xx/24

-B(nxos)#

Thanks again

[Wajeeh Ghouri · Accepted solution]

Thank you both of you.. i got my answer  default GW device important to validate the mgmt readability. 

One quick question .. if you guys can share an idea why .. that would be great..

---snippet -- from the previous post.

I further dig into it and connected to nxos prompt on FI, found that the mgmt ip is down, not sure why its force down? And i was trying to go into the 'configure terminal' prompt to go into the interface and bring this interface UP, however i could not get into the Config t, as there is no option to config it... OR it could be that reason that i can not ping the other FI mgmt IP address. 

-B(nxos)#

!Command: show running-config interface mgmt0
!Time: Fri Apr xxx.xx.xxx.

version 5.0(3)N2(3.21b)

interface mgmt0
shutdown force
ip address xx.xx.xx.xx/24

-B(nxos)#

[Wes Austin · Accepted solution]

Again, L1-L2 are for heartbeat communications only, not ICMP.

The mgmt interface being down in nxos expected and you will see that regardless of the connectivity.

[Wajeeh Ghouri]

Thank you so much!!!

[Wajeeh Ghouri]

Guys, 

One more important question if i need to bring up the FI - mgmt interface in nxos prompt How can i enable or (no shutdown) the FI - mgmt interface which is currently at force shutdown state.

while being on the F1_A# (prompt) i went to 'connect nxos' to bring the interface up. howerever i am unable to see 'configure terminal' command to get into configuration mode to get into the mgmt inter and do 'no shutdown' to bring up the interface. 

appreciate your help ..

Thanks 

[Kirk J]

As Wes has already mentioned, that is not something that you need to configure.

You do not have conf level access via NXOS.  

Changes are implemented via the UCSM GUI or UCSM CLI commands.

The mgmt interfaces will automatically come up, assuming you assign them valid IP info, and the upstream switchport is correctly configured.  The nxos level output of mgmt interfaces will always show down on FIs, regardless of whether they are actually up or down.

Thanks,

Kirk...

[pranavgu]

Just to add to kirk reply config will not work on the fabric interconnect . If you will config them in switching mode then you should be able to run the config on these however this is not recommended and is not how the UCS was designed. Its design is to work in end host mode to avoid any sort of STP and it adds redundancy to the network.

also mgmt interface down is done on purpose:
https://tools.cisco.com/bugsearch/bug/CSCta43727/?reffering_site=dumpcr

Hope this should answer your query.