Intermittent network connection problem ping timeout between two Cisco UCS B200 m3

[MARCELO LOUREIRO]

I'm having a problem with my Cisco UCS B200 M3 and was hoping someone could give me some advice on locating the problem.

Detected a network connectivity problem, intermittent between two Cisco UCSB-B200-M3. The ping timeout.

The problem occurs only between two blades may be in the same chassis or different. Exchanged the card

UCSB-MLOM-40G-01= and the problem continues.

Leaving one of the blades alone in the chassis the problem does not occur.

Inserting the second blade in the chassis occur problem only between the two blades even in different chassis.

UCS B200 M3 not include the components.

Does anyone know the part number cisco for UCS B200 M3 blade server full?

[Keny Perez]

Hi,

Could you please give us more details of the network?

-What OS are you using on these blades? If it is ESXi, do you use VM-FEX or use a vSwitch for switching between them?

-Is there any ACL or firewall somewhere in the middle?

-Are those chassis sharing the same pair of FIs?

-UCSM version running in your environment?

Is there any diagram you may facilitate to avoid making this a 1000 questions reply? 

-Kenny

[carlonicolini]

Hi,

I'm facing with a similar issue.

We have 3 chassis on the same pair of FIs (model: UCS 6248UP ).

Blades Model: UCS B200 M3

Chassis #1 with 7 blades (2 of them are switched off)

Chassis #2 with 6 blades (all of them switched on)

Chassis #3 with 6 blades (all of them switched on)

We have problem with blade #5 of Chassis #1

On this server we have intermittent replay to ping requests.

OS on the servers is ESXi 5.5
There are no firewalls between the blade servers.

UCSM version 2.2 (2c).

Regards,

Carlo

[Keny Perez]

Are those blades going through the same fabric interconnect and belong to the same vlan?  Are both of the IP addresses involved in the same vlan?

-Kenny

[carlonicolini]

Same FIs on the same VLAN. Both blades belong to the same VLAN.

Carlo

[Keny Perez]

Ok, and the vNICs you are using to ping each other are pointing to the same FI?  I am trying to see if your ICMP traffic is being handle locally by the same FI or if it has to go to the upstream network to get to the other blade in the other FI.

What UCSM version are you running?

Could you please try to ping your servers from the FIs and see what happens when you do that?  To ping the blades, go to a CLI session and enter "connect local A" (or B, depends on which FI you are in) and ping the blades IP from there and let us know if it is still not consistent.

-Kenny

[carlonicolini]

"What UCSM version are you running?"
[Carlo: UCSM version 2.2 (2c)]

"Could you please try to ping your servers from the FIs and see what happens when you do that?"
[Carlo: ping are intermittent as well below an extract of ping activity output
64 bytes from 10.157.246.17: icmp_seq=1 ttl=63 time=0.183 ms
64 bytes from 10.157.246.17: icmp_seq=2 ttl=63 time=0.146 ms
64 bytes from 10.157.246.17: icmp_seq=3 ttl=63 time=0.145 ms
64 bytes from 10.157.246.17: icmp_seq=4 ttl=63 time=0.146 ms
64 bytes from 10.157.246.17: icmp_seq=10 ttl=63 time=0.165 ms
64 bytes from 10.157.246.17: icmp_seq=11 ttl=63 time=0.143 ms
64 bytes from 10.157.246.17: icmp_seq=12 ttl=63 time=0.151 ms
64 bytes from 10.157.246.17: icmp_seq=13 ttl=63 time=0.158 ms
64 bytes from 10.157.246.17: icmp_seq=14 ttl=63 time=0.143 ms
64 bytes from 10.157.246.17: icmp_seq=20 ttl=63 time=0.186 ms
64 bytes from 10.157.246.17: icmp_seq=21 ttl=63 time=0.150 ms
64 bytes from 10.157.246.17: icmp_seq=22 ttl=63 time=0.163 ms
64 bytes from 10.157.246.17: icmp_seq=23 ttl=63 time=0.167 ms
64 bytes from 10.157.246.17: icmp_seq=24 ttl=63 time=0.146 ms
64 bytes from 10.157.246.17: icmp_seq=30 ttl=63 time=0.151 ms
64 bytes from 10.157.246.17: icmp_seq=31 ttl=63 time=0.143 ms
64 bytes from 10.157.246.17: icmp_seq=32 ttl=63 time=0.150 ms
64 bytes from 10.157.246.17: icmp_seq=33 ttl=63 time=0.147 ms
64 bytes from 10.157.246.17: icmp_seq=34 ttl=63 time=0.155 ms
64 bytes from 10.157.246.17: icmp_seq=40 ttl=63 time=0.149 ms
64 bytes from 10.157.246.17: icmp_seq=41 ttl=63 time=0.152 ms
64 bytes from 10.157.246.17: icmp_seq=42 ttl=63 time=0.150 ms
64 bytes from 10.157.246.17: icmp_seq=43 ttl=63 time=0.153 ms
64 bytes from 10.157.246.17: icmp_seq=44 ttl=63 time=0.157 ms
64 bytes from 10.157.246.17: icmp_seq=50 ttl=63 time=0.156 ms
64 bytes from 10.157.246.17: icmp_seq=51 ttl=63 time=0.148 ms

The behaviour is the same from both FIs. 

I just noticed that we have the same issue on 2 (not only one) blades of 7 on the same chassis.

The other 2 Chassis with 6 blades for each in the same UCS farm have no problems.

Regards,
Carlo

[Keny Perez]

Ok, and that happens when you ping the blades CIMC IP address or when you ping the OS on top of the blade?
 

-Kenny

[carlonicolini]

CIMC IP replies properly both from FIs and from other clients.

We also tried to chage IPs to these blades, but we continue to face with intermittent pinging as well.

Regards,

Carlo

[Keny Perez]

OK, so if the CIMC IP responds properly, I think the server itself is unlikely to be the problem and I would try to go over the steps here:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004109

Please update the thread if you find the reason or if you need further help.

-Kenny

[mizhang]

In order to isolate the issue, we need to understand the data path between the two VMs.  Pls. track the MAC address between the two.

login to each FI, and [connect nx a ] and [connect nx b ], and [show mac address vlan xx | in {mac address} ].

If both MAC addresses show up on either FI-A or FI-B, then the data path will be within UCS POD (including FI), otherwise it will go to the upstream switches and then back down.  Hopefully it will help narrow down the scope.

Like Kenny asked, need to know what OS is running on the blades, and what type of switch as well.  For example, for ESXi, you can have VMWare standard switch, nexus 1000v, VMWare DVS etc. 

Another thing to check is to see if all the firmware, driver are compatible.

http://www.cisco.com/web/techdoc/ucs/interoperability/matrix/matrix.html