How to prove pause frames?

[nicholaskriz]

I have a pair of Cisco Nexus 5596 switches which are connected to a Dell Compellent SAN. I am trying to prove that the SAN is sending MAC Pause frames (flow control) to the Nexus. The Nexus shows the "RxPause" counter incrementing for each interface connected to the SAN. However, the support representative from Dell Compellent does not see the corresponding "TxPause" counter on the SAN incrementing. 

In other words: the Nexus says it is receiving pause frames, but the SAN says it is not sending them. This is a direct fiber connection, so one of the two devices is lying. I'm pretty sure it's the SAN, but I need more evidence to show it.

I attempted a traffic capture using the "monitor session" commands, but later learned that mac pause frames will not be forwarded in a SPAN session like this because the receiving interface intercepts and handles them before they can be forwarded. 

My question is this: how can I demonstrate beyond a doubt that I am receiving these pause frames? Ideally I'd like to have this in a PCAP file because I know the Dell support folks will believe what they can see in Wireshark. Failing that, what else can I do?

Thanks!

Nick

[Tom Kohnen]

We are encountering the same types of issues with Write Performance and RX Pauses.  Are Read Performance is fine.   We have the Chelsio T3 cards as well using SCOS 6.7.05 and have talked to Dell Compellent about this and gave them the Reference number provided in this Thread.  However, Dell says that Reference number does not exist.  Can you verify the Case number was indeed SR 925374479?  Did this issue start after implementing Virtual Ports on the SAN?  We have worked with Microsoft, Cisco and Dell Compellent and everything seems to be pointing back to the SAN, but we are getting push back from Dell and we just need to get the issue resolved.

Any would would be greatly appreciated.

[nicholaskriz]

Tom - this post was a while ago, but I still had notifications turned on!

I know this isn't a very helpful answer for you, but the solution we've turned to is "don't by Dell, don't by Compellent". We're replacing all of this infrastructure, and they will not be considered for the replacement. We aren't doing this because of one little software or hardware bug, that happens all the time. We're doing this because Dell won't acknowledge the problem at all.

Given the information I've found in this thread and other sources, the problem does appear to be the Chelsio cards - that's been a common factor in nearly every reference to this issue. We've demonstrated it every way we can (including bringing in consultants to back our findings), but Dell refuses to acknowledge the information.

All of my ranting aside, I believe you can replace those Chelsio cards, I just don't know what you would replace them with.

[Tom Kohnen]

Did your issue start from the beginning or did it seem to start after implementing Virtual Ports?  Just trying to understand how you found the issue and when it seemed to start.

Thanks Nicholas.

[nicholaskriz]

We never found definitive evidence, but it seemed to be right from the start.

[nicholaskriz]

Oh by the way, it does look like we have the Chelsio NICs you mention.

-N

[runforitbee]

Just wanted to bump this back up. We have an SC4020 running SCOS 7.2. It has T320 Chelsio 10G cards connected to our Nexus 9396 switches. We're getting crazy amounts of pause frames being generated from the Chelsio cards and we're seeing the RX counter jump by the order of hundreds every couple seconds on each of the four physical links on the Nexus.

Anyone happen to stumble across a solution other than a forklift replacement of the Compellent itself?