Cat4K to Citrix NetScaler trunk issues

sean.pedersen · ‎09-10-2013

We’ve introduced several Citrix NetScalers into our DC environment. Multiple connections between the NetScalers and our NX-OS switches (N7K VDCs, N5K access, etc.) are up and fine without trouble. We have a couple of access ports from a C6K up as well without issue.

I’m having a hell of a time with a connection from a C4K running a SupV-10GE, though. I have a GLC-SX-MMD installed in Gi1/4 (one of the 4x SFP slots in the IV-10GE), connected to a 1GE Ethernet port (Finisar 1000BASE-SX SFP) on the NS. The connection will not come up.

Layer 1 checks out fine. The fiber has been tested, and light can be seen from both ends as well as from both SFPs. Each side’s interface is enabled. The SFPs have been replaced. I’ve verified that the SFP slots are good. Yet, both sides remain down.

Our side is a simple 802.1Q trunk with two VLANs allowed across.

!

Interface GigabitEthernet1/4
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 197,201
switchport mode trunk
spanning-tree portfast trunk
!

The NS side is set up as follows:

Speed: Auto

Duplex: Auto

Flow Control: Off

Auto Negotiation: Yes

HA Monitoring: Off

Tag all VLANs: On

I’ve tried everything I can think of, but we’re still down. Both sides are trunks (tagging for the NS), both sides are 802.1Q, both sides are enabled, both sides are tagging the same VLANs, both sides are set to auto/auto, cabling good, hardware good...link, not good. From the switch:

switch>sh int status | i 1/4

Gi1/4 Trunk | NetScaler notconnect 1 full 1000 1000BaseSX

switch>sh int desc | i 1/4

Gi1/4 down down Trunk | NetScaler

From the NS:

Interface 1/5 (Gig Ethernet, SRIOV virtual interface) #8
        flags=0x6042 <ENABLED, DOWN, down, 802.1q, tagall>
        MTU=1514, native vlan=1, MAC=26:ff:13:f6:66:b7, downtime 957h41m38s
        RX: Pkts(0) Bytes(0) Errs(0) Drops(0) Stalls(0)
        TX: Pkts(0) Bytes(0) Errs(0) Drops(0) Stalls(0)
        NIC: InDisc(0) OutDisc(0) Fctls(0) Stalls(0) Hangs(0) Muted(0)
        Bandwidth thresholds are not set.
        VMAC=00:00:5e:00:01:81

I’m convinced our side is fine, or would be if we were talking to another Cisco switch or device. I feel like there is something needed on our side to “help” the NS establish the trunk, and/or there is still some configuration work to be done on the NS.

I was hoping that someone else had run into the same problem, and might have some insight. I’m sure it’s something silly that my fatigued brain is forgetting.

Any help would be greatly appreciated!

Mike Williams · ‎09-10-2013

Have you tried manually setting the port speed to 1000/full on the switch? Have you verified the fiber is crossed and not straight through?

Regards,
Mike

Sent from Cisco Technical Support Android App

Marvin Rhoads · ‎09-10-2013

I'd suggest the hard-coding speed too. Never mind that the SFP only runs at one speed - sometimes you just need to do that when interfacing with third party equipment.

Since you have the MMD variant of the transceiver at your end, you can verify received light levels with "show int gi1/4 transceiver details". Normally not an issue in the same room but it never hurts to check.

sean.pedersen · ‎09-11-2013

It's an SFP, so I can't hard-set speed and duplex, but I can disable negotiation. That forces my side up, but his side remains down. The NS can set speed and duplex, though, so I can try and disable negotiation on my side and have him hard-set to 1000/full to see what happens.

I just did a hard loop toward the C4K and toward the NS. The C4K looped up fine, but the NS interface was dark. I'm still convinced there is an issue on the NS side.

Unfortunately, it doesn't look like I can view transceiver details. DOM requires 12.2(25)EWA and we're on 12.2(50)SG1 - maybe the difference in train is ruining things for me.

sean.pedersen · ‎09-11-2013

Yep, Tx/Rx are fine. We can't hard-set speed, but we can turn auto-negotiation off, which is more or less the same. Our side came up, but his remained down. However, his end was still set to auto/auto, so that was probably a wash. I'm going to have them set their side to 1000/full and I'll set mine to nonegotiate and we'll see what happens.

sean.pedersen · ‎09-11-2013

I've confirmed that this is an issue with the NetScaler. This morning I looped up interface 1/5 on both of their NetScalers and they were dark. I moved the SFP from 1/5 to 1/7 and was able to hard loop their interface. I reconnected the fiber jumper back to the C4K and the link came up.

Since it's the same interface on two different NetScalers that has the problem, the evidence points to a configuration problem on their end. This finally proves it!