Cisco vPC with F5 Viprion

Yannick Vranckx · ‎09-11-2017

Hello,

We are facing an issue it seems with our data center design. The design is really simple and straight forward.

We have 2 F5 Viprion chassis connected to the access layer in the data center. The access layer is hosted on N9K. The core is N7K.

So chassis 1 forms a vPC with the N9K1 and N9K2, chassis 2 forms a vPC with the N9K3 and N9K4. This connection is fully LACP.

interface port-channel141
  description F5 Viprion 1 - 4*10G - vPC channel
  switchport mode trunk
  switchport trunk native vlan 999
  switchport trunk allowed vlan 310,312,410-415
  spanning-tree port type edge trunk
  spanning-tree bpduguard disable
  spanning-tree guard root
  spanning-tree bpdufilter disable
  logging event port link-status
  logging event port trunk-status
  storm-control broadcast level 80.00
  storm-control multicast level 80.00
  lacp mode delay
  vpc 141

This is an example on how the port-channel towards the F5 is configured. It uses LACP mode delay so it will await the LACP negotiation from the viprion i suppose.

The problem is that when we access an application that is hosted on the F5, the throughput is really low. So for example: A remote desktop application will only achieve 5Mbps when pushed throught the F5, a remote desktop application straight to the server will achieve 70Mbps. Please note that the server is also behind another 9K ToR setup. So the problem only occurs when we push traffic through the F5. It also has an issue with a sharepoint server that is connected and only gets 40KBps of download speed.

My guess is that there is a problem between the F5 and the Nexus, is there documentation on how this is setup the best way? F5 only has this: https://support.f5.com/csp/article/K13142

An engineer from F5 is tasked to look at the F5 but is not finding anything.

Austin Sabio · ‎09-12-2017

First, I am posting below link as a reference- hoping its going to be helpful in one way or another.

https://supportforums.cisco.com/t5/lan-switching-and-routing/nexus-7k-port-channel-to-f5-ltm-load-balancer/td-p/2629364

Mainly, with F5 and Nexus VPC - validate your configs are according to cisco and f5 best practices also troubleshoot from F5 to F5 (source and destination data flow) with show vpc commands and show port-channel commands that are specfic to vpc141 and vpc14 (same with other half vpc 241 and 24)- also check logs from all partciapated devices.

I hope this helps and good luck!

Yannick Vranckx · ‎09-13-2017

Hello thanks for your message,

I have looked at the thread but there are some differences.

We did not connect our F5 Viprion's to the Core layer of the data center, we connected it to the edge with a Nexus 9K.

We are using a ToR design aswell with vPC.

This means that one chassis is connected to 2 ToR 9K that form a vPC towards the chassis and are a vPC domain on themselves. They then form a vPC towards the core which is a 7K.

The other chassis is connected the exact same way.

The config goes as follows:

on the left end of the picture

interface port-channel141
  description Chassis 1 - vPC channel
  switchport mode trunk
  switchport trunk native vlan 999
  switchport trunk allowed vlan 310,312,410-415
  spanning-tree port type edge trunk
  spanning-tree bpduguard disable
  spanning-tree guard root
  spanning-tree bpdufilter disable
  logging event port link-status
  logging event port trunk-status
  storm-control broadcast level 80.00
  storm-control multicast level 80.00
  lacp mode delay
  vpc 141

interface Ethernet1/1
  description Po141: Chassis1 1.1 - 10G FO MM - vPC channel member
  lacp port-priority 1
  no cdp enable
  switchport mode trunk
  switchport trunk native vlan 999
  switchport trunk allowed vlan 310,312,410-415
  logging event port trunk-status
  storm-control broadcast level 80.00
  storm-control multicast level 80.00
  channel-group 141 mode active

interface Ethernet1/2
  description Po141: Chassis 1 1.2 - 10G FO MM - vPC channel member
  no cdp enable
  switchport mode trunk
  switchport trunk native vlan 999
  switchport trunk allowed vlan 310,312,410-415
  logging event port trunk-status
  storm-control broadcast level 80.00
  storm-control multicast level 80.00
  channel-group 141 mode active

The only thing i see different is that we give 1 port LACP port-priority 1, i do not know if that affects it. We also use lacp mode delay, so the port-channel will wait for the host to setup the connection i guess.

Austin Sabio · ‎09-13-2017

Clear the counters for the port-channel and both individual interfaces then wait for at least a min then post.

show interface ethernet 1/1

show interface ethernet 1/1 counters detailed all

show interface ethernet 1/2

show interface ethernet 1/2 counters detailed all

show interface port-channel 141

show interface port-channel 141 counters detailed all

check below

14.3.7 F5 LTM Load Balancer Configuration

https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-733639.html#_Toc413737130

Yannick Vranckx · ‎09-13-2017

Hello,

Thanks for the message.

I will do this

We have also done a bit of troubleshooting on the F5 and noticed that it's actually the server that seems to not respond towards the F5's packet. So the server receives a connection from the F5's float IP, but very often the server does not respond anymore and the F5 has send DUP ACK and eventually retransmit the connection.

What i did notice:

The MTU of the physical interface of the viprion is set to 9198, but the MTU of a vCMP is set to 1500. The MTU of the switch is also set to 1500, do we need to match everything here to avoid problems? Or is this like a eventuality that the F5 is ready to receive jumbo frames or does it always expect them.

Also i noticed that when i perform a traceroute from the server the next hop is the Nexus core (which is normal in this case, because the F5 uses SNAT), but it sometimes changes the first hop. Sometimes it's the first nexus core and sometimes it's the other nexus core, i have noticed that peer-gateway command is on the vPC domain. So this would say that both cores can proxy for the VIP address of HSRP, but is it normal that i see the next hope then change in the tracert on the server? Should i just always see the VIP HSRP address?

Austin Sabio · ‎09-13-2017

Correct. I was about to mention that in my response that's why I am asking for show counters result - this is a typical issue where one side is set with jumbo frames and the other is not as in your case both the switch or the server are set to MTU 1500. In regards to trace route path, I would say its an expected behavior as its load balancing between the two links is based on its own hash. I hope this was helpful.