Nexus 7K port channel to F5 LTM load balancer

[josephsmar]

I am hoping some has set this up already, but I have not been able to find any examples in this forum or on the internet. What we are trying to do is setup a port channel from our nexus 7k to a F5 LTM load balancer. The links are two 10gig. One thing I found was the when applying the channel group to the interface you need to use the following command: 

channel-group XX mode active

This is for the lacp portion of the interface. We set that and setup the LTM, but still cannot get traffic to pass.

[josephsmar]

So this is what I get when I setup the F5 and Nexus the way you said to. Also I am not getting a mac from the port-channel anymore. I was getting one when I had the standard PO config with one port in it.

show port-channel summary

Flags:  D - Down        P - Up in port-channel (members)
        I - Individual  H - Hot-standby (LACP only)
        s - Suspended   r - Module-removed
        S - Switched    R - Routed
        U - Up (port-channel)
        M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port-       Type     Protocol  Member Ports
      Channel
--------------------------------------------------------------------------------
2012    Po2012(SU)    Eth      LACP      Eth1/27(s) Eth3/27(P)

[Bilal Nawaz]

Hi Joseph, if you could please give a little more insight as to what you have configured... Show the running config of the physical ports E1/27 and E3/27, along with Po2012, i'd also like to see "show int e1/27" - as its in suspended please do the command, and then a "shut" "no shut", and then "show log last 20"

With regards to the F5 I'd like to see the trunk configuration, the vlan information, STP settings and the physical interface parameters. 

The fact that we have one of the links to "up" state is promising, so i think this will be config thing with regards to vlan trunking.

Thank you

Bilal

[josephsmar]

Bilal,

 I actually took lacp off and they came up fine. I just have to finish testing to make sure it passes traffic like it is supposed to.

[Bilal Nawaz]

Hi Joseph, sounds like good news. Please let us know how you get on. Perhaps the original problem was that LACP was not configured on the F5 side...

[josephsmar]

Actually I had Lacp configured on both sides. I tried all the combinations possible and it never would work. I also turned off flow control on the f5 at the same time so that may have something to do with it as well. I am going to play around with the settings a bit and figure it out for sure. I will post up the final config once I have it all figured out.

[BenBen]

Hi Bilal,

I am not sure if you still monitor this post,, but thank you for sharing your configuration. After reading through the F5 concept documents, it seems clear that F5 is a bridge/switch device. In my opinion, if F5 is a bridge/switch, we should be cautious to configure the ports connecting to the F5 as "edge port" or "portfast port" unless we can guarantee that thee F5 will not form a loop to the network.

In your post, if the Prod and NPE having the same VLANs running on it, and also if both of them passing data, with port edge configured, there will be a loop. Considering F5 is configured as "pass through", if Prod port receiving BPDU, it should forward it to NPE port. Ideally, the BPDU guard function on NPE should shutdown the port. But if F5 failed to forward the BPDU (this happens for a model of Cisc0 IP phone due to a bug), NPE port may not be disabled which will result in a loop in your network.

Actually, I saw a configuration with F5 configured as "pass through", and with two ports connected to a Nexus switch. The Nexus switch is configured with Rapid PVST+, the F5 didn't pass the received BPDU packets to the other port so both ports are shown as up. Luckily, there is no common VLAN on these two ports so there is no broadcast storm created but I am concerned.

I am still trying to figure out if my above analysis makes sense and try to find out a safe solution for the F5 integration. But before a solution is developed, I may more intend to NOT configure the port to F5 as "edge port" or "portfast port".

If you can give some inputs or thoughts, or let me know if my concern is valid that will be much appreciated. 

Thanks.

Ben

The following is some inform from F5 doc just for reference.

"The BIG-IP® system is a port-based switch that includes multilayer processing capabilities. "

"The BIG-IP system learns the interfaces that correspond to various MAC entries as frames pass through the
system, and automatically adds entries to the table accordingly. These entries are known as dynamic entries.
You can also add entries to the table manually, and these are known as static entries. Entering static entries
is useful if you have network devices that do not advertise their MAC addresses. The system does not
automatically update static entries.
The BIG-IP system does not always need to use the L2 forwarding table to find an interface for frame
transmission. For instance, if a VLAN has only one interface assigned to it, then the BIG-IP system
automatically uses that interface.
Occasionally, the L2 forwarding table does not include an entry for the destination MAC address and its
corresponding BIG-IP system interface. In this case, the BIG-IP system floods the frame through all interfaces
associated with the VLAN, until a reply creates an entry in the L2 forwarding table."

[josephsmar]

If I may, why put the spanning tree port set to edge? Also if we are running spanning tree in MST what would the F5 need to be to get this to work?

[Bilal Nawaz]

Why not put the spanning-tree port type to edge? It is to some extent a host on the network whereby we only serve it vlan access via trunking. The F5 won't be extending out our L2 VLANs to other switch's / bridges. The main pro of this feature is to leverage the port fast capability, to automatically transition to STP forwarding state without passing through blocking/learning states.

Please notice that I have BPDU Guard enabled just as a backup in any case, for whatever reason due to misconfig/bug etc.. was to occur.

On the F5 I use STP mode "Pass Through". Here is a definition from F5 of how it operates, which is probably best explained...

Specifies that when the BIG-IP system receives spanning tree frames (BPDUs), it forwards them to all other interfaces. This is the default setting. When you use Pass Through mode, the BIG-IP system is transparent to spanning tree BPDUs. When set to Pass Through mode, the BIG-IP system is not part of any spanning tree. Note that Pass Through mode is not part of the IEEE spanning tree protocol specifications.

However, it does support STP, RSTP and MSTP to my recollection, if you were to configure these on the F5, then they would operate as normal network switches and handle BPDU's the same way as a switch.

In my view, when I thought about forming a configuration template for many installations, the pro's and con's of either, I thought it would be prudent to just hard code the F5 as an edge device with no BPDU's here to confuse matters. Along with this, F5 (or another Load Balancer) is/can be central to some or most organisations providing services to the end users / clients which should be "UP" ASAP in the case of any outage.

Here is an F5 link for your reference.

https://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/tmos_management_guide_10_1/tmos_stp.html

hth

Bilal

[Andrew Devine]

Hi Bilal,

I'm curious as to why you define the native VLAN as 1103.  Is this to protect against misconfiguration on the F5 side?

My understanding is that in 'Pass Through' mode the LTM drops all BPDUs unless they arrive on an untagged VLAN. 

So as you are using LACP, which should be a single logical interface from an STP point of view, the LTM normally does not pass though this BPDU back to the Nexus.  If the LTM was misconfigured it may then switch between these interfaces and pass through the IEEE BPDU over VLAN 1103 and BPDU guard should kick in to protect against a loop?

Cheers,

Andrew

[Bilal Nawaz]

Hi Andrew, 1103 was used for management of the vCMP guest, I cant remember why i untagged it. The pass through mode disregards of any BPDU's all be it, tagged or untagged.

Yes, the bpduguard I use by default for all end hosts that do not participate with network L2 switching. Since LTM was configured not to participate, I turned bpduguard in case of misconfig or mis-"patch" for loop prevention.