Spanning tree port type edge vs switchport mode trunk

[zheepern0826]

Hi All,

Recently we encounter some issue on reaching our server in the server farm.

What’s going on is that everything seems to be OK, but then out of nowhere, we will get communication failures between specific machines. It looks like it’s an ARP issue. Using PING, it works fine in one direction, but we get an “unreachable” error when going the other way, unless we ping from the target back to the source first.

For example: we have servers, “A” and “B”. Ping A to B fails with “unreachable”. Ping “B” to “A” works fine. However after pinging “B” to “A”, we can now ping “A” to “B”, at least for a while until the entry falls out of the ARP cache. If we go into server “A” and set a static ARP entry (“arp –s”) for server “B”, everything works OK. Through all this both server “A” and server “B” have no issues communicating with any other machines.

We have do some failover test on the core switch and monitor the arp entry. We notice when we are using primary, everything works fine. When failover to secondary, we start having the intermittent issue and found out the arp entry was not tally in the arp table compare with the primary core switch.

We tried google around and notice some similar case in the vmware community.
https://communities.vmware.com/thread/421560

In the end of the post, they mention we need to configure the "spanning-tree port type edge trunk" on the interface where our server is connected.

Below is the current config on the FEX where the server is connected.
interface ethernet 101/1/15
channel-group 200
!
interface ethernet 101/1/16
channel-group 200
!
interface port-channel 200
description ESXServer
switchport mode trunk
switchport trunk allowed vlan 1,2,3,4,5

From what i understanding, spanning-tree port type edge basically is like the portfast in catalyst switch, set the port to forwarding without passing through listening and blocking state.
https://www.cisco.com/c/m/en_us/techdoc/dc/reference/cli/nxos/commands/l2/spanning-tree-port-type-edge.html

Can anyone advise on this ?
Thank you very much in advance.
Eric

[nazimkha]

As you mentioned the 'spanning-tree port type edge' is similar to port-fast

Configuring it may not solve the ping problem but will not hurt and will conserve compute resources.

BTW what is your topology like ? Did you get it troubleshooted with Cisco TAC?

[David Castro F.]

Hello Eric,

I hope you are doing great,

I was analizing the information you just explained and I have a couple of questions:

- Is the ESXi NICs configured as "active"/"standby" or a teaming between the 2 vnics?

- Have you checked the logs to see if there is any mac-address move changed? you could use a command like:

Nexus-5000(config)# mac address-table notification mac-move

Nexus-5000(config)# Logging level spanning-tree 6
Nexus-5000(config)# Logging level fwm 6
Nexus-5000(config)# Logging monitor 

N7K-1 %L2FM-4-L2FM_MAC_MOVE:

You will get a log as above and some more indicating that there is a mac move, now why am I recommending that?, usually if you have a Firewall in your network it could a proxy-arp for networks that it has directly attached or not directly attached, and that could cause an issue like the one you are getting, also depending on how you have configured the VDS switch VNICs on the vsphere it could also cause an issue of active/active vnics stating that they have the MAC address and having a whole MAC-address learning on the N5k for those VLANs, there are somehow possibilities for your scenario, though it would be helpful if you replicate the scenario of the ping and gets us the following info:

Before replicating:

N5ks
- Show mac address-table

- show log last 20

If you have an ASA:

- show arp

- show run all sysopt

- show run nat (only the NATs involved in the IP addressing for those VLANs)

After replicating the issue:

N5ks
- Show mac address-table

- show log last 20

If you have an ASA:

- show arp

Also check the time MAC address expiring time in the N5K,

Keep us posted!

Please qualify all the helpful answers!

Thanks,

David Castro,

[paul driver]

Hello

Fex host interfaces dont support stp ,Do you need a PC trunk to the server? can it not be PC in access mode.

What is the Nx-OS release are your running?