Trunk between Cisco an Aruba

[mulbreizh]

Hello and sorry for my bad english,

I have a lag/etherchannel between my Cisco C3850 and two Aruba 8100 in stack. I have troubles because there is a lot of packets discarded as you can see below on picture. 

My configuration on Cisco is like this :
interface Port-channel32
description aruba
switchport trunk native vlan 999
switchport mode trunk
switchport nonegotiate

interface TenGigabitEthernet1/0/39
description Aruba A8100
switchport trunk native vlan 999
switchport mode trunk
switchport nonegotiate
channel-group 32 mode active
!
interface TenGigabitEthernet1/0/40
description Aruba A8100
switchport trunk native vlan 999
switchport mode trunk
switchport nonegotiate
channel-group 32 mode active

and on Aruba :

interface lag 1 multi-chassis
description LACP-to-Coeur
no shutdown
no routing
vlan trunk native 999
vlan trunk allowed 1-2,5,15,17,21-22,25-26,45,51,54,56,61-62,70,89,100,102,104,110,999
lacp mode active
spanning-tree bpdu-filter
spanning-tree rpvst-filter

interface 1/1/47
description aggr LAG1 to core
no shutdown
lag 1
!
interface 1/1/48
description aggr LAG1 to core
no shutdown
lag 1

do you think this configuration is bad ? 

[mulbreizh]

i see but it is not everytime, if we wait 2 hours, i think we will see again drops

[mulbreizh]

look at this graph, i have a lot of drops when traffic is high. In pink it is drops packets

[Joseph W. Doherty]

"look at this graph"

Wow!  Possibly a text book example of drops slowing transmission rate.  Seriously, look at peak rates, followed by drops as queues overflow then followed by reduction in transmission rate and drop rate.  A classic saw tooth pattern.

"I have a lot of drops when traffic is high."

That's not unusual.

If you thought changing softmax would eliminate all your drops, that's a bad assumption.  If fact, it can make things worse (it usually doesn't).  Often increasing its value is beneficial for transient congestion but not so for sustained congestion.  The latter requires complex QoS, which switches often don't well support.  In LAN environments increasing bandwidth is.usually an easier solution.  (Of course, as you never answered my question asking for member link stats, I cannot tell if much of your bandwidth is not being used effectively.)

BTW if bandwidth is over subscribed, total elimination of drops is usually not possible.  In such cases you try to obtain the best goodput rate.  I.e. close to 100% utilization at the lowest possible drop and retransmission rates.

[MHM Cisco World]

Received 4241643 broadcasts (2507321 multicasts) <<-this number increase rapidly' I think this is issue here 

Since both broadcast and mukticast then we need to start check l2 loop.

How many link between two SW ?

Share the 

Show port-channel summary 

Or 

Show etherchannel summary 

Thanks 

MHM

[mulbreizh]

I have principaly 2 port-channel with 4 interfaces in each port-channel connect to 4 aruba stack two by two.

[MHM Cisco World]

Show ether channel/port channel summary

Show stp 

Let me check it 

MHM

[mulbreizh]

Here on image : 
also check my response to Joseph W. Doherty for description of infrastructure

[MHM Cisco World]

2 PO from Cisco SW to aruba and you run bpdufilter sure this issue here.

You generate L2 loop.

Since Aruba use bpdufilter then control plane of SW can not detect l2 loop and make both PO FWD and this lead to make broadcast ingress from one PO and egress from other make Loop effect PO and SW at all.

Check STP in cisco SW you should see Both PO FWD if that correct' inform me what is STP mode you use in Cisco SW and aruba 

Note:- As fast workaround disable one PO make your network stable until we solve STP issue between two vendors 

MHM

[Joseph W. Doherty]

---

@MHM Cisco World wrote:

You generate L2 loop.

---

Hmm, if there was an active L2 loop, I would expect things to be much worse.

[mulbreizh]

Spanning tree on cisco with the 2 port-channel to Aruba : 

Po32 Desg FWD 1 128.2408 P2p
Po33 Desg FWD 1 128.2409 P2p

In Cisco i use RVPST and on Aruba is MSTP

[MHM Cisco World]

Yes mismatch mode, and what make case worse is you disable stp from Aruba side by run bpdufilter this lead to l2 loop.

As I mentioned disable one PO and check if network be stable or not. 

MHM

[mulbreizh]

Yes on Aruba side, it is configure like this :

interface lag 1 multi-chassis
description LACP-to-Coeur
no shutdown
no routing
vlan trunk native 999
vlan trunk allowed 1-2,5,15,17,21-22
lacp mode active
spanning-tree bpdu-filter
spanning-tree rpvst-filter

you think is better to disable spanning-tree bpdu-filter from Aruba side ?

[MHM Cisco World]

No unfortunately' 

Mismatch stp mode solve by 

1- use bpdufilter in one side BUT thatnwork ONLY if there is one link conmect two SW

2- use MST in both SW' as you mention you use rpvst and aruba use mst' so you need to  change mode in cisco to use mst.

But before change stp mode let be sure that this issue from loop' 

Did you disable one PO?

MHM

[mulbreizh]

Aruba switch was configure by external company, i see that Aruba have rpvst capability

8100-48X-E011-124(config)# spanning-tree mode
mstp Multiple spanning tree mode(Default)
rpvst Rapid PVST mode

so the best way is to change MST to rpvst  on Aruba ?

i will test disable one PO

[MHM Cisco World]

Rpvst for other vendor different than cisco' cisco use it stp except mst cisco use IEEE standard which is same for all other vendor.

But to be more check I will do double review compatibility between rpvst of aruba and cisco 

Until that time disable one PO and see if there is anymore traffix burst in monitor view.

MHM