Catalyst 3850 - Output drops - Buffer tweak applied but seeing HSRP and OSPF Flaps

cadburysch · ‎08-29-2017

Good afternoon, i ran into the known issue with seeing output drops on some 3850 interfaces, especially when under load.

After referencing the following Cisco.com article, i applied the work around to tweak the softmax buffer, which resolved the issue with output drops.

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-3850-series-switches/200594-Catalyst-3850-Troubleshooting-Output-dr.html

Unfortunately, i seem to now have run into the issue which is highlighted in that some control packets are being dropped as i'm seeing the OSPF and HSRP adjacencies with neighbour switch go down periodically.

The document suggests a two class model to preserve the priority queue however i'm having trouble with a class map that will correctly identify control traffic which i can use as part of a policy map to preserve said priority queue.

Has anyone had any success with this please?

CURRENT CONFIGURATION

qos queue-softmax-multiplier 1200

!

policy-map NODROP
class class-default
bandwidth percent 100
queue-buffers ratio 100

!

interface GigabitEthernet1/0/24
description OSPF_ADJACENCY
no switchport
ip address X.X.X.X / X.X.X.X
ip pim sparse-dense-mode
ip ospf dead-interval 3
ip ospf hello-interval 1
service-policy output NODROP

!

interface GigabitEthernet1/1/1
description TRUNK
switchport mode trunk
switchport nonegotiate
service-policy output NODROP

PROPOSED CONFIGURATION

qos queue-softmax-multiplier 1200

!

policy-map NODROP

class CTRL-TRAFFIC - TROUBLE FINDING A MATCH STATEMENT THAT WILL WORk

priority level 1 percent 10
class class-default
bandwidth percent 90
queue-buffers ratio 90

bersoare · ‎08-29-2017

Hi,

You can either use an access-list to match the traffic, then associate it under a class-map or you can configure matching the protocols explicitly under the class map.

Example:

ip access-list extended NODROP

permit pim any any

permit ospf any any

permit udp any any eq 1985 (hsrp)

and so on...

class-map match-any NODROP

match protocol pim

match protocol ospf

By default, most control plane traffic (PIM, OSPF, EIGRP, BGP) are marked with ip prec of 6. So, unless you are remarking traffic, you can simply use a match on ip precedence of 6.

Thanks.

cadburysch · ‎08-29-2017

Thank you for the detailed explanation, will give it a go.

Regards

cadburysch · ‎08-29-2017

Configuration has been applied, will see if we coninue to experience adjaceny drops, looking at the stats we seem to have traffic in the priority queue but not matching IP PREC

show platform qos queue stats gigabitEthernet 1/1/1
DATA Port:0 Enqueue Counters
-------------------------------
Queue Buffers Enqueue-TH0 Enqueue-TH1 Enqueue-TH2
----- ------- ----------- ----------- -----------
0 0 0 0 284408
1 0 0 0 2387735198
2 0 0 0 0
3 0 0 0 0
4 0 0 0 0
5 0 0 0 0
6 0 0 0 0
7 0 0 0 0
DATA Port:0 Drop Counters
-------------------------------
Queue Drop-TH0 Drop-TH1 Drop-TH2 SBufDrop QebDrop
----- ----------- ----------- ----------- ----------- -----------
0 0 0 0 0 0
1 0 0 0 0 0
2 0 0 0 0 0
3 0 0 0 0 0
4 0 0 0 0 0
5 0 0 0 0 0
6 0 0 0 0 0
7 0 0 0 0 0
AQM Broadcast Early WTD COUNTERS(In terms of Bytes)
--------------------------------------------------
PORT TYPE ENQUEUE DROP
--------------------------------------------------
UPLINK PORT-0 N/A 0
UPLINK PORT-1 N/A 0
UPLINK PORT-2 N/A 0
UPLINK PORT-3 N/A 0
NETWORK PORTS 0 0
RCP PORTS 0 0
CPU PORT 0 0
Note: Queuing stats are in bytes

----------------------------------------------------------------------

sh policy-map interface g1/1/1
GigabitEthernet1/1/1

Service-policy output: NODROP

queue stats for all priority classes:
Queueing
priority level 1

(total drops) 0
(bytes output) 195264

Class-map: CTRL-TRAFFIC (match-any)
0 packets
Match: ip precedence 6
0 packets, 0 bytes
5 minute rate 0 bps
Priority: 10% (100000 kbps), burst bytes 2500000,

Priority Level: 1

Class-map: class-default (match-any)
0 packets
Match: any
Queueing

(total drops) 0
(bytes output) 1564761243
bandwidth 90% (900000 kbps)
queue-buffers ratio 90

bersoare · ‎08-30-2017

sh policy-map interface g1/1/1
GigabitEthernet1/1/1

Service-policy output: NODROP

queue stats for all priority classes:
Queueing
priority level 1

(total drops) 0
(bytes output) 195264 <<<<

It looks like it is matching, however the policy-map counters are not increasing, even for class-default:

Class-map: class-default (match-any)
0 packets <<
Match: any
Queueing

Might be a cosmetic bug. Please monitor the adjacencies and see if it will flap again.

cadburysch · ‎08-31-2017

Thank you for your help, unfprtunately the policy didnt have any effect so have reverted to the original.

Increasing the timers has resolved it though.

Hello interval set to 3 seconds and dead time to 10 seconds.

Regards