Default Routes Cause CLI Lag

[JRChadwick]

I am deploying a ring of Cisco Catalyst 9600 series layer 3 switches and have been struggling to resolve some odd issues.

The devices require a default route to be configured for anyone connected to the switches to reach anything outside of their immediate network.  However, once said default route is configured, the devices suffer extreme lag.  It makes it very difficult to manage them because we have to wait several seconds for the devices to respond to ever command entered.

We also have no power redundancy.  The devices are connected to 15 Watt outlets on UPS which should be supplying the necessary 220V and the power redundancy-mode has been set to redundant but turning off one of the three power supplies causes the two blades with all the access ports to be set to power-deny on a show mod.

This is my first post to this board and any assistance would be greatly appreciated.

[Giuseppe Larosa]

Hello @JRChadwick ,

just to clarify you have built an OSI layer 2 ring or the switches are connected to each other using routed ports ?

in case it is a L2 ring are all the switches pointing to the same default gateway for their default route?

If using L3 links between devices you should run a routing protocol like OSPF and only one or two devices should be injecting a default route in OSPF domain.

with lag you mean a great delay ?

Hope to help

Giuseppe

[JRChadwick]

@Giuseppe Larosa Thank you for your reply.

The backbone links between all the switches are layer 2 connections.

The default routes are the interface VLAN IP addresses for the backbone ports of the neighboring switch.

The switches are running OSPF and are not having any issues forming neighbor adjacencies.

Yes, by lag I mean a great delay.

[Giuseppe Larosa]

Hello @JRChadwick ,

if the links are L2 and you are running OSPF this means this  is a single network segment  / VLAN / broadcast domain.

The device connecting to the backbone could inject a default route in OSPF.

Are the switches in area  0 or in a non backbone area ?

if using a stub area or NSSA area the ABR(s) can inject a default route in OSPF.

However, your issues look like related to the way the ring topology is managed.

How  many switches are in the L2 ring ?

if you are using STP this is a very important number.

if the number of switches is for example 20 or more STP with default settings even using Rapid STP can have issues in managing the topology. ( the root generated BPDUs can expire after too many switch hops)

use

show spanning tree vlan X detail

to understand if it is stable or not ( if using Rapid PVST)

Hope to help

Giuseppe

[JRChadwick]

@Giuseppe Larosa,

Yes, you are correct. There is a single OSPF area, which is Area 0.

Injecting the default route in router ospf 1 is an interesting solution to consider, I'll research it and possibly try it out.

There are seven switches in the ring I'm troubleshooting.   That amount shouldn't be causing such issues.

Below is the output for one of the backbone VLANs.

169#show spanning-tree vlan XXXX detail

VLANXXXX is executing the rstp compatible Spanning Tree protocol
Bridge Identifier has priority 32768, sysid XXXX, address YYYY.YYYY.YYYY
Configured hello time 2, max age 20, forward delay 15, transmit hold-count 6
We are the root of the spanning tree
Topology change flag not set, detected flag not set
Number of topology changes 2 last change occurred 01:02:16 ago
from TwentyFiveGigE1/0/46
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0, aging 300

Port 46 (TwentyFiveGigE1/0/46) of VLANXXXX is designated forwarding
Port path cost 2000, Port priority 128, Port Identifier 128.46.
Designated root has priority 36768, address YYYY.YYYY.YYYY
Designated bridge has priority 36768, address YYYY.YYYY.YYYY
Designated port id is 128.46, designated path cost 0
Timers: message age 0, forward delay 0, hold 0
Number of transitions to forwarding state: 1
Link type is point-to-point
Loop guard is enabled by default on the port
BPDU: sent 1871, received 4

[Georg Pauwen]

Hello,

when you say ring topology, what does that actually look like ? Do you have a drawing ? The output you have posted is from the root switch, what is curious is that there have been two topology changes during the last hour. You might want to configure the root switch with the 'spanning-tree vlan x priority' command...

[JRChadwick]

@Georg Pauwen @Giuseppe Larosa

Sorry for the delay in continuing to reply.  Calling it a ring isn't completely accurate, there are a few redundant paths, though most switches only have two backbone links.  There is a second ring joined to the one I'm currently working on cutting over but currently, none of the components of the second ring are taking any network load.  We have halted the cutover from continuing until we can solve the issue.  I've tried to open up a ticket with Cisco but I don't know my company's Cisco ID number.  I've asked my company to give that to me but they haven't responded yet.

[JRChadwick]

@Georg Pauwen @Giuseppe Larosa

It's been a while since I posted.  It turns out that the design engineer who made the original configurations added an ISE server that didn't exist.  For some reason, when a default route is added, CLI commands will apparently be confirmed through ISE, fails over from the nonexistent ISE server, and then passed by the correct ISE server address that was also part of the configurations.

[Georg Pauwen]

Hello,

who would have thought of that ! Good that you finally found out what was causing the lag...