DNAC Application policy causes ISIS neighbours to drop

[eagles-nest]

Hi

Excuse me if I'm doing this wrong but I posted a discussion in the DNA Center board but think it may be better posted here.  The discussion is at

https://community.cisco.com/t5/cisco-digital-network/dna-center-application-policy-causes-isis-neighbors-to-drop/td-p/4004507 

I tried to see if I could move the discussion here but couldn't find a way to do so.

In short I have an issue where I was testing DNAC application policies.  When I deployed a test policy to my fabric the ISIS neighbours all dropped.  I was able to resolve this by connecting to each device via the management interface and remove the policy from the /30 interlinks between Border and Edge.  I expected a pretty standard QoS policy to be pushed out that I could inspect and tweak to suit my needs.  However, this has caused quite a serious issue in that DNAC can push out a policy which effectively destroys my underlay.  I am using DNAC version 1.3.1.4 with Cat9200 Edge devices and a Cat9300 Border?Control node.  This is just a lab test environment but the implications of this in a live fabric would be very serious.

Has anyone seen this before?

Thanks for any input, Stuart.

[eagles-nest]

This seems specific to my edge devices.  I have 9200's at the edge and a 9300 Border/control node.

When I initially deployed the Application policy the ISIS neighbours dropped.  I have changed the deployment for troubleshooting purposes and just deployed to the Border.  So there is now a QoS policy on the /30 interfaces connected to the edge devices.  The ISIS neighbours do not drop.  As soon as I apply the policy to the Edge the neighbours drop.

I then log onto the devices and manually remove the policy from the Edge and the ISIS neighbours immediately re-form.  At this point the border still has the policy.

So it seems that the issue is the edge devices drop ISIS frames when the policy is applied to the links to the Core.

Regards, Stuart.

[jedolphi]

Hi Stuart,

I've searched previous TAC cases and known issues and could not find anything. What you are describing should work. I hope you are running SDA supported code versions on all the switches? It looks like your best choice right now for 9200 is 16.12.1s (note the 's' , 16.12.1-not-s is not certified), then later once it comes out (next few weeks probably) the next version of 16.12.x  https://www.cisco.com/c/en/us/solutions/enterprise-networks/software-defined-access/compatibility-matrix.html

Best regard, Jerome

[eagles-nest]

Jerome

Thank you for your reply.  I am running 16.11.1 at the edge but not 1c as mentioned in the support matrix.  1c (and 12.1s) are not on the downloads area of CCO.  Only 16.11.1 and 16.12.1 are.  However if I click in the image name in the compatibility matrix it takes me to another area, special release, where the c and s images are.  Strange they are not in the standard product downloads area.

I will change to the c or s version when I get a change and update.  Is there any stipulation as to whether to use bundle mode or install mode?

Thanks for your input, Stuart.

[eagles-nest]

Jerome

I have upgraded to 16.12.1c in install mode (if that's relevant) and the same problem has occurred.  As can be seen below immediately DNAC (10.0.0.1) applies the policy the ISIS neighbours drop.  When I manually remove the policy from one interface its neighbour recovers.  Something in that policy dumps ISIS but the policy is DSCP based so I can't work out why it would affect ISIS.  The interfaces are logging output drops which I am assuming is the ISIS traffic.  There is no data traffic of any type flowing here.

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
* 1 32 C9200-24P 16.12.1s CAT9K_LITE_IOSXE INSTALL

Edge-1#
*Jan 2 2020 12:17:26.618 UTC: %SYS-5-CONFIG_I: Configured from console by XXXXX on vty1 (10.0.0.1)
*Jan 2 2020 12:17:33.932 UTC: %CLNS-5-ADJCHANGE: ISIS: Adjacency to Border (GigabitEthernet1/0/24) Down, neighbor forgot us
*Jan 2 2020 12:17:34.055 UTC: %CLNS-5-ADJCHANGE: ISIS: Adjacency to Border (GigabitEthernet1/0/23) Down, neighbor forgot us
*Jan 2 2020 12:17:34.217 UTC: %CLNS-5-ADJCHANGE: ISIS: Adjacency to Border (GigabitEthernet1/0/23) Down, neighbor forgot us
*Jan 2 2020 12:17:34.833 UTC: %CLNS-5-ADJCHANGE: ISIS: Adjacency to Border (GigabitEthernet1/0/24) Down, neighbor forgot us
% Incomplete command.

Edge-1#show isis nei
Edge-1#show isis neighbors

System Id Type Interface IP Address State Holdtime Circuit Id
Border L1 Gi1/0/23 172.16.2.1 INIT 23 Edge-1.01
Border L2 Gi1/0/23 172.16.2.1 INIT 21 Edge-1.01
Border L1 Gi1/0/24 172.16.2.5 INIT 22 Edge-1.02
Border L2 Gi1/0/24 172.16.2.5 INIT 24 Edge-1.02
Edge-1#
Edge-1#show run int g1/0/23
Building configuration...

Current configuration : 201 bytes
!
interface GigabitEthernet1/0/23
description ## Link to Border-1 G1/0/21 ##
no switchport
ip address 172.16.2.2 255.255.255.252
ip router isis
service-policy output DNA-dscp#APIC_QOS_Q_OUT
end

Edge-1#show run int g1/0/24
Building configuration...

Current configuration : 157 bytes
!
interface GigabitEthernet1/0/24
no switchport
ip address 172.16.2.6 255.255.255.252
ip router isis
service-policy output DNA-dscp#APIC_QOS_Q_OUT
end

Edge-1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Edge-1(config)#int g1/0/23
Edge-1(config-if)#no service-policy output DNA-dscp#APIC_QOS_Q_OUT
Edge-1(config-if)#end
Edge-1#
*Jan 2 2020 12:19:02.475 UTC: %CLNS-5-ADJCHANGE: ISIS: Adjacency to Border (GigabitEthernet1/0/23) Up, new adjacency
*Jan 2 2020 12:19:02.479 UTC: %SYS-5-CONFIG_I: Configured from console by XXXXX on vty0 (192.168.255.1)
*Jan 2 2020 12:19:03.628 UTC: %CLNS-5-ADJCHANGE: ISIS: Adjacency to Border (GigabitEthernet1/0/23) Up, new adjacency
Edge-1#

[jedolphi · Accepted solution]

Understood, thanks for the update. Shouldn't be happening. Please open a TAC case so we can root cause and fix for everyone. Perhaps consider posting the resolution here for others that might also hit this issue? Cheers, Jerome

[Giuseppe Larosa]

Hello @eagles-nest , @jedolphi 

in stand alone switches there is an IOS and then IOS XE system queue that can handle non IP packets like STP and IS-IS PDUs.

So I agree that this should be a new bug where the SD WAN policy is DSCP based but it is missing a system queue to deal with non IP traffic the end result is that IS-IS adjacencies are turned down

Hope to help

Giuseppe