SD-WAN AAR Strict option

[MichielVercoutter]

Hi,

I have a question about how the AAR SLA Class strict option works (vManage version 20.9, IOS-XE cEdges).

Imagine a router with 2 colors: mpls and public-internet

By using an SLA class, you want to force FTP traffic over the public-internet color.

When the SLA class fails for the public-internet color, the FTP traffic should be dropped. It should never use the mpls color, even if this one meets the SLA while the public-internet color does not.

For this requirement, is it sufficient to have an AAR policy that matches FTP traffic, and sets action preferred color public-internet + strict option? Or will this cause it to fall back to mpls if this color matches the SLA while public-internet doesn't?

The documentation (https://www.cisco.com/c/en/us/td/docs/routers/sdwan/configuration/policies/ios-xe-17/policies-book-xe/application-aware-routing.html) states the following:

" sla-class sla-class-name preferred-color color—To set a specific tunnel to use when data traffic matches an SLA class, include the preferred-color option, specifying the color of the preferred tunnel. If more than one tunnel matches the SLA, traffic is sent to the preferred tunnel. If a tunnel of the preferred color is not available, traffic is sent through any tunnel that matches the SLA class. If no tunnel matches the SLA, data traffic is sent through any available tunnel. In this sense, color preference is considered to be a loose matching, not a strict matching, because data traffic is always forwarded, whether a tunnel of the preferred color is available or not.

If no tunnel matches the SLA, data traffic is sent through any available tunnel. In this sense, color preference is considered to be a loose matching, not a strict matching, because data traffic is always forwarded, whether a tunnel of the preferred color is available or not. When no tunnel matches the SLA, you can choose how to handle the data traffic:

• strict—Drop the data traffic."

This would lead me to believe it would fall back to mpls. However, during my labbing, FTP seems to get blackholed when the SLA is down for public-internet, even when the SLA is still met for the mpls color.

Another way of asking the same question: can AAR by itself be used to force traffic over a specific color (and only this color)? Or do you need to accompany it with a Traffic Data policy to make sure it doesn't fall back to a color that meets the SLA Class?

Thanks.

[Kanan Huseynli]

Hello,

what version of route do you use?

I also surprised from the statements of the docs, I used to know that preferred color+strict will work, however it seems it doesn't (based on doc).

In your case, below should work:

AAR: preferred-color internet+strict configured

DataPolicy: Local-color FTP (even you may add strict here as well, but seems not mandatory)

Below is the picture from Ciscolive(Advanced SD-WAN Policies Troubleshooting / BRKENT-3797 / 2024):

When there is path matching SLA and Data Policy shows different colors, then STRICT option from AAR is checked. If AAR strict is configured, then traffic is drop.

[MichielVercoutter]

The router is an 8000v running 17.09.03a.

If my lab is correct, only using AAR preferred-color internet+strict (without using an extra data policy) does give the required result, but this contradicts the documentation.

I did see the Cisco Live presentation, but in my case no data policy is used, only AAR, so i'm not sure if the result is the same.

[Kanan Huseynli]

I always supposed and know that, if strict+preferred color is configured, then traffic is drop if none of preferred color matches SLA (the same behavior from your lab). BUT whatever official doc I have checked, none of them shows this behavior.

Just checking: do you have route over both interfaces towards remote network (where FTP server)? AAR works only over routing-ECMP paths.

[MichielVercoutter]

Yes I have a route over both interfaces, see the screenshots of my other comment.

When I introduce delay over the public-internet transport, the 'Simulate Flows' feature shows blackhole for the TCP/21 I match in the AAR policy, but still shows 4 paths for any other TCP port, so it is definitely the AAR policy that is blocking the traffic.

[MHM Cisco World]

let me double check 

MHM

[MichielVercoutter]

If you follow this diagram for the case where public-internet is down and mpls is up, you get the following flow:

This is in line with the Cisco documentation I listed, but in my lab, only using AAR preferred-color internet+strict (without using an extra data policy) does give the required result

[MHM Cisco World]

In your Lab

Show sdwan app-route stats remote-system-ip x.x.x.x

Share this please let me look

MHM

[MichielVercoutter]

This is the AAR policy that is applied (no data policy):

The SLA class i'm testing with has index 1:

r21#show sdwan app-route sla-class
APP PROBE
INDEX NAME LOSS LATENCY JITTER CLASS ID APP PROBE CLASS FALLBACK BEST TUNNEL
-------------------------------------------------------------------------------------------------------------------------------------------
0 __all_tunnels__ 0 0 0 0 None None
1 TEST-SLA 10 50 200 0 None None

When I increase the latency on public-internet, you can see only the mpls<->mpls matches the SLA:

r21#show sdwan app-route stats remote-system-ip 1.1.1.51
app-route statistics 10.2.254.22 10.0.52.2 ipsec 12386 12386
 remote-system-ip         1.1.1.51
 local-color              mpls
 remote-color             mpls
 sla-class-index          0,1
 fallback-sla-class-index None
 app-probe-class-list None
  mean-loss    0
  mean-latency 2
  mean-jitter  0
  interval 0
   total-packets     1
   loss              0
   average-latency   2
   average-jitter    0
   tx-data-pkts      0
   rx-data-pkts      0
   ipv6-tx-data-pkts 0
   ipv6-rx-data-pkts 0
app-route statistics 10.2.254.46 10.0.52.2 ipsec 12386 12386
 remote-system-ip         1.1.1.51
 local-color              public-internet
 remote-color             mpls
 sla-class-index          0
 fallback-sla-class-index None
 app-probe-class-list None
  mean-loss    0
  mean-latency 201
  mean-jitter  3
  interval 0
   total-packets     1
   loss              0
   average-latency   201
   average-jitter    3
   tx-data-pkts      0
   rx-data-pkts      0
   ipv6-tx-data-pkts 0
   ipv6-rx-data-pkts 0

This causes traffic to be blackholed:

Other TCP ports still work as expected so no routing issue:

So it seems to give the result that I want, but according to the documentation, it should fall back to mpls (which it doesn't) and only blackhole/drop when the SLA class is not met for both colors.

[MHM Cisco World]

Last point' 

There are four path as you share 

Mpls-mpls 

Internet-internet 

Mpls-internet

Internet-mpls 

Then why the AAR show only two path (check only two path)?

Please check this 

MHM

[MichielVercoutter]

The destination site has 2 routers, one with color mpls, one with color public-internet.

The AAR only show 2 paths because it is only towards one router (1.1.1.51)

This is both routers:

r21#show sdwan app-route stats remote-system-ip 1.1.1.51
app-route statistics 10.2.254.22 10.0.52.2 ipsec 12386 12386
 remote-system-ip         1.1.1.51
 local-color              mpls
 remote-color             mpls
 sla-class-index          0,1
 fallback-sla-class-index None
 app-probe-class-list None
  mean-loss    0
  mean-latency 2
  mean-jitter  0
  interval 0
   total-packets     1
   loss              0
   average-latency   2
   average-jitter    0
   tx-data-pkts      0
   rx-data-pkts      0
   ipv6-tx-data-pkts 0
   ipv6-rx-data-pkts 0
app-route statistics 10.2.254.46 10.0.52.2 ipsec 12386 12386
 remote-system-ip         1.1.1.51
 local-color              public-internet
 remote-color             mpls
 sla-class-index          0
 fallback-sla-class-index None
 app-probe-class-list None
  mean-loss    0
  mean-latency 201
  mean-jitter  4
  interval 0
   total-packets     1
   loss              0
   average-latency   201
   average-jitter    4
   tx-data-pkts      0
   rx-data-pkts      0
   ipv6-tx-data-pkts 0
   ipv6-rx-data-pkts 0

r21#show sdwan app-route stats remote-system-ip 1.1.1.52
app-route statistics 10.2.254.22 10.0.53.2 ipsec 12386 12406
 remote-system-ip         1.1.1.52
 local-color              mpls
 remote-color             public-internet
 sla-class-index          0,1
 fallback-sla-class-index None
 app-probe-class-list None
  mean-loss    0
  mean-latency 1
  mean-jitter  1
  interval 0
   total-packets     1
   loss              0
   average-latency   1
   average-jitter    1
   tx-data-pkts      0
   rx-data-pkts      0
   ipv6-tx-data-pkts 0
   ipv6-rx-data-pkts 0
app-route statistics 10.2.254.46 10.0.53.2 ipsec 12386 12406
 remote-system-ip         1.1.1.52
 local-color              public-internet
 remote-color             public-internet
 sla-class-index          0
 fallback-sla-class-index None
 app-probe-class-list None
  mean-loss    0
  mean-latency 202
  mean-jitter  2
  interval 0
   total-packets     1
   loss              0
   average-latency   202
   average-jitter    2
   tx-data-pkts      0
   rx-data-pkts      0
   ipv6-tx-data-pkts 0
   ipv6-rx-data-pkts 0

[ChrisNewnham_]

This is interesting, like you say an AAR policy *should not* have the ability to blackhole traffic if at least 1 TLOC path is meeting SLA. This is often a source of confusion, the difference between AAR and "application pinning" (which does require a Traffic Data Policy like you say).

Are you completely sure that the mpls circuit is still meeting SLA? My working assumption would be this is a bug, hopefully someone from Cisco will see this.

[MichielVercoutter]

Hi Chris,

In one of my other comments, I show the output of command "show sdwan app-route stats remote-system-ip x.x.x.x" to the 2 cEdges of the destination site.

For both the app-route stats with local color mpls, the sla-class-index shows "0,1". If I understand this output correctly, this means the SLA class I defined (which has index 1) is met. The app-route stats with local color public-internet have sla-class-index "0", which shows the SLA is not met (because the mean-latency of 201 > the max latency of 50 I put in my SLA class).

Even with this state, I get a blackhole result when using the 'Simulate Flows' feature for the TCP/21 I match in the AAR policy. For any other TCP port, it still shows all paths to that destination (also see my screenshots in the other comment).

I am running this in a virtual environment with 8000v routers, so this could indeed be a bug.

Thanks for confirming that the behaviour I am seeing is not expected.

[dijix1990]

1. If you want to drop traffic because it doesn't meet sla you need strict option in AAR (sets action preferred color public-internet + strict option)

2. For you plans it's enough only using AAR (can AAR by itself be used to force traffic over a specific color (and only this color)?

I use AAR + data policy for packet duplication 

In my environment Ftp only goes via public-Internet and biz-Internet (aar + strict option) and after it duplication via both channel, data policy with match ftp and action local-tloc (public-internet and biz-internet) + restrict option and loss correction

Notice that it can be used only from release 20.6.1

https://www.cisco.com/c/en/us/td/docs/routers/sdwan/configuration/policies/ios-xe-17/policies-book-xe/application-aware-routing.html

Click Strict/Drop to perform strict matching of the SLA class. If no data plane tunnel is available that satisfies the SLA criteria, traffic is dropped.

[peradetlic · Accepted solution]

I know it's been a long time, but there is a bug in vManage 20.9. The simulate flow will always show like the traffic is blakcholed
See the bug CSCwe49131
You can use command directly on cEdge cli to test what color is used for specific protocol/port:

show sdwan policy service-path vpn 3 interface gig3.10 source-ip.......