Don't use packet-duplication with AAR, it doesn't work properly

[dijix1990]

I have general policy for all my edges (isr1111x-8p / isr4331) and recently I changed one of my isr4331 to c8200 and found that pkt dup doens't work, before it worked.

Policy from vSmart accepted, counter for my traffic which is needed to use pkt-dup increase (sh sdwan policy data-policy-filter), pktdup-capable is true, but when I do command sh sdwan tunnel statistics pkt-dup I see that counter is zero

pktdup-rx 0
pktdup-rx-other 0
pktdup-rx-this 0
pktdup-tx 0
pktdup-tx-other 0
pktdup-capable true

[dijix1990]

for 17.12.3a the same situation

[dijix1990]

It started to work with software 17.12.4, but it's funny and awful the same time, I bumped into a bug which I found almost 3 years ago, aar policy + pkt dup doesn't work correctly. I have aar for sending some of my traffic via Internet (two channels) only (I have extra channel - mpls for voip) and after upgrade traffic duplicate to mpls. I did research, checked that everything works correctly for isr4331 (17.9.5) and upgrade it to 17.12.4 and got the bug... So 17.12.4 has stupid bug, data policy override aar policy (https://www.networkacademy.io/ccie-enterprise/sdwan/aar-alongside-data-policy) 

[dijix1990]

So it does not work properly and it is unknown when it will be, in complex variations when you have 2 Internet channels and an mpls channel, you cannot adequately apply the AAR policy (when you need to send voice traffic to mpls, and the rest of the traffic to the Internet)+ pkt-dup for Internet channels, at first it seems to work, but then I start to notice that traffic that should not go to mpls appears there, that is, it mixes, data policy, as stated in conjunction with AAR, does not work normally, cisco tac confirms this, but when it is corrected it is not known, the recommendation, use channels of the same size to duplicate traffic on all Internet + mpls channels, for three years now I have been looking at cisco sdwan and I understand that compared to vmware this is an extremely crude product

[dijix1990]

For my experience:

I have branches with 2 ISP's (100 mbit) and 1 mpls (5 mbit)

AAR policy from branch for VPN's 10-11,14-15

from-vsmart app-route-policy _VPN-Branch-AAR-data_aar-data-branch-with-l3vpn
 vpn-list VPN-Branch-AAR-data
  sequence 1
   match
    source-ip      172.19.0.0/16
    destination-ip 10.10.0.0/16
   action
    count                      aar-data-jabber-dc_1517252540
    backup-sla-preferred-color biz-internet public-internet
    sla-class       Realtime
    no sla-class strict
    sla-class preferred-color mpls
  sequence 11
   match
    source-ip      172.19.0.0/16
    destination-ip 172.19.0.0/16
    dscp           24 40 46
   action
    count                      aar-data-jabber-br_1517252540
    backup-sla-preferred-color biz-internet public-internet
    sla-class       Realtime
    no sla-class strict
    sla-class preferred-color mpls
  sequence 21
   match
    source-ip      0.0.0.0/0
    destination-ip 0.0.0.0/0
   action
    count aar-data_1517252540
    sla-class       Default
    no sla-class strict
    sla-class preferred-color biz-internet public-internet
 

 AAR policy from DC for VPN 1 and 14

from-vsmart app-route-policy _VPN1-Fusion
 vpn-list VPN1-Fusion
  sequence 1
   match
    source-ip      10.10.0.0/16
    destination-ip 172.19.0.0/16
   action
    count                      aar-data-jabber_-1471657029
    backup-sla-preferred-color biz-internet public-internet
    sla-class       Realtime
    no sla-class strict
    sla-class preferred-color mpls
  sequence 11
   match
    source-ip                    0.0.0.0/0
    destination-data-prefix-list cc_networks
   action
    count aar-data-cc_-1471657029
    sla-class       cc_default
    no sla-class strict
    sla-class preferred-color biz-internet public-internet
  sequence 21
   match
    source-ip      0.0.0.0/0
    destination-ip 0.0.0.0/0
   action
    count aar-data-other_-1471657029
    sla-class       Default
    no sla-class strict
    sla-class preferred-color biz-internet public-internet

Before data policy AAR works perfect,

Now I want to use pkt-dup function

Data policy from branch (only for VPN 14 from service)

 data-policy Data-Policy-from-branch-vpn14
    sequence 1
     match
      source-ip 172.17.135.0/24
      destination-ip 172.17.133.0/24
     !
     action accept
      count pkt-dup-vpn14
      loss-protect pkt-dup
      loss-protection packet-duplication
      set
       local-tloc-list 
        color biz-internet public-internet
        encap ipsec
        restrict
      !
     !
    !
  default-action accept

Data policy from DC (only for VPN 14 from service)

 data-policy Data-Policy-from-dc-vpn14
    sequence 1
     match
      source-ip 172.17.133.0/24
      destination-ip 172.17.135.0/24
     !
     action accept
      count pkt-dup-vpn14
      loss-protect pkt-dup
      loss-protection packet-duplication
      set
       local-tloc-list 
        color biz-internet public-internet
        encap ipsec
        restrict
      !
     !
    !
  default-action accept

so, by logic packet flow AAR polciy for branches ( sequence 21) meets with sla and AAR policy matches with data policy.

AAR polciy for DC ( sequence 21) meets with sla and AAR policy matches with data policy. Traffic needs to be works via internet color only, but on the zabbix I can see that traffic goes into MAIN IN / BACKUP IN and OUT and VOIP OUT although Traffic must go MAIN IN and OUT / BACKUP IN and OUT

BTW after reloading traffic goes correctly sometime and don't use VOIP (MPLS COLOR)