08-21-2024 01:37 PM
Although we have configured all traffic to route through Private1(SD-WAN Router 1) using AAR, there is still some traffic passing through biz-int. We have built the SD-WAN environment with the configuration shown in the attached image.
On the vSmart, we have implemented the following settings to ensure that all data traffic is routed through Private1 using AAR.
app-route-policy Site01_AAR
vpn-list VPN3301
sequence 9999
action
count AAR_Sequence_9999
sla-class SLA_Class_0 preferred-color private1
policy
sla-class SLA_Class_0
loss 100
!
However, when checking with Zabbix (a network monitoring tool), we observed that there is traffic (about 20Mbps) passing through the WAN(Biz-internet) interface of SD-WAN Router 2.
Our expectation was that all traffic would be routed through the WAN of SD-WAN Router 1.
Do you have any idea why this might be happening? Could the traffic bursts observed in Zabbix be related to this issue?
08-22-2024 09:57 PM
sh flow monitor sdwan_flow_monitor cache format table
Cache type: Normal (Platform cache)
Cache size: 32768
Current entries: 3
High Watermark: 11
Flows added: 35
Flows aged: 32
- Active timeout ( 60 secs) 1
- Inactive timeout ( 10 secs) 31
IPV4 SRC ADDR IPV4 DST ADDR TRNS SRC PORT TRNS DST PORT IP VPN ID IP PROT tcp flags intf input intf output bytes long pkts long time abs first time abs last flow end reason conn initiator intf overlay session id input intf overlay session id output conn conn id long drop cause id bytes drop long sdwan sla not met sdwan preferred color not met sdwan queue id pkts drop long ip dscp app name
=============== =============== ============= ============= ========== ======= ========= ==================== ==================== ==================== ==================== ============== ============= ==================== ================= ============================= ============================== =================== ============= ==================== ================== ============================== =============== ==================== ======= ================================
172.18.62.16 172.26.32.1 0 2048 15 1 0x02 Gi0/0/1.936 Null 252 3 07:57:30.456 07:57:32.494 Not determined Reverse initiator 19 0 0x00000000000074CD 0 0 0 0 2 0 0x00 layer7 ping
172.18.7.22 172.26.32.1 0 2048 15 1 0x02 Gi0/0/1.938 Null 1512 18 07:57:04.159 07:57:31.317 Not determined Reverse initiator 8 0 0x00000000000074CB 0 0 0 0 2 0 0x00 layer7 ping
08-23-2024 04:48 AM
First check counter in path vedge/cedge router
Second use
Show sdwan policy service-path vpn 3301 ....
To see if both edges routers use same path or not
Share show sdwan policy service-path here
Thanks alot
MHM
08-23-2024 07:27 AM
@MHM Cisco World wrote:
First check counter in path vedge/cedge router
Second use
Show sdwan policy service-path vpn 3301 ....
To see if both edges routers use same path or not
Share show sdwan policy service-path here
Thanks alot
MHM
It was before, from Web gui, aar seems to be correct, I suggested to try to check via netflow, and maybe it's bug
08-23-2024 09:01 AM
Thank you!!
I checked the GUI and confirmed that no Localized policy was created.
It seems that the configuration was done via CLI (as described above).
When I checked several other routers with the same Netflow settings, I found that some display Cache information while others do not.
Additionally, it appears that Netflow data has not been reaching Solarwinds (the traffic monitoring tool) from both SD-WAN-Router1 and 2 for several months, so there might indeed be some kind of bug, as you mentioned.
We can not check netflow data for these router, do you have any idea that we can check?
08-23-2024 09:10 AM
Thank you for your comment!
I did show sdwan policy service-path.
And, both router use private1.
08-23-2024 09:20 AM - edited 08-23-2024 09:28 AM
Let me check one point
MHM
08-23-2024 09:29 AM
Which point?
08-23-2024 12:07 PM
@dijix1990
I could see the cache info for SD-WAN Router 1, bur no cache entry for SD-WAN Router 2..
SD-WAN-Roouter1#show flow monitor sdwan_flow_monitor cache format table
Cache type: Normal (Platform cache)
Cache size: 80000
Current entries: 55
High Watermark: 348
Flows added: 21512868
Flows aged: 21512813
- Active timeout ( 60 secs) 379891
- Inactive timeout ( 10 secs) 21132922
IPV4 SRC ADDR IPV4 DST ADDR TRNS SRC PORT TRNS DST PORT IP VPN ID IP PROT tcp flags intf input intf output flow sampler id bytes long pkts long time abs first time abs last flow end reason intf overlay session id input intf overlay session id output conn conn id long drop cause id bytes drop long sdwan sla not met sdwan preferred color not met sdwan queue id pkts drop long ip dscp app name
=============== =============== ============= ============= ========== ======= ========= ==================== ==================== =============== ==================== ==================== ============== ============= ==================== ============================= ============================== =================== ============= ==================== ================== ============================== =============== ==================== ======= ================================
192.168.210.33 192.168.228.241 8305 43029 3301 6 0x10 Vl1427 Gi0/0/0 4 104 2 04:04:48.052 04:04:57.734 Not determined 0 257 0xEA207930000A6CE5 0 0 0 0 2 0 0x00 layer7 ssl
10.242.79.210 10.240.8.17 61386 80 3301 6 0x10 Vl1427 Gi0/0/0 4 80 2 04:04:47.779 04:04:57.420 Not determined 0 250 0xEA67FE90000EA6E2 0 0 0 0 2 0 0x00 layer7 ms-sms
192.168.210.105 192.168.199.146 443 49675 3301 6 0x10 Vl1427 Gi0/0/0 4 67104 48 04:04:22.837 04:04:59.893 Not determined 0 257 0xEA9763500018A5E2 0 0 0 0 2 0 0x00 layer7 ssl
192.168.210.2 10.240.58.38 0 0 3301 1 0x00 Vl1427 Gi0/0/0 4 168 2 04:04:51.554 04:04:56.958 Not determined 0 255 0x00000000002D992E 0 0 0 0 2 0 0x00 layer7 ping
10.242.75.5 10.242.82.8 23711 3777 3301 6 0x18 Vl1427 Gi0/0/0 4 2224 2 04:04:55.652 04:04:58.133 Not determined 0 269 0xEA4299300015F412 0 0 0 0 2 0 0x00 layer7 statistical-p2p
192.168.214.253 192.168.210.33 42356 8305 3301 6 0x10 Gi0/0/0 Vl1427 4 1398 1 04:04:56.892 04:04:56.892 Not determined 255 0 0xEA4197F0000CEF52 0 0 0 0 2 0 0x00 layer7 unknown
10.240.8.17 10.242.79.210 80 61386 3301 6 0x10 Gi0/0/0 Vl1427 4 2600 2 04:04:48.272 04:04:55.646 Not determined 250 0 0xEA67FE90000EA6E2 0 0 0 0 2 0 0x00 layer7 ms-sms
10.242.73.3 10.240.164.20 53 50355 3301 17 0x00 Vl1427 Gi0/0/0 4 338 1 04:04:59.037 04:04:59.037 Not determined 0 249 0xEA2185600007678D 0 0 0 0 2 0 0x00 layer7 ms-live-accounts
10.240.58.38 192.168.210.2 0 2048 3301 1 0x00 Gi0/0/0 Vl1427 4 84 1 04:05:00.190 04:05:00.190 Not determined 255 0 0x00000000002D9932 0 0 0 0 2 0 0x00 layer7 ping
192.168.210.60 192.168.196.61 49964 161 3301 17 0x00 Vl1427 Gi0/0/0 4 82 1 04:04:51.879 04:04:51.879 Not determined 0 250 0xEA4784B000192517 0 0 0 0 2 0 0x00 port snmp
10.242.76.66 192.168.196.125 59251 445 3301 6 0x10 Vl1427 Gi0/0/0 4 41 1 04:04:56.186 04:04:56.186 Not determined 0 250 0xEFAEF5600013F63F 0 0 0 0 2 0 0x00 layer7 cifs
192.168.210.33 10.241.119.220 8305 49459 3301 6 0x10 Vl1427 Gi0/0/0 4 52 1 04:04:58.495 04:04:58.495 Not determined 0 258 0xEA42DA70000D5F3D 0 0 0 0 2 0 0x00 layer7 unknown
192.168.238.5 192.168.210.60 0 0 3301 1 0x00 Gi0/0/0 Vl1427 4 51 1 04:04:57.933 04:04:57.933 Not determined 257 0 0x00000000002D9931 0 0 0 0 2 0 0x00 layer7 ping
192.168.210.33 192.168.214.253 8305 42356 3301 6 0x10 Vl1427 Gi0/0/0 4 52 1 04:04:52.380 04:04:52.380 Not determined 0 255 0xEA4197F0000CEF52 0 0 0 0 2 0 0x00 layer7 unknown
10.242.82.8 10.242.75.5 3777 23711 3301 6 0x18 Gi0/0/0 Vl1427 4 156 3 04:04:50.659 04:04:58.166 Not determined 269 0 0xEA4299300015F412 0 0 0 0 2 0 0x00 layer7 statistical-p2p
10.242.73.1 10.241.96.113 5246 5252 3301 17 0x00 Vl1427 Gi0/0/0 4 125 1 04:04:50.816 04:04:50.816 Not determined 0 257 0xE9EDC400001770D7 0 0 0 0 2 0 0x30 port capwap-control
10.242.104.125 10.242.73.1 5256 5247 3301 17 0x00 Gi0/0/1.30 Vl1427 4 195 1 04:04:56.369 04:04:56.369 Not determined 135 0 0xE9F461F00017FD25 0 0 0 0 2 0 0x30 port capwap-data
192.168.210.60 10.240.65.11 53994 161 3301 17 0x00 Vl1427 Gi0/0/0 4 83 1 04:04:59.973 04:04:59.973 Not determined 0 260 0xF38E9F0000064CFC 0 0 0 0 2 0 0x00 port snmp
192.168.210.33 10.128.251.9 8305 55029 3301 6 0x10 Vl1427 Gi0/0/0 4 52 1 04:04:54.199 04:04:54.199 Not determined 0 254 0xF3A524200007D097 0 0 0 0 2 0 0x00 layer7 ssl
192.168.244.253 192.168.210.33 57709 8305 3301 6 0x10 Gi0/0/0 Vl1427 4 1398 1 04:04:54.818 04:04:54.818 Not determined 245 0 0xEA7B97900018FB23 0 0 0 0 2 0 0x00 layer7 ssl
192.168.210.2 10.240.69.43 0 0 3301 1 0x00 Vl1427 Gi0/0/0 4 84 1 04:04:55.313 04:04:55.313 Not determined 0 260 0x00000000002D992F 0 0 0 0 2 0 0x00 layer7 ping
10.241.96.2 10.242.79.3 22 44884 3301 6 0x18 Gi0/0/0 Vl1427 4 912 3 04:04:54.418 04:04:57.512 Not determined 257 0 0xEA4EA2A00018CC48 0 0 0 0 2 0 0x30 port ssh
10.242.79.3 192.168.199.80 49109 161 3301 17 0x00 Vl1427 Gi0/0/0 4 181 1 04:04:53.321 04:04:53.321 Not determined 0 257 0xEA8CE2A0001062C5 0 0 0 0 2 0 0x00 port snmp
10.240.58.84 192.168.210.2 0 2048 3301 1 0x00 Gi0/0/0 Vl1427 4 84 1 04:04:57.229 04:04:57.229 Not determined 255 0 0x00000000002D9930 0 0 0 0 2 0 0x00 layer7 ping
10.242.79.3 10.241.96.2 44884 22 3301 6 0x10 Vl1427 Gi0/0/0 4 40 1 04:04:58.452 04:04:58.452 Not determined 0 257 0xEA4EA2A00018CC48 0 0 0 0 2 0 0x04 port ssh
192.168.210.2 10.240.44.6 80 61228 3301 6 0x11 Vl1427 Gi0/0/0 4 40 1 04:05:00.564 04:05:00.564 Not determined 0 255 0xEA5C1E800001DDD2 0 0 0 0 2 0 0x00 layer7 oracle-ebsuite-unsecured
10.241.250.22 192.168.210.105 445 55486 3301 6 0x18 Gi0/0/0 Vl1427 4 360 1 04:04:56.180 04:04:56.180 Not determined 270 0 0xEA6B8D0000094936 0 0 0 0 2 0 0x00 layer7 cifs
10.247.247.73 10.242.73.200 53511 445 3301 6 0x18 Gi0/0/0 Vl1427 4 328 1 04:04:52.671 04:04:52.671 Not determined 248 0 0xEA76AA0000194999 0 0 0 0 2 0 0x00 layer7 cifs
10.241.96.15 10.242.79.3 22 34172 3301 6 0x10 Gi0/0/0 Vl1427 4 40 1 04:04:51.567 04:04:51.567 Not determined 257 0 0xEA803FB0001B3F9C 0 0 0 0 2 0 0x30 port ssh
192.168.214.141 10.242.73.6 443 42812 3301 6 0x10 Gi0/0/0 Vl1427 4 968 1 04:04:53.705 04:04:53.705 Not determined 255 0 0xEA453A30000A8162 0 0 0 0 2 0 0x00 layer7 vmware-vsphere
192.168.210.60 10.240.8.28 45820 55791 3301 6 0x18 Vl1427 Gi0/0/0 4 196 1 04:04:53.082 04:04:53.082 Not determined 0 250 0xF08B7510000698CA 0 0 0 0 2 0 0x00 layer7 ms-rpc
10.240.8.27 10.242.79.204 445 54811 3301 6 0x18 Gi0/0/0 Vl1427 4 144 1 04:04:58.390 04:04:58.390 Not determined 250 0 0xF459340000145CBA 0 0 0 0 2 0 0x00 layer7 cifs
10.240.8.16 10.242.76.79 65172 50485 3301 6 0x10 Gi0/0/0 Vl1427 4 40 1 04:04:59.687 04:04:59.687 Not determined 250 0 0xF39CE69000054D73 0 0 0 0 2 0 0x00 layer7 ms-netlogon
10.80.254.10 10.242.76.32 443 52720 3301 6 0x10 Gi0/0/0 Vl1427 4 40 1 04:04:51.056 04:04:51.056 Not determined 246 0 0xEA98A440000FA42E 0 0 0 0 2 0 0x00 layer7 ssl
192.168.199.146 192.168.210.105 49675 443 3301 6 0x10 Gi0/0/0 Vl1427 4 520 13 04:04:47.092 04:04:58.976 Not determined 257 0 0xEA9763500018A5E2 0 0 0 0 2 0 0x00 layer7 ssl
10.242.73.1 10.241.96.134 5246 5248 3301 17 0x00 Vl1427 Gi0/0/0 4 109 1 04:04:55.176 04:04:55.176 Not determined 0 257 0xF470F70000132F23 0 0 0 0 2 0 0x30 port capwap-control
192.168.210.33 192.168.196.252 8305 44911 3301 6 0x10 Vl1427 Gi0/0/0 4 52 1 04:05:00.364 04:05:00.364 Not determined 0 250 0xF399E4F00010D3BC 0 0 0 0 2 0 0x00 layer7 ssl
192.168.223.54 10.242.73.3 60675 53 3301 17 0x00 Gi0/0/0 Vl1427 4 58 1 04:04:55.965 04:04:55.965 Not determined 251 0 0xEA8C8DE0000B964F 0 0 0 0 2 0 0x00 layer7 bing
10.242.73.1 10.242.104.145 5246 5248 3301 17 0x00 Vl1427 Gi0/0/1.30 4 109 1 04:04:51.462 04:04:51.462 Not determined 0 135 0xF321FD20001499F4 0 0 0 1 2 0 0x30 port capwap-control
192.168.210.2 192.168.202.207 80 59185 3301 6 0x10 Vl1427 Gi0/0/0 4 1398 1 04:04:57.828 04:04:57.828 Not determined 0 255 0xF38FD580000E4E0A 0 0 0 0 2 0 0x00 layer7 oracle-ebsuite-unsecured
192.168.194.150 10.242.73.6 443 41714 3301 6 0x10 Gi0/0/0 Vl1427 4 52 1 04:04:59.904 04:04:59.904 Not determined 259 0 0xEA8961F000116323 0 0 0 0 2 0 0x00 layer7 vmware-vsphere
192.168.228.241 192.168.210.33 43029 8305 3301 6 0x10 Gi0/0/0 Vl1427 4 1398 1 04:04:59.327 04:04:59.327 Not determined 257 0 0xEA207930000A6CE5 0 0 0 0 2 0 0x00 layer7 ssl
192.168.194.84 192.168.210.60 135 45791 3301 6 0x10 Gi0/0/0 Vl1427 4 40 1 04:04:51.513 04:04:51.513 Not determined 259 0 0xF38F574000122619 0 0 0 0 2 0 0x00 layer7 ms-rpc
SD-WAN-Roouter1
SD-WAN-Roouter2#show flow monitor sdwan_flow_monitor cache format table
Cache type: Normal (Platform cache)
Cache size: 80000
Current entries: 0
High Watermark: 75
Flows added: 296232
Flows aged: 296232
- Active timeout ( 60 secs) 23
- Inactive timeout ( 10 secs) 296209
There are no cache entries to display.
SD-WAN-Roouter2#
08-25-2024 12:17 AM
Today I bumped into a bug with aar
I had a rule (aar) for some sites and for isr1000/4000 it works, but when I added new site to rule (with c8200) I found that it doesn't work for this model, so I already haven't surprised , sdwan from cisco has extremely a lot of bugs
08-25-2024 07:25 PM
Thank you for your support!
Regarding NetFlow, there might be a bug, so I will open a case with Cisco to confirm. For the current issue, where traffic is flowing through the biz-internet despite AAR being configured to route all traffic through MPLS, I would appreciate it if you could provide any key points to check for investigating the cause.
08-26-2024 08:30 AM
Each edge in local site use mpls to forward traffic remote site'
The tloc extensions is use so one edge router is use mpls to forward traffic.
But without color strict then remote site will use
Public-private color which is more prefer than private-private
Using strict in remote site will solve issue and traffic both way will pass through mpls
MHM
08-26-2024 09:10 AM
Thank you for your comment!
We use group ID for color restrict.
Private 1 is only allowed to Tunnel to the opposite Private1.
08-27-2024 02:36 AM
Hi,
how is your routing configured? There might be case, for some remote subnets you have preferred (or only) route via biz-int and AAR is not subject for this destination, thus you have some traffic on biz-int.
11-09-2024 03:18 AM
@dijix1990 You mention you have some issue when run data and AAP policy (overlapping)
this flow can help you
11-09-2024 10:39 PM
Hi, sorry but I don't have problem like this (data and AAP policy (overlapping)) I have problem when AAR didn't work on c8200
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide