cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1117
Views
0
Helpful
1
Replies

Cisco ACI fTriage in Multi-Site

jan.janovic.sk
Beginner
Beginner

Hello colleagues,

is there somebody please with an experience with fTriage (end-to-end ELAM automation tool) in ACI Multi-Site environment? According to the available (although limited) information and examples in the command help, multi-site should be supported but I haven't got it working yet. 

 

From ftriage example help section:

# MSITE
> ftriage route -ii 2::msite1-leaf1:Eth1/12 -ie 651 -ei 3::msite3-leaf1:Eth1/12 -ee 655 -sip 56.1.0.11 -dip 56.1.4.31

 

I've tried this approach and ftriage identified correctly path from the leaf to spine switch, it has seen even packet exiting to the ISN network through the correct interface and with the correct External Routable TEP address as a next-hop, but then it stops and cannot continue triaging in the other site. 

I've also tried manually continuing ftriage in site 2, and although I was able to catch the incoming packet from ISN correctly, ftriage didn't take into account the VNID and sClass translation tables on Spines so as a result there was COOP table miss for this communication and the analysis stopped on Spines:

 

aci-apic21# ftriage -user ***** route  -ii 2::Spine:External -ei LEAF:aci-leaf211 -sip 10.177.16.16 -dip 10.177.0.207 
fTriage Status: {"dbgFtriage": {"attributes": {"operState": "InProgress", "pid": "30816", "apicId": "1", "id": "0"}}}
Starting ftriage
Log file name for the current run is: ftlog_2022-03-03-15-54-25-312.txt
2022-03-03 15:54:25,321 INFO     /controller/bin/ftriage -user **** route -ii 2::Spine:External -ei LEAF:aci-leaf211 -sip 10.177.16.16 -dip 10.177.0.207
Request password info for username: *******
Password:
2022-03-03 15:55:02,429 INFO     ftriage:     main:1185 Invoking ftriage with username: *****
2022-03-03 15:56:16,433 INFO     ftriage:     main:721  Capturing L3 packet Fex: False on node: aci-spine291 IF: Eth1/31
2022-03-03 15:56:41,964 INFO     ftriage:     main:721  Capturing L3 packet Fex: False on node: aci-spine291 IF: Eth1/32
2022-03-03 15:57:06,936 INFO     ftriage:     main:721  Capturing L3 packet Fex: False on node: aci-spine292 IF: Eth1/32
2022-03-03 15:57:32,123 INFO     ftriage:     main:721  Capturing L3 packet Fex: False on node: aci-spine292 IF: Eth1/31
2022-03-03 15:57:50,305 INFO     ftriage:     main:844  L3 packet Seen on aci-spine292 Ingress: Eth1/31 Egress: Eth1/11 Vnid: 3014656
2022-03-03 15:57:50,306 INFO     ftriage:   pktrec:490  aci-spine292: Collecting transient losses snapshot for LC module: 1
2022-03-03 15:58:10,520 ERROR    ftriage:      fib:323  aci-spine292: EP not found in COOP! for VRF VNID: 3014656
2022-03-03 14:58:10,520 ERROR    ftriage:      fib:323  aci-spine292: EP not found in COOP! for VRF VNID: 3014656
2022-03-03 15:58:11,194 WARNING  ftriage:      fib:563  aci-spine292: mac-da-key is not valid for bridged packet!
2022-03-03 14:58:11,195 WARNING  ftriage:      fib:563  aci-spine292: mac-da-key is not valid for bridged packet!
2022-03-03 15:58:13,479 INFO     ftriage:     main:938  SIP 10.177.16.16 DIP 10.177.0.207
2022-03-03 15:58:13,481 ERROR    ftriage:  unicast:1248 aci-spine292: We should aither be proxy or transit or local but none set
2022-03-03 14:58:13,482 ERROR    ftriage:  unicast:1248 aci-spine292: We should aither be proxy or transit or local but none set
2022-03-03 15:58:13,482 INFO     ftriage:  unicast:1252 aci-spine292: Enter dbg_sub_nexthop with Noneinst: ig infra: False glbs.dipo: 172.18.4.1
2022-03-03 15:58:13,601 ERROR    ftriage:  unicast:1373 aci-spine292: EP is unknown in COOP. Ftriage will exit but continue with further fault isolation
2022-03-03 14:58:13,601 ERROR    ftriage:  unicast:1373 aci-spine292: EP is unknown in COOP. Ftriage will exit but continue with further fault isolation
2022-03-03 15:58:13,602 INFO     ftriage:  unicast:1377 aci-spine292: Checking EP on eg nodes [aci-leaf211]
2022-03-03 15:58:13,922 ERROR    ftriage:  unicast:1407 aci-spine292: EP is unknown on all the eg nodes
2022-03-03 14:58:13,922 ERROR    ftriage:  unicast:1407 aci-spine292: EP is unknown on all the eg nodes
2022-03-03 15:58:15,653 INFO     ftriage:  unicast:1409 : Ftriage Completed with hunch: EPM has failed to learn the EP on egress nodes
fTriage Status: {"dbgFtriage": {"attributes": {"operState": "Idle", "pid": "0", "apicId": "0", "id": "0"}}}
aci-apic21#

Thanks a lot for any help or comment, suggestion.

Jan  

1 Reply 1

yohasan
Cisco Employee
Cisco Employee

Hello @jan.janovic.sk 

The ftriage logs refer to missing EP details in COOP database, it seems the COOP does not have the EXPORT flag on EP and this could results in COOP inconsistency between sites.

I'd suggest to check the following steps:
1> Check if the spines have EP "10.177.16.16" entry programmed to handle proxy forwarding "please note flood is required for un-known unicast "
2> For the leaf that is performing policy enforcement, check if you have the appropriate contract
3> Check if BGP have the VNIDs registered correctly

Hope this helps and if there are things which are still not clear, let us know.

Regards,
Josef

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Recognize Your Peers