01-17-2018 05:58 PM - edited 03-01-2019 06:06 AM
The unmanaged mode is also known as Network only switching, which is introduced in Brazos release. It adds the flexibility for customer to use only network automation for service appliance.
If a device is configured as “un-managed” i.e. managed setting is set to false, APIC does not program the device. APIC only allocates the network resources and programs the VLAN/VXLAN on fabric side. The configuration of the L4-L7 device is left to be done by customer.
This article will give a configuration example of the unmanaged mode using ASAv in routed mode, how to verify and troubleshoot scenarios will also be given as reference.
Following settings are NOT needed when a device cluster is configured un-managed
Please note that APIC still needs to know the topology information (LIF, CIF) for the device cluster and devices. This is needed so that APIC can program the appropriate ports on leaf and also can use this information for troubleshooting wizard purposes.
Integration of L4-7 services in ACI requires specific bridge domain networking configuration. The requirements can be summarized as:
ASAv version: 9.4(2)
APIC version: 2.3(1f)
VMware integration
Centos7 VM as a web client
Centos7 VM as a web server
Network Diagram
logical relationship diagram as below.
Basic configuration on the ASAv.
ASA Version 9.4(2) ! hostname ASAv-1 interface GigabitEthernet0/0 nameif inside security-level 100 ip address 10.10.10.10 255.255.255.0 ! interface GigabitEthernet0/1 nameif outside security-level 0 ip address 20.20.20.20 255.255.255.0 ! access-list out extended permit icmp any any access-list out extended permit http any any access-group out in interface outside
L4-L7 Configurations on ACI side
Before configuring the L4-L7 integration, make sure the Access Policies/VMM Domain configured and interfaces up and in service, the basic configuration of the two EPGs, BDs etc will not be included in this article. From the output below, the web client and web server are learned locally, also we can see they belongs to different BDs.
Leaf101# show system internal epm endpoint mac 0050.5691.5bf0 MAC : 0050.5691.5bf0 ::: Num IPs : 1 Vlan id : 22 ::: Vlan vnid : 11193 ::: VRF name : lindawa:prod BD vnid : 16285616 ::: VRF vnid : 2490374 Phy If : 0x16000001 ::: Tunnel If : 0 Interface : port-channel2 Flags : 0x80005c05 ::: sclass : 49158 ::: Ref count : 5 EP Create Timestamp : 11/01/2017 10:52:31.488217 EP Update Timestamp : 11/01/2017 21:06:31.088306 EP Flags : local|vPC|MAC|host-tracked|sclass|timer| :::: Leaf101# show system internal epm endpoint mac 0050.5691.6bb9 MAC : 0050.5691.6bb9 ::: Num IPs : 1 Vlan id : 29 ::: Vlan vnid : 11210 ::: VRF name : lindawa:prod BD vnid : 16089028 ::: VRF vnid : 2490374 Phy If : 0x16000001 ::: Tunnel If : 0 Interface : port-channel2 Flags : 0x80004c05 ::: sclass : 49156 ::: Ref count : 5 EP Create Timestamp : 11/01/2017 15:53:33.982626 EP Update Timestamp : 11/01/2017 21:06:31.086101 EP Flags : local|vPC|MAC|sclass|timer| ::::
Important Notice: There is no need to create EPGs explicitly for the firewall internal and external interfaces, instead of that, shadow EPGs will be created by APIC automatically, we will go there later in this article.
Step1: Creating l4-l7 device
Step2: Creating a service graph
Step3. Apply service graph.
Verification Part
Network Label configuration is also done automatically to ASAv.
Then in order to verify the connectivity, you can run the ping test from the web client to server and vice versa.
On the firewall side, you could be able to see the hit count for the acl is actually increasing.
ASAv-1# show access-list access-list cached ACL log flows: total 0, denied 0 (deny-flow-max 4096) alert-interval 300 access-list out; 2 elements; name hash: 0x5589cfea access-list out line 1 extended permit icmp any any (hitcnt=4) 0x4f3e126c access-list out line 2 extended permit http any any (hitcnt=0) 0xb4296acc ASAv-1#
The logical topology looks like below.
The contract is applied between the provider and consumer, however the contracts which are applied between the shadow EPGs will not be shown on this graph view, we can check the zoning-rules on leaf.
Verify if the shadow EPGs get deployed from the leaf101
Dynamic Endpoints:
Tenant : lindawa
L4L7 Device : [uni/tn-lindawa/lDevVip-lindawa-ASAv]-ctx-prod
L4L7 Graph : lindawa-ASAv-2ctxprod-N-BD_Web-C-consumer
End Point MAC IP Address Node Interface Encap Multicast Address
----------------- ---------------------------------------- ---------- ------------------------------ --------------- ---------------
00:50:56:91:33:C1 --- 101 102 vpc lindawa-vPC-SVR37 vlan-3041 not-applicable
Tenant : lindawa
L4L7 Device : [uni/tn-lindawa/lDevVip-lindawa-ASAv]-ctx-prod
L4L7 Graph : lindawa-ASAv-2ctxprod-N-prod-C-provider
End Point MAC IP Address Node Interface Encap Multicast Address
----------------- ---------------------------------------- ---------- ------------------------------ --------------- ---------------
00:50:56:91:A0:5A --- 101 102 vpc lindawa-vPC-SVR37 vlan-3022 not-applicable
Tenant : lindawa
Application : lindawa-svcs
AEPg : lindawa_svr
End Point MAC IP Address Node Interface Encap Multicast Address
----------------- ---------------------------------------- ---------- ------------------------------ --------------- ---------------
00:50:56:91:5B:F0 --- 101 102 vpc lindawa-vPC-SVR37 vlan-3001 not-applicable
Tenant : lindawa
Application : lindawa-web
AEPg : web
End Point MAC IP Address Node Interface Encap Multicast Address
----------------- ---------------------------------------- ---------- ------------------------------ --------------- ---------------
00:50:56:91:31:A5 --- 101 102 vpc lindawa-vPC-SVR37 vlan-3018 not-applicable
00:50:56:91:6B:B9 --- 101 102 vpc lindawa-vPC-SVR37 vlan-3018 not-applicable
Total Dynamic Endpoints: 6
Total Static Endpoints: 0
apic1#
Leaf101# show vlan ex | grep -A 2 -B 2 ASA 18 lindawa:prod active Eth1/33, Po2 22 lindawa:lindawa-svcs:lindawa_svr active Eth1/33, Po2 27 lindawa:lindawa_ASAvctxprod:provider active Eth1/33, Po2 29 lindawa:lindawa-web:web active Eth1/33, Po2 30 lindawa:lindawa_ASAvctxprod:consumer active Eth1/33, Po2 18 enet CE vxlan-16285616 22 enet CE vlan-3001 27 enet CE vlan-3022 29 enet CE vlan-3018 30 enet CE vlan-3041 Leaf101# show system internal epm vlan 27 detail // Shadow EPG for firewall provider connector VLAN 27 VLAN type : FD vlan hw id : 22 ::: sclass : 32776 access enc : (802.1Q, 3022) fabric enc : (VXLAN, 11214) Object store EP db version : 4 BD vlan id : 18 ::: BD vnid : 16285616 ::: VRF vnid : 2490374 Valid : Yes ::: Incomplete : No ::: Learn Enable : Yes pol_ctrl_flags: Endpoint count : 1 ::: Local Endpoint count : 1 :::: Leaf101# show system internal epm vlan 30 detail // Shadow EPG for firewall consumer connector VLAN 30 VLAN type : FD vlan hw id : 23 ::: sclass : 32777 access enc : (802.1Q, 3041) fabric enc : (VXLAN, 11233) Object store EP db version : 4 BD vlan id : 28 ::: BD vnid : 16089028 ::: VRF vnid : 2490374 Valid : Yes ::: Incomplete : No ::: Learn Enable : Yes pol_ctrl_flags: Endpoint count : 1 ::: Local Endpoint count : 1 :::: ::::
As mentioned in the beginning, each unique interface/direction must be mapped to a unique Bridge Domain.
Which means, the two EPGs should belongs to different BDs.
If the web client and web server are in same BD, then only the shadow EPG for the ASAv(inside interface) would be created, if you check the connection connectors, you would see that APIC allocates same the encap vlan and sclass for both consumer and provider connectors, which means only one shadow EPG will be created here.
Now, if we would like to apply the same service graph template to a different pair of EPGs, what would happen?
Short answer is, the service graph instance will not be deployed successfully.
Fault F1690 is raised with description: Configuration is invalid due to Encap is already allocated for a virtual cluster interface.
This is because the consumer/provider connectors on the firewall has already been taken.
Here is the correct procedure in order to do so.
1. You need to modify the ASAv VM, an extra pair of network adapters need to be created.
2.Then go to the L4-L7 devices, create two more concreted interfaces and cluster interfaces for the ASAv device.
Select the device, then right-click on it, choose the "Create Concrete Interface" option, add two more interfaces here: inside2, outside2.
Create two cluster interfaces as well, don't forget this. You need to use the concreate interfaces created just now.
3. While applying the graph template, do NOT forget to manually change the consumer/provider connectors to the corresponding cluster interfaces, see as below. Otherwise, you are going to see the same fault as before.
In order to understand how the contract works for this use case, let's take a look at the zoning-rules and stats in policy-mgr on leaf switch.
Before applying the service graph.
Leaf101# show zoning-rule scope 2490374 Rule ID SrcEPG DstEPG FilterID operSt Scope Action Priority ======= ====== ====== ======== ====== ===== ====== ======== 4152 0 16386 implicit enabled 2490374 permit any_dest_any(15) 4153 0 0 implicit enabled 2490374 deny,log any_any_any(20) 4154 0 0 implarp enabled 2490374 permit any_any_filter(16) 4155 0 15 implicit enabled 2490374 deny,log any_vrf_any_deny(21) 4104 0 16388 implicit enabled 2490374 permit
Leaf101#
Leaf101# show system internal policy-mgr stats | grep 2490374 Rule (4104) DN (sys/actrl/scope-2490374/rule-2490374-s-any-d-16388-f-implicit) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4152) DN (sys/actrl/scope-2490374/rule-2490374-s-any-d-16386-f-implicit) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4153) DN (sys/actrl/scope-2490374/rule-2490374-s-any-d-any-f-implicit) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4154) DN (sys/actrl/scope-2490374/rule-2490374-s-any-d-any-f-implarp) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4155) DN (sys/actrl/scope-2490374/rule-2490374-s-any-d-15-f-implicit) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0
Leaf101#
After applying the service graph.
so here we can see how does the contract get applied between the EPGs.
The contract is applied between the web client(sclass: 49156) and the shadow EPG(sclass: 32777) of ASAv Outside interface.
The contract is applied between the web client(sclass: 49158) and the shadow EPG(sclass: 32776) of ASAv Inside interface.
Leaf101# show zoning-rule scope 2490374 Rule ID SrcEPG DstEPG FilterID operSt Scope Action Priority ======= ====== ====== ======== ====== ===== ====== ======== 4152 0 16386 implicit enabled 2490374 permit any_dest_any(15) 4153 0 0 implicit enabled 2490374 deny,log any_any_any(20) 4154 0 0 implarp enabled 2490374 permit any_any_filter(16) 4155 0 15 implicit enabled 2490374 deny,log any_vrf_any_deny(21) 4104 0 16388 implicit enabled 2490374 permit any_dest_any(15) 4158 49156 32777 default enabled 2490374 permit fully_qual(6) 4159 32777 49156 default enabled 2490374 permit fully_qual(6) 4160 32776 49158 default enabled 2490374 permit src_dst_any(8) 4161 49158 32776 default enabled 2490374 permit src_dst_any(8) Leaf101#
leaf101# show zoning-filter filter default
FilterId Name EtherT ArpOpc Prot MatchOnlyFrag Stateful SFromPort SToPort DFromPort DToPort Prio Icmpv4T Icmpv6T TcpRules
======== =========== ====== ========= ======= ====== ======= ======= ==== ==== ==== ========= ======= ======== ========
default any unspecified unspecified unspecified no no unspecified unspecified unspecified unspecified def unspecified unspecified
The policy-mgr stats also show the same changes.
Leaf101# show system internal policy-mgr stats | grep 2490374 Rule (4104) DN (sys/actrl/scope-2490374/rule-2490374-s-any-d-16388-f-implicit) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4152) DN (sys/actrl/scope-2490374/rule-2490374-s-any-d-16386-f-implicit) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4153) DN (sys/actrl/scope-2490374/rule-2490374-s-any-d-any-f-implicit) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4154) DN (sys/actrl/scope-2490374/rule-2490374-s-any-d-any-f-implarp) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4155) DN (sys/actrl/scope-2490374/rule-2490374-s-any-d-15-f-implicit) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4158) DN (sys/actrl/scope-2490374/rule-2490374-s-49156-d-32777-f-default) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4159) DN (sys/actrl/scope-2490374/rule-2490374-s-32777-d-49156-f-default) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4160) DN (sys/actrl/scope-2490374/rule-2490374-s-32776-d-49158-f-default) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4161) DN (sys/actrl/scope-2490374/rule-2490374-s-49158-d-32776-f-default) Ingress: 0, Egress: 0, Pkts: 0 RevPkts: 0 Leaf101#
Reference links:
Cisco APIC Layer 4 to Layer 7 Services Deployment Guide
Service Graph Design with Cisco Application Centric Infrastructure White Paper
Cisco Application Centric Infrastructure Best Practices Guide
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: