We recently migrated a pair of 4451 CUBE routers into our ACI fabric. Once we migrated layer 3 into the fabric, we started seeing issues where the CUBE VIP would go unresponsive for a minute or two - which takes the SIP trunk offline and drops all active calls. When the VIP was unresponsive, we didn't see any logs on the router stating that an interface was dropping.
Looking in the APIC, we saw that the VIP was attaching and detaching, constantly.
Is there a best practice configuration on the ACI side to handle a pair of routers with a VIP for HA?
Since you have migrated Cube HA into Cisco ACI and if I assume you have connected Cube just as an external device, two things will happen here
The Router will respond to ARP using the vMAC and VIP.
Cube Routers sources the traffic from VIP IPbut instead of vMAC it will use Physical MAC address.
Which means you have two different MAC addresses are associated with same IP. This actually leads to issue in case with Cisco ACI since it expects the traffic to be sourced from the same VIP and vMAC because ACI can learn IP from the data plane, so it will see the same IP (VIP) associated with two MAC addresses (vMAC and Physical MAC) which will cause flapping between two MAC addresses.
There is a Bug CSCvj66014 for CUBE (CUBE-HA should Use the Same VMAC AND VIP to source SIP Packets), but it is not a CUBE feature related issue issue, it’s more on underlying platform for HA design.
The best possible solution from ACI Deployment perspective :
- Connect CUBE router via L3out to avoid EP learning Challenge
Also, In ACI Deployment with release 4.0(1h) or later IP data plane learning for the VRF can be disabled and can be an option, however the changes will have an impact that needs to be considered before changing the settings. This option is located at Tenant > Networking > VRFs
With ACI release 5.2(1g) ACI allows IP data plane learning to be disabled at the EPG and BD Level as well.