Cisco ACI-CNI Integration

[mdebellis1006]

Hi all - 

We have been working to integrate Rancher Kubernetes into our ACI environment utilizing the Cisco ACI-CNI. Having issues with understanding our the External Routers (Cat 8kV) is suppose to be configured for the exposed service IP address. Can anyone provide any insight? What is the best way to troubleshoot the that PBR device is actually getting the routes for translate the External IP space into a POD/Cluster IP? 

Appreciate the help!

[Ali Aghababaei]

Hi @mdebellis1006 
Let's begin with the basics, if this not helpful then we will dive deeper:

To configure external routers (Catalyst 8000V) for Kubernetes exposed service IP addresses within a Cisco ACI environment using the ACI-CNI, follow these steps:

1. **ACI Configuration**:
- Ensure that your ACI fabric is set up correctly with the necessary tenants, VRFs, and L3Out configurations.
- Configure an L3Out for external connectivity and ensure that it is associated with the correct VRF.

2. **External Router Configuration**:
- On the Catalyst 8000V router, configure the necessary static routes or dynamic routing protocols (e.g., BGP) to advertise the Kubernetes service IP ranges.
- Ensure that the router can reach the ACI L3Out subnet.

3. **Service IP Configuration**:
- Use `acc-provision` to configure the ACI CNI for Kubernetes, ensuring that the service IP subnet is correctly specified in the configuration file.
- Verify that the service IPs are correctly advertised and reachable from the external network.

4. **Troubleshooting**:
- Ensure that the PBR (Policy-Based Routing) device is receiving the correct routes by checking the route tables and verifying route advertisements.
- Use tools like `ping`, `traceroute`, and `kubectl get services` to verify connectivity from external networks to the Kubernetes services.