ACI TEP Pool Sizing Requirements

[dm2020]

I'm currently working on a new multi-pod design and I'm confused about the infra TEP pool sizing requirements.

The ACI design guide recommends a /16 or /17 to ensure that there are no scaling issues with a high-level summary of factors that need to be considered, however I cant find any further details of exactly how the TEP pool is utilized so that I can plan scale in my environment. For example, how many IP addresses are used when implementing vPCs? Does ACI provide any reports (GUI or CLI) of TEP pool utilization (percent used/percent available) following implementation?

I'm currently designing a 4-pod solution that will have a maximum of 100 leaf switches per pod. The simplest option is to allocate a /16 per pod and have done, but justifying the address space is going to be challenging without details of how the addresses are going to be used. Is a /16 required for pods of this size, or would a /17, /18 or even /19 be more than adequate for what I need?

[AlexI1978]

We used in the past /20 with 16 Leaf’s and 4 Spines for each POD. We had 3 POD's and no issues. 

This is a great questions and would love to hear others provide there feedback. 

[RedNectar]

Hi @dm2020 ,

Although this does not answer your question, you may find the logic behind why (my belief is that) Cisco recommends that the VTEP IP address range does not overlap with existing address allocations.  Once you understand that, you may be willing to take the risk of using IP addresses that are allocated elsewhere.

https://community.cisco.com/t5/application-centric-infrastructure/overlapping-or-non-overlapping-vtep-pool/m-p/3038671/highlight/true#M2783

 Another answer to the problem of address allocation is to remember that 100.64.0.0/10 and 198.18.0.0/15 are private address ranges that may be suitable to use in your environment. See RFCs 2544 and 6598 respectively, or check out https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml

[Remi-Astruc]

Hello,

The number of Leaves and Spines is not the only criteria. 

The consumption of TEP Pool IP range in a Pod is depending on the following:

• The number of Leaves and Spines
• The number of APIC Nodes
• The number of vPC channels
• The number of Opflex infra VMM Nodes (AVE, K8S, ...)

The /16 size recommendation supports the high scale of these criteria. /21 works well with a small/Lab Pod without Opflex VMM (at your own risk of scalability).

And there's everything in between...

[AshSe]

• The number of vPC channels
• The number of Opflex infra VMM Nodes (AVE, K8S, ...)

@Remi-AstrucCould you please provide a supporting link for your statement? From my understanding, TEP (Tunnel Endpoint) IP addresses are dynamically assigned by DHCP in the APIC to both spine and leaf switches. Leaf switches, referred to as Physical TEPs, and spine switches, known as Proxy TEPs, utilize these addresses for VXLAN purposes. I am not aware of any other uses for TEP addresses. I apologize for my lack of knowledge and appreciate any links you can share in advance. Thank you!