Yes you can have mixed models in a cluster. Do it one APIC at a time. The cluster scale will remain at the lowest member so you won’t get the higher scale until all three are replaced.  ... View more

Yes you can vMotion between pods in a Multi-Pod fabric.  ... View more

There are some features that are not supported but not many and as releases go by it is getting closer.    However I would not recommend using FEX in a new install. You will have very limited  TCAM available if you try to push multiple top of rack switches into a single parent leaf switch. You will get much better scale out of the network by using Nexus 9K leaf switches and not FEX.  ... View more

I have tested this now. My ingress leaves were second generation and the egress (border) leaves were first generation. My ingress leaf was a Remote Leaf, which may also change things.   The behaviour I saw was that the endpoint on the ingress leaf was learnt at the border when the traffic was to an L3 Out destination.   Here is the ingress leaf route: Rleaf202# show ip route vrf common:Lab 10.0.0.1/32, ubest/mbest: 2/0 *via 10.220.240.32%overlay-1, [200/41], 01w03d, bgp-65000, internal, tag 65000 *via 10.220.240.35%overlay-1, [200/41], 01w03d, bgp-65000, internal, tag 65000   These are the TEP IPs of the two border leaves:   Leaf101# show ip int brief vrf overlay-1 | egrep "10.220.240.3[2|5]" lo0 10.220.240.32/32 protocol-up/link-up/admin-up Leaf102# show ip int brief vrf overlay-1 | egrep "10.220.240.3[2|5]" lo0 10.220.240.35/32 protocol-up/link-up/admin-up   I pinged from a switch that was acting as a client in the fabric:   Client_SW# ping 10.0.0.1 source 10.33.33.33 PING 10.0.0.1 (10.0.0.1) from 10.33.33.33: 56 data bytes 64 bytes from 10.0.0.1: icmp_seq=0 ttl=251 time=0.877 ms 64 bytes from 10.0.0.1: icmp_seq=1 ttl=251 time=0.628 ms 64 bytes from 10.0.0.1: icmp_seq=2 ttl=251 time=0.641 ms 64 bytes from 10.0.0.1: icmp_seq=3 ttl=251 time=1.793 ms 64 bytes from 10.0.0.1: icmp_seq=4 ttl=251 time=0.605 ms --- 10.0.0.1 ping statistics --- 5 packets transmitted, 5 packets received, 0.00% packet loss round-trip min/avg/max = 0.605/0.908/1.793 ms IP address: 10.220.240.35, IP subnet: 10.220.240.35/32   Now we check the endpoint learning:   Rleaf202# show endpoint ip 10.33.33.33 Legend: s - arp H - vtep V - vpc-attached p - peer-aged R - peer-attached-rl B - bounce S - static M - span D - bounce-to-proxy O - peer-attached a - local-aged m - svc-mgr L - local E - shared-service +-----------------------------------+---------------+-----------------+--------------+-------------+ VLAN/ Encap MAC Address MAC Info/ Interface Domain VLAN IP Address IP Info +-----------------------------------+---------------+-----------------+--------------+-------------+ 4 vlan-30 00b7.6666.7777 L eth1/52 common:Lab vlan-30 10.33.33.33 L eth1/52   The leaf attached to the client machine has learned the IP.   One of the border leaves has learned the IP.   Leaf101# show endpoint ip 10.33.33.33 Legend: s - arp H - vtep V - vpc-attached p - peer-aged R - peer-attached-rl B - bounce S - static M - span D - bounce-to-proxy O - peer-attached a - local-aged m - svc-mgr L - local E - shared-service +-----------------------------------+---------------+-----------------+--------------+-------------+ VLAN/ Encap MAC Address MAC Info/ Interface Domain VLAN IP Address IP Info +-----------------------------------+---------------+-----------------+--------------+-------------+ common:Lab 10.33.33.33 tunnel18   So with generation 2 ingress and generation 1 egress the L3Out traffic does trigger a learn.   ... View more

So after reading the whitepaper I think you're correct. Traffic from endpoints on the ingress leaf does not trigger and endpoint learn at the border leaf (egress) when the traffic is to an L3 Out and the enforcement is set to ingress. Often the border leaves are also compute leaves in smaller fabrics, so the border leaves will have endpoint learns present due to endpoint to endpoint traffic flows. ... View more

This might be an optimization to stop L3 Out data flows from keeping endpoints alive at the borders. I'd have to test it to see the behaviour with second gen gear. Where is the reference you've included documented? ... View more

The border still learns remote IP endpoints from the ingress leaves (edit: when the traffic is to an endpoint attached to the border and not to the L3 Out).   The prefix to EPG mapping is programmed on the ingress leaves so policy enforcement can happen as soon as the packet enters the fabric. With the mode set to egress the prefixes are only programmed on the border leaves and enforcement happens at the border.    There is also an option to stop IP endpoint learns on the borders altogether (Diasable Remote EP Learn) but this is a separate setting.  ... View more

Hi Andreas. Glad it helped.    We also had to upgrade the server firmware as there were bugs in the Broadcom NIC behaviour as well. Try updating the server firmware. ... View more

Are they in the same EPG? If not do you have a contract between them?   Is the BD set for ARP flooding? Is ARP resolving? Could be an issue with multicast in the IPN.  ... View more

Here is some documentation for it. https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/2-x/L3_config/b_Cisco_APIC_Layer_3_Configuration_Guide/b_Cisco_APIC_Layer_3_Configuration_Guide_chapter_010010.html ... View more

You need to create a Layer 3 Out to advertise BD subnets out of the fabric. This could be via GOLF or regular L3 Out. For a regular L3 Out you’ll need physical connections from your leaves to the IPN or intermediate devices.  ... View more

Slight correction on your slide, the option that removes bi directional stats and reduces TCAM like the vzAny config is referred to as Optimize Contract Performance and you configure it in the Subject by checking the “no-stats” box. This option is a bit misleading in the GUI.    Policy Compression is on FX and later and you set it on the filter. ... View more

Hi RedNectar, I usually don't recommend the vzAny unidirectional contract workaround unless absolutely necessary. It's simpler just to use the option 1 solution. If you have generation 1 leaves then yes, option 3 might need to be used.   With EX and later leaves on 3.2+ you can forgo directional statistics and achieve the same kind of TCAM saving as you get from the vzAny trick. See  Optimize Contract Performance at this link. FX take it a step further with policy indirection which is also documented in the link above under the heading Policy Compression. This requires 4.0+ and is a relatively new feature so I would recommend testing it carefully.   ... View more

It would but it would also waste TCAM because you would still have the entry programmed with destination port 80.  ... View more