Yes, a few things are happening here. How vpc works with vxlan, the switches in the pair ignore routes from the other switch in the pair (since they should be shared over cfs). However, since this is a type 5 route it shouldn't be ignored, but it looks like the mac address associated with it is being ignored. The packets never get encapsulated in vxlan because it sees the mac address over the peer link.
The peer-gateway feature makes it so the local switch can route packets for the other one, which makes it loop internally on sw1.
The routing table clearly shows it's destined to the vxlan tunnel but it's not encapsulating it, or it is internally and then picking it back up on the same switch, can't tell there.
To me it seems like this is a bug and it's not being programmed right in the hardware. It works from other VTEPS to get to orphan type 5 route on sw1 (which is what the advertise-pip fixed) but the pip should also fix the between vpc pair routes so you don't have to have a per-vrf ibgp session sharing the routes and it doesn't look like that is working.
The type-5 routes have the router mac inserted in them which matches the mac of the vpc peer and i think that is where it isn't getting programmed right.
Have to do some more digging. It seems silly that the other vpc pair can't access an orphan connected host that is on another interface.
example: putting layer3 port and connecting host on sw1, with /29 on layer3 port , advertising this /29 to type5
interface eth1/5
description server
vrf member VNI1000
ip address 6.6.6.1/29
then in bgp, network 6.6.6.0/29
this works for all other vteps on the network to connect to this host EXCEPT the VPC peer switch :/
obviously it would work in the old scenario where you have the per-vrf ibgp sessions between the vpc peers but that's really bad design option.
Also, orphan connecting hosts to vpc pairs isn't best practice either, but sometimes we have to do it , and this should work.
