Latency due to OSPF between 2 Nexus 7k switches in same site?

[pugster73]

Recently, we decided to deploy a polycom infrastructure and we have jitters and latency issues with it.  It took a long time for us to find out the problem but I can try to describe the problem the best I can.  So in our Co-location datacenter, we have an Cisco UCS (where the polycom infrastructure is located which is a bunch of VM's) is connected to both of our 2 -Cisco Nexus 7004 switches via a VPC (I think).  On our main site located about 35 miles away, we have 2 Cisco 4511's connected via VSS mode.  Each of Cisco 4500 switch on our main site is connected to a Cisco N7K via an VPLS link so physically.  So we have 2 VPLS links connected to the 2 sites.

So from our main site (on the 4511) I do a ping to one of the servers on the Colo via the ethernet port connected VPLS link A I get an about 3ms latency which is fine.  However, when I do a ping via the ethernet port connected to link B, I get a latency of 72ms!!!  On other servers connected to Colo, I notice that I get a latency ping of 72ms via ping from link A, but get a latency of 3ms on link B.  

Anybody know what could've caused this issue?  Thanks.

[Philip D'Ath]

One of the links is saturated?

[pugster73]

No, none of the links are saturated.  I did more troubleshooting and I am starting to find out what the source of the problem is.  We have another site located in london which is about a 70ms ping from here.  I noticed that when I did a trace to the ip address of the secondary vpls link it bounces thru London and the trace comes back with 140ms latency, whereas when I pinged thru the primary link I got 8ms latency.  

I do have a question though.  On our main site with the 4500 on vss on both switches, and when I run show ip ospf nei, I noticed that it has ospf information for the ethernet ports running on each switch.  On our co-lo site, we run vps on each switch, so on each switch when I run the same command, I only see ospf information running on the ethernet port on each switch, but NOT on the trunk port between switches.  Do I have to specify it on the vps trunk?  Thanks.

[pugster73]

Okay, I think I know what the problem is.  On the 2 switches on the Nexus 7k, I noticed when I ran show ip route, it is apparently learned to route packets from here to London which is causing the 70ms delay.  This is what I get when I ran

Nexus 7k Switch A: ( this one is connected via 172.20.1.x network)

172.20.1.254/32, ubest/mbest: 1/0, attached
*via 172.20.1.254, Eth3/43, [0/0], 2y20w, local
172.20.2.0/24, ubest/mbest: 1/0
*via 172.20.1.30, Eth3/43, [110/45], 3w2d, ospf-1, intra

Nexus 7k switch B: (this one is connected via 172.20.2.x network)

172.20.1.0/24, ubest/mbest: 2/0
*via 172.20.2.10, Eth3/43, [110/20], 5d18h, ospf-1, intra
*via 172.20.2.30, Eth3/43, [110/20], 5d18h, ospf-1, intra
172.20.2.0/24, ubest/mbest: 1/0, attached
*via 172.20.2.254, Eth3/43, [0/0], 2y34w, direct

The ip addresses of the london connection links are 172.20.1.30 and 172.20.2.30.  I was wondering how did it learn the other network and how did it route to the wrong ip address.  Also, I also noticed that 172.20.2.254 is direct connected whereas 102.20.1.254 is a local.  Why does that happen?  Thanks.