Issue with Nexus9000s - Interface loopback

[ISRAEL QUIRARTE]

Hello Everyone

I have an odd issue with 2 Nexus9000s (DS01 and DS02), these nexus have 2 port-channels:

• Po13 for the peer Iink
• Po5 to the switches cores Nexus7700

I created the vlan 550 with network 10.21.50.0/24, the hsrp is .1, the primary switch core has the .2 (it is the active) and the secondary switch core has the .3 (standby), Then I created one loopback for each switch

DS01 - loop0 - 10.21.6.5/32

DS02 - loop0 - 10.21.6.6/32

I put a static route in each switch core for the loopbacks

In RX01 & RX02

ip route 10.21.6.5/32 10.21.50.4
ip route 10.21.6.6/32 10.21.50.5

I am able to ping the loopback 10.21.6.6 from both switch cores, but i cannot ping the 10.21.6.5, and from both switches core I am able to ping the ips .4 & .5

Checking the spanning-tree in DS01, I see that the Po5 is the root port:

CPNDFW1-DS01# sh spanning-tree vlan 550

VLAN0550
Spanning tree enabled protocol rstp
Root ID Priority 33318
Address 0023.04ee.be01
Cost 1
Port 4100 (port-channel5)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 33318 (priority 32768 sys-id-ext 550)
Address 0023.04ee.be05
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po5 Root FWD 1 128.4100 (vPC) Network P2p
Po13 Desg FWD 1 128.4108 (vPC peer-link) Network P2p

Thanks in advance. Please let me know if any further information is required.

[Reza Sharifi]

Hi,

Can you issue a trace route from RX01 and 02 to 10.21.6.5 and post it?

[ISRAEL QUIRARTE]

It looks like there is a loop

RX01# traceroute 10.21.6.5
traceroute to 10.21.6.5 (10.21.6.5), 30 hops max, 40 byte packets
1 10.21.50.5 (10.21.50.5) 1.143 ms 1.015 ms 1.017 ms
2 10.21.50.3 (10.21.50.3) 1.322 ms 1.407 ms 1.688 ms
3 10.21.50.5 (10.21.50.5) 1.48 ms 0.986 ms 1.173 ms
4 10.21.50.2 (10.21.50.2) 0.967 ms 0.904 ms 0.721 ms
5 10.21.50.5 (10.21.50.5) 0.838 ms 1.315 ms 1.076 ms
6 10.21.50.2 (10.21.50.2) 0.99 ms 10.21.50.3 (10.21.50.3) 1.264 ms 1.344 ms
7 10.21.50.5 (10.21.50.5) 1.045 ms 1.047 ms 1.016 ms
8 10.21.50.3 (10.21.50.3) 1.291 ms 1.33 ms 1.634 ms
9 10.21.50.5 (10.21.50.5) 0.856 ms 1.284 ms 0.785 ms
10 10.21.50.2 (10.21.50.2) 1.237 ms 0.707 ms 10.21.50.3 (10.21.50.3) 1.274 ms
11 10.21.50.5 (10.21.50.5) 0.854 ms 0.974 ms 1.009 ms
12 10.21.50.3 (10.21.50.3) 1.759 ms 1.51 ms 10.21.50.2 (10.21.50.2) 0.823 ms
13 10.21.50.5 (10.21.50.5) 0.904 ms 1.232 ms 1.002 ms
14 10.21.50.2 (10.21.50.2) 1.269 ms 0.822 ms 0.935 ms
15 10.21.50.5 (10.21.50.5) 1.291 ms 1.497 ms 1.302 ms
16 10.21.50.2 (10.21.50.2) 0.903 ms 10.21.50.3 (10.21.50.3) 1.511 ms 10.21.50.2 (10.21.50.2) 0.807 ms
17 10.21.50.5 (10.21.50.5) 1.182 ms 1.23 ms 0.873 ms
18 10.21.50.3 (10.21.50.3) 1.422 ms 10.21.50.2 (10.21.50.2) 0.844 ms 0.854 ms
19 10.21.50.5 (10.21.50.5) 1.275 ms 1.033 ms 0.977 ms
20 10.21.50.2 (10.21.50.2) 1.027 ms 10.21.50.3 (10.21.50.3) 1.392 ms 1.86 ms
21 10.21.50.5 (10.21.50.5) 1.071 ms 0.966 ms 1.255 ms
22 10.21.50.2 (10.21.50.2) 1.458 ms 10.21.50.3 (10.21.50.3) 1.258 ms 1.262 ms
23 10.21.50.5 (10.21.50.5) 1.294 ms 1.059 ms 1.299 ms
24 10.21.50.2 (10.21.50.2) 0.912 ms 10.21.50.3 (10.21.50.3) 1.479 ms 1.221 ms
25 10.21.50.5 (10.21.50.5) 1.019 ms 1.074 ms 1.241 ms
26 10.21.50.2 (10.21.50.2) 0.716 ms 0.779 ms 10.21.50.3 (10.21.50.3) 1.551 ms
27 10.21.50.5 (10.21.50.5) 1.094 ms 1.122 ms 0.98 ms
28 10.21.50.3 (10.21.50.3) 1.441 ms 10.21.50.2 (10.21.50.2) 0.76 ms 0.763 ms
29 10.21.50.5 (10.21.50.5) 1.051 ms 1 ms 1.155 ms
30 10.21.50.3 (10.21.50.3) 1.396 ms 10.21.50.2 (10.21.50.2) 1.499 ms 10.21.50.3 (10.21.50.3) 1.623 ms

RX02# traceroute 10.21.6.5
traceroute to 10.21.6.5 (10.21.6.5), 30 hops max, 40 byte packets
1 10.21.50.5 (10.21.50.5) 1.186 ms 1.057 ms 1.572 ms
2 10.21.50.2 (10.21.50.2) 1.641 ms 10.21.50.3 (10.21.50.3) 0.756 ms 10.21.50.2 (10.21.50.2) 1.306 ms
3 10.21.50.5 (10.21.50.5) 0.857 ms 0.802 ms 0.989 ms
4 10.21.50.2 (10.21.50.2) 1.244 ms 1.319 ms 1.476 ms
5 10.21.50.5 (10.21.50.5) 1.273 ms 0.985 ms 1.206 ms
6 10.21.50.2 (10.21.50.2) 1.566 ms 1.255 ms 10.21.50.3 (10.21.50.3) 0.867 ms
7 10.21.50.5 (10.21.50.5) 1.048 ms 0.879 ms 1.127 ms
8 10.21.50.2 (10.21.50.2) 1.69 ms 1.419 ms 1.869 ms
9 10.21.50.5 (10.21.50.5) 1.002 ms 0.909 ms 1.491 ms
10 10.21.50.2 (10.21.50.2) 1.606 ms 10.21.50.3 (10.21.50.3) 0.835 ms 10.21.50.2 (10.21.50.2) 1.402 ms
11 10.21.50.5 (10.21.50.5) 1.904 ms 1.044 ms 1.112 ms
12 10.21.50.3 (10.21.50.3) 1.22 ms 10.21.50.2 (10.21.50.2) 1.271 ms 1.288 ms
13 10.21.50.5 (10.21.50.5) 1.025 ms 0.875 ms 0.872 ms
14 10.21.50.2 (10.21.50.2) 1.56 ms 1.561 ms 1.328 ms
15 10.21.50.5 (10.21.50.5) 1.198 ms 1.29 ms 1.177 ms
16 10.21.50.3 (10.21.50.3) 1.135 ms 10.21.50.2 (10.21.50.2) 1.524 ms 1.31 ms
17 10.21.50.5 (10.21.50.5) 1.069 ms 1.272 ms 1.026 ms
18 10.21.50.2 (10.21.50.2) 1.446 ms 10.21.50.3 (10.21.50.3) 0.77 ms 10.21.50.2 (10.21.50.2) 1.369 ms
19 10.21.50.5 (10.21.50.5) 1.076 ms 0.949 ms 0.955 ms
20 10.21.50.2 (10.21.50.2) 1.729 ms 1.41 ms 10.21.50.3 (10.21.50.3) 0.904 ms
21 10.21.50.5 (10.21.50.5) 0.884 ms 1.185 ms 1.009 ms
22 10.21.50.2 (10.21.50.2) 1.319 ms 1.478 ms 10.21.50.3 (10.21.50.3) 0.96 ms
23 10.21.50.5 (10.21.50.5) 0.933 ms 1.131 ms 1.126 ms
24 10.21.50.2 (10.21.50.2) 1.575 ms 1.546 ms 1.598 ms
25 10.21.50.5 (10.21.50.5) 1.131 ms 0.97 ms 1.115 ms
26 10.21.50.2 (10.21.50.2) 1.369 ms 10.21.50.3 (10.21.50.3) 0.718 ms 0.91 ms
27 10.21.50.5 (10.21.50.5) 1.328 ms 1.098 ms 1.018 ms
28 10.21.50.3 (10.21.50.3) 0.816 ms 0.931 ms 10.21.50.2 (10.21.50.2) 1.531 ms
29 10.21.50.5 (10.21.50.5) 1.216 ms 1.293 ms 1.561 ms
30 10.21.50.2 (10.21.50.2) 1.585 ms 1.486 ms 1.393 ms

But I cannot find the root cause of this

[Reza Sharifi]

Defiantly you have a loop.

Can you post the actual config from the devices?

[ISRAEL QUIRARTE]

Hello

I think i need to add the command peer-gateway into the vpc, I will share the results

Thanks for your help Reza

[ISRAEL QUIRARTE]

I still have the same issue.

I attached the configurations, but It looks like a design issue.

[Reza Sharifi]

Can you explain the physical connectivity? Is each 9k connected to both 7ks and vice versa?

Are the 7ks vPCed together?  

[ISRAEL QUIRARTE]

Hi Reza

Yes, each 9k is connected to both 7ks and vice versa.

And yes, the 7ks are VPCed together.

[Reza Sharifi]

Hi Israel,

Thanks for the info and the drawing.  Are these in production? Can you remove the cross connect fiber or bring down the ports (one from each switch) and test the Loopback again?

HTH 

[ISRAEL QUIRARTE]

Yes , They are, I am checking some best designs for the vpcs, But I will try that in a maintenance window.

Thanks a lot for the follow up Reza!

Regards

Israel