VPC + SVI problem

[magedis0383]

Hello,

We have the topology in Attachement. and we have problem with SVI and VPC

The configuration:

N5K1:

vpc domain 100
  peer-switch
  role priority 100
  system-priority 1024
  peer-keepalive destination 192.168.21.1
  peer-config-check-bypass
  delay restore 150
  peer-gateway
  auto-recovery
  ip arp synchronize

vlan 801
  name DEV_WAN

interface Vlan801
  description IP DEV
  no shutdown
  no ip redirects

interface Vlan1000
  no shutdown
  no ip redirects
  ip address 192.168.22.5/30

interface port-channel1000
  switchport mode trunk
  spanning-tree port type network
  spanning-tree guard loop
  vpc peer-link

interface port-channel401
  description LACP-SRV1

  switchport mode trunk
  speed 1000
  duplex full
  vpc 401

interface Ethernet1/1
  description "TRUNK VPC"
  no cdp enable
  switchport mode trunk
  spanning-tree port type network
  spanning-tree bpdufilter enable
  channel-group 1000 mode active

interface Ethernet1/2
  description "TRUNK VPC"
  switchport mode trunk
  spanning-tree port type network
  channel-group 1000 mode active

interface Ethernet1/5
  description SRV1_GB2
  switchport mode trunk
  speed 1000
  duplex full
  channel-group 401 mode active

interface Ethernet1/29
  description Uplink N5K3
  switchport mode trunk

N5K2:

vpc domain 100
  peer-switch
  role priority 110
  system-priority 1024
  peer-keepalive destination 192.168.21.2
  peer-config-check-bypass
  delay restore 150
  peer-gateway
  auto-recovery
  ip arp synchronize

vlan 801
  name DEV_WAN

interface Vlan801
  no shutdown
  ip address 202.168.72.1/29

interface Vlan1000
  description VPC-N5K
  no shutdown
  no ip redirects
  ip address 192.168.22.6/30

interface port-channel1000
  switchport mode trunk
  spanning-tree port type network
  spanning-tree guard loop
  vpc peer-link

interface port-channel401
  description LACP-SRV1
  switchport mode trunk
  speed 1000
  duplex full
  vpc 401

interface Ethernet1/1
  description "TRUNK VPC"
  switchport mode trunk
  spanning-tree port type network
  channel-group 1000 mode active

interface Ethernet1/2
  description "TRUNK VPC"
  switchport mode trunk
  spanning-tree port type network

  channel-group 1000 mode active

interface Ethernet1/5
  description SRV1_GB4
  switchport mode trunk
  speed 1000
  duplex full
  channel-group 401 mode active

SRV1 IP: 202.168.72.2/29

When i plug the cable from SRV1 to N5K1 and N5K2 i can't ping SRV1 from ADM

when i unplug the cable from SRV1 to N5K2 i can't ping SRV1 from ADM

when i unplug the cable from SRV1 to N5K1 i CAN ping SRV1 from ADM

between N5K1, N5K2 and N5K3 we have OSPF

Thks !

[Jon Marshall]

Hi Bilal

No, you explained it really well but looking at your first logical diagram.

The issue is N5K1 does not have an IP address for vlan 801 so it won't be advertising an LSA for that to N5K3.

Which as far as I can tell means N5K3 should only see one route to vlan 801 via N5K2.

If N5K1 had an IP on that SVI I can totally understand what you are saying.

Perhaps it's me being a bit thick :-)

Jon

[magedis0383]

Evenif N5K1 has an ip, the IP GW for SRV1 is on N5K2, so it will the same problem no ?

[Jon Marshall]

It would definitely be a problem if N5K1 had an IP yes. See Bilal's latest post for why.

But at the moment I am trying to work out why N5K3 is seeing two equal cost paths if N5K1 is not advertising an LSA.

It may be my misunderstanding but it could be you have a different issue.

Can you post the "sh ip route" output for that network and the output of the type 1 LSAs for that network from N5K3.

Jon

[Bilal Nawaz]

Oh, I see. Yes you're right Jon, that means traffic should always go via N5K2 directly. Suggestion is to put HSRP between N5K1 and N5K2 for vlan 801. Only have N5K2 advertise vlan 801 to N5K3. Try it and lets see.

[Jon Marshall]

Suggestion is to put HSRP between N5K1 and N5K2 for vlan 801. Only have N5K2 advertise vlan 801 to N5K3. Try it and lets see.

If we add an IP to N5K1 vlan 801 then we can't stop the LSA being advertised to N5K3.

We could filter it from going into the RIB on N5K3 I suppose but there seems to be something else happening here at the moment.

I'm not sure what it is but as far as I can tell N5K3 should not be seeing N5K1 as a next hop currently.

The easiest solution I would have thought is just to stop peering with N5K1 ie. change the link to N5K2 from N5K3 to be either a L3 P2P link or use a non vPC vlan.

But that depends on whether there are non vPC vlans on N5K3 that are being used elsewhere within the network.

You wouldn't lose anything here  although it may mean traffic from SRV1 would choose it's link to N5K1 to reach ADM but that should be okay because ADM is on an orphan port so there is no vPC loop in either direction as far as I can tell.

But you know these switches a lot better than me so perhaps the best solution is HSRP ?

Jon

[Bilal Nawaz]

What Im trying to say is we only advertise out what we configure to get advertised out.

On N5K1, lets not put vlan 801 in OSPF, but still have the P2P neighborship between N5K1 and N5K3. N5K2 will do the advertising for us.

Sure there will be some asymmetric traffic patterns, but thats how we can get around it I think. I just want to see what happens :)

[Jon Marshall]

Ahh, okay I understand now, sorry I didn't realise you were talking about not including it under the OSPF configuration.

My only worry is it isn't currently supposed to be advertising anything out and it clearly is which is confusing to say the least :-)

Jon

[magedis0383]

Iam going to try with HSRP

[Jon Marshall]

For the OPs benefit.

Bilal may well be right about using HSRP but you would need to stop an LSA from being received by N5K3 from N5K1.

The suggestion I made could have one potential issue.

If N5K3 is receiving some routes only from N5K1 because N5K2 does not have an interface or isn't advertising the same routes for example, then you don't want to lose that peering even if it is giving you issues at the moment.

It's really difficult to say what will and won't work without seeing some outputs from N5K3.

Jon

[Jon Marshall]

Just to add to this.

You gain nothing by peering with N5K1 over the vPC peer link.

N5K2 will pass on the OSPF routes from N5K3 anyway.

If N5K2 goes down you have lost the peering to N5K1 anyway so it is not doing anything for you.

Jon

[magedis0383]

Yes. 

in few month we will have a direct connection to N5K1 too.

For the moment we want to add redundancy on SRV1 with 2 connection (N5K1 and N5K2) but it seems that is not possible 

[Jon Marshall · Accepted solution]

What I was saying was if you peer N5K3 to N5K2 on a non vPC vlan you should be able to connect SRV1 with a vPC to both N5Ks.

Having N5K3 peer with N5K1 is not giving you redundancy because it is only connected to N5K2.

So it is more an illusion of redundancy.

From my last post you can see I am still trying to understand the exact loop it is seeing but Bilal knows this better than me so I'm hoping he can explain.

If you do connect N5K3 to both don't use a vPC otherwise you will see the very problems Bilal has mentioned.

Jon

[magedis0383]

Evenif n5k3 are connected with n5k1 and n5k2 it Will not Work too?

[Jon Marshall]

With a vPC no it won't.

But there are other ways of connecting. Have a read of this document which explains what would happen and what appears to be happening now although I am still a little unsure about that because vlan 801 on N5K1 does not have an IP on the SVI but that may be me misunderstanding.

Here is the link -

http://bradhedlund.com/2010/12/16/routing-over-nexus-7000-vpc-peer-link-yes-and-no/

Jon

[magedis0383]

Can i do this: