Nexus 7000 OTV site vlan down

[BalajiDasarathan]

Hello all

I am setting up a lab to emulate OTV network with 2 Nexus 7010, each with a M2 and F2e module. I created three VDCs in both the Nexus 7010, one for SVI, one for OTV and one for WAN connectivity and am able to ping from one OTV to other (Nexus A to Nexus B). However, even though the overlay interface is up, the site vlan continues to show up as "down".

Would someone be able to help me with the troubleshooting on this?

Thanks for your time and appreciate your help.

- Balaji

[Bilal Nawaz]

Hello Balaji,

What is your OTV vdc configuration? is your site VLAN being trunked from the LAN vdc to the OTV VDC's on both N7Ks?

Bilal

[BalajiDasarathan]

Hi Bilal

The site VLAN is NOT trunked from the LAN VDC to the OTV VDC. In fact, the LAN VDC doesnt have the site VLAN present in its VLAN table.

My understanding of the site vlan is that it is required only in the OTV VDC and that it should not be trunked or be part of the extended VLAN.

I can redesign the lab if required.

-Balaji

[Bilal Nawaz]

Hello Balaji,

I think maybe misunderstanding here. OTV site vlan must be trunked from the LAN VDC's to the OTV VDC's in order for AED elections to take place etc... The way I remember to do this when configuring is by thinking of it as heartbeat L2 vlan for OTV devices at same site. (Terms used very loosely)

However you are correct in that the Site VLAN should NOT be extended on the overlay.

If in doubt, I can share some of my config if it helps.

/Bilal

[BalajiDasarathan]

Hi Bilal

This is a test setup with only one Nexus device per site, hence, there is no need for AED election, however, it would be really helpful if you could share the config. What is puzzling me is that the OTV configuration guide doesnt say anything about site VLAN to be created in the LAN VDC and to be trunked back to the OTV VDC. It would be certainly helpful to have your config for reference.

Cheers

Balaji

[Bilal Nawaz]

Oh I see. But I still think that the site VLAN is required even in stand alone.

In my case I have 2 N7K's per DC. What we've done is to create a vPC to the OTV vdc for resilience.

LAN VDC....

DC1-N7K1# show run int po2

interface port-channel2
  description PO-to-OTV-VDC-ON-DC1-N7K1
  switchport mode trunk
  switchport trunk native vlan 999
  switchport trunk allowed vlan 6,18,24,200-202,800,3967
  logging event port link-status
  logging event port trunk-status
  mtu 9216
  vpc 2

DC1-N7K1# show vlan id 3967

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
3967 OTV-SITE-VLAN                    active    Po1, Po2, Po3, Eth6/1, Eth6/2
                                                Eth7/1, Eth7/2

OTV VDC:

vlan 3967
  name OTV-SITE-VLAN

otv site-vlan 3967
otv site-identifier 0x100

interface Vlan3967
  no ip redirects
  ip address 192.168.1.29/30
  description OTV-SITE-VLAN-WITH-BFD
  no shutdown

interface port-channel2
  description link to both Cores LAN VDC for Stretched VLANs
  switchport mode trunk
  switchport trunk native vlan 999
  switchport trunk allowed vlan 6,18,24,200-202,800,3967
  logging event port link-status
  logging event port trunk-status
  mtu 9216

interface Overlay1
  otv join-interface Ethernet6/10
  otv control-group 239.1.1.1
  otv data-group 238.1.1.0/28
  otv extend-vlan 6, 18, 24, 200-202, 800
  no shutdown

Hope this helps. Let me know how it goes... Ignore the interface config in the otv vdc unless you want BFD.

[BalajiDasarathan]

Thank you very much Bilal. Some understanding, assumptions and clarifications;

1. VLAN 3967 is created in LAN VDC and is spanned through the trunk port to the OTV VDC

2. Both the chassis has OTV VDC and is dual connected to the LAN VDC

3. Eth 6/1 and 6/2 and Po2 connect to the OTV VDC in Nexus 1

4. Eth 7/1 and 7/2 and Po3 connect to the OTV VDC in Nexus 2

Now,

5. What does Po1 does on LAN VDC?

6. Where does Eth6/10 - Join interface connect to, is it in the same Nexus in a different VDC (like WAN VDC)?

7. Did you try Unicast only OTV without enabling multicast?

Also

8. If I have only one Nexus per site, I certainly dont need the Port channel and dual connect, hence, configuring just the L2 vlan on LAN VDC and trunking it to the OTV VDC should do good, is that right?

I appreciate your time and response, Thank you very much sir.

Cheers
Balaji

[Bilal Nawaz]

Hi Balaji, You are pretty much correct but the reverse with regards to Po interfaces etc..

1. VLAN 3967 is created in LAN VDC and is spanned through the trunk port to the OTV VDC

Yes

2. Both the chassis has OTV VDC and is dual connected to the LAN VDC

Yes, One OTV VDC is dual homed to N7K1 and N7K2 LAN VDC's - Good Design to have.

3. Eth 6/1 and 6/2 and Po2 connect to the OTV VDC in Nexus 1

No, 6/1 is part of the vPC PEER Link to the other N7K, 6/2 is to the OTV VDC in N7K2. No, Po2 is a vpc trunking to the OTV vdc N7K2. Po3 is to the local otv vdc.

4. Eth 7/1 and 7/2 and Po3 connect to the OTV VDC in Nexus 2

No, 7/1 is part of the vPC PEER Link to the other N7K we have 20Gb peer link, 7/2 is  to the OTV VDC in N7K1 (local). Po3 is local.

So in essence, N7K1 and 2 LAN VDC's provide N7K1 OTV VDC with a vpc 3.

and then N7K1 and 2 LAN VDC's provide N7K2 OTV VDC with a vpc 2

Both these vpc's are carrying the otv site vlan.

5. What does Po1 does on LAN VDC?

Po1 on the LAN vdc's on both N7Ks is the vpc PEER Link.

6. Where does Eth6/10 - Join interface connect to, is it in the same Nexus in a different VDC (like WAN VDC)?

Yes, absolutely. This is the OTV VDC's connection to the WAN network connects to local N7K in WAN VDC.

DC1-N7K1-OTV# show run int e6/10

interface Ethernet6/10
  description OTV JOIN Interface to DC1-N7K1-WAN E7/10
  rate-mode dedicated force
  no switchport
  mtu 9216
  no ip redirects
  ip address 192.168.1.2/30
  ip ospf authentication message-digest
  ip ospf message-digest-key 1 md5 3 aaaaaaaaaaaaaa
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 1 area 0.0.0.0
  ip igmp version 3
  no shutdown

7. Did you try Unicast only OTV without enabling multicast?

I did before, but since we have multiple sites, it was easy with mcast - had to implement MSDP for IPTV and easy deployment.

8. If I have only one Nexus per site, I certainly dont need the Port channel and dual connect, hence, configuring just the L2 vlan on LAN VDC and trunking it to the OTV VDC should do good, is that right?

Yes, correct - it is not a requirement, but we have implemented 2 N7K's per DC so for high resilience and quick failover we did this.

Hope this helps

[BalajiDasarathan]

Thank you very much for the clarification Bilal, I will test the configuration and will get back to you.

Have a good day.

Cheers

Balaji

[BalajiDasarathan]

Hi Bilal

I did a couple of test scenarios and was able to get the OTV working FINALLY. I would like to share my observation on the site VLAN configuration. My test design was with single Edge device per site with separate VDCs for the LAN and the WAN. The OTV Site VLAN doesnt need to be configured on the LAN VDC and trunked across to the OTV VDC, simply defining a L2 VLAN on the OTV VDC is sufficient for the functioning of the OTV.

Once we create a L2 VLAN and bring the state as active (this has to be explicitly done it seems, I have to type "state active" in the L2 VLAN configuration mode), we can use that as the OTV Site VLAN and doesnt need to be part of the trunk from the LAN VDC.

I would now be testing with a dual site edge device with vPC and will share my observation.

In short, I believe, the reason why my previous attempt to configure OTV was not working and was showing "site vlan down" is because the site vlan status was not made "active" on the OTV VDC.

Thanks for your time and guidance.

-Balaji

[BalajiDasarathan]

The configuration I used is as follows;

LAN VDC;

interface Ethernet1/1
  switchport mode trunk                         ----------> trunk to OTV VDC
  switchport trunk allowed vlan 100,200,999
  no shutdown

vlan 1,100,200,999

interface Vlan1

interface Vlan100
  no shutdown                            -----------------> Local VLANs
  ip address 10.100.100.1/24

interface Vlan200
  no shutdown                            ------------------>Local VLANs
  ip address 10.100.200.1/24

interface Vlan999
  no shutdown                              ----------------> VLAN to be extended
  ip address 10.254.99.1/24

WAN VDC;

interface Ethernet2/9                    ------------------> L3 Link to OTV VDC
  ip address 192.168.99.1/30
  no shutdown

interface Ethernet2/10                  ------------------->WAN Link to other Nexus
  ip address 192.168.100.1/30
  no shutdown

OTV VDC;

vlan 1,10,999
vlan 10
  name OTV-Site-VLAN
vlan 999
  name OTV-Extended-VLAN

otv site-vlan 10

interface Overlay1
  otv join-interface Ethernet2/8
  otv extend-vlan 999
  otv use-adjacency-server 192.168.99.5 unicast-only
  no shutdown

interface Ethernet2/5
  description OTV Internal Interface
  switchport
  switchport mode trunk
  switchport trunk allowed vlan 10,100,200,999
  no shutdown

interface Ethernet2/8                     --------------> Join Interface

  ip address 192.168.99.2/30
  ip igmp version 3
  no shutdown
otv site-identifier 0x2

sh otv

OTV Overlay Information
Site Identifier 0000.0000.0002

Overlay interface Overlay1

 VPN name            : Overlay1
 VPN state           : UP
 Extended vlans      : 999 (Total:1)
 Join interface(s)   : Eth2/8 (192.168.99.2)
 Site vlan           : 10 (up)
 AED-Capable         : Yes
 Capability          : Unicast-Only
 Is Adjacency Server : No
 Adjacency Server(s) : 192.168.99.5 / [None]

[Bilal Nawaz]

Hey that's great news! and great feedback too. I must have got muddled with the ASR's! Let me know how it goes with the dual site edge devices... I have never done this but maybe we could just have trunk between OTV VDC's for site adjacency and AED. I assume that it must work since all you did was create the vlan on the OTV VDC. :)

[BalajiDasarathan]

Thank you sir, I am assuming that even the trunk is not required and probably the vPC should get the adjacency going, however, I will update this thread with my topology, configuration and observation.

Not just create the VLAN on the OTV VDC but to have the state as active... ;-) I believe that made the difference.

Have a wonderful day.