Introduction
This document describes a scenario where user is trying to achieve redundancy for OTV setup using dual ISP and single VDC. There are two separate provider links connecting the main site to the provider site which are L2 links. At the main site there are two Nexus 7009 (NXOS 6.0) and at DR there is one Nexus 7009 (NXOS 6.0) Trying to leverage OTV so that can have the VLANs extended across both links in a redundant fashion.
About OTV
The core principles on which OTV operates are the use of a control protocol to advertise MAC address reachability information (instead of using data plane learning) and packet switching of IP encapsulated Layer 2 traffic (instead of using circuit switching) for data forwarding. OTV is a MAC-in-IP method that extends Layer 2 connectivity across a transport network infrastructure. OTV uses MAC address-based routing and IP-encapsulated forwarding across a transport network to provide support for applications that require Layer 2 adjacency.
With OTV you can deploy virtual computing resources and clusters across geographically distributed data centers, delivering:
Transparent workload mobility
Business resilience
Superior computing resource efficiency
Description
Each N7k has a "main VDC" (with all interfaces and vlan) and a "OTV VDC". The connection between ISP and N7k's is as follows:
1. Main site switch1-OTV VDC <---> ISP1 <---> DR site switch1 OTV VDC
2. Main site switch2-OTV VDC <---> ISP2 <---> DR site switch1 OTV VDC
The first connection works fine. For the second connection the site vlan at the main site was extended only from switch #1 to switch #2 and then build the same overlay interface with the same control group and data group extending the same vlans at switch one. When trying to move over to DR and create a second overlay interface joining the second provider link wont work because you could not extend the same vlans controlgroup and data group within the same OTV VDC.
Nexus 7000 Configuration
Switch1 main site OTV runn config:
version 6.1(2)
switchname OTV
feature otv
vrf context management
vlan 1,10,20,58,100,295,298,300,302-303
otv site-vlan 100
interface Overlay1
otv join-interface Ethernet9/24
otv control-group 239.1.1.1
otv data-group 232.0.0.0/8
otv extend-vlan 10, 20, 58, 295, 298, 300, 302-303
no shutdown
interface Ethernet9/20
description eth9-10-SW1-Core-VDC
switchport
switchport mode trunk
mtu 9216
no shutdown
interface Ethernet9/21
interface Ethernet9/22
interface Ethernet9/23
interface Ethernet9/24
description PPP-XO DR-OTV-VDC
mtu 9216
ip address 10.175.1.1/30
ip igmp version 3
no shutdown
line vty
otv site-identifier 0000.0000.0001
Switch2 main site OTV runn-config:
version 6.1(2)
hostname OTV
feature otv
feature lacp
ip domain-lookup
vrf context management
vlan 1,10,20,58,100,295,300,303
otv site-vlan 100
interface port-channel8
description po8.SW2-Core-VDC
switchport
switchport mode trunk
interface Overlay1
otv join-interface Ethernet9/23
otv control-group 239.1.1.1
otv data-group 232.0.0.0/8
otv extend-vlan 10, 20, 58, 295, 300, 303
interface Ethernet9/20
description eth9-10.sw2-Core-VDC
switchport
switchport mode trunk
channel-group 8 mode active
no shutdown
interface Ethernet9/21
description eth9-11.sw2-Core-VDC
switchport
switchport mode trunk
channel-group 8 mode active
no shutdown
interface Ethernet9/23
description PPP-Fidelity-DR-OTV
mtu 9100
no ip redirects
ip address 10.175.1.5/30
ip igmp version 3
no shutdown
interface Ethernet9/24
line vty
otv site-identifier 0000.0000.0001
OTV VDC config:
version 6.1(2)
switchname OTV
feature otv
feature lacp
vrf context management
vlan 1,10,20,58,101,295,298,300,302-303
otv site-vlan 101
interface port-channel8
description Po8.DR-Core-VDC
switchport
switchport mode trunk
mtu 9216
interface Overlay1
otv join-interface Ethernet9/24
otv control-group 239.1.1.1
otv data-group 232.0.0.0/8
otv extend-vlan 10, 20, 58, 295, 298, 300, 302-303
no shutdown
interface Overlay2
otv join-interface Ethernet9/23
interface Ethernet9/19
description eth9-12.DR-Core-VDC
switchport
switchport mode trunk
mtu 9216
channel-group 8 mode active
no shutdown
interface Ethernet9/20
description eth9-17.DR-Core-VDC
switchport
switchport mode trunk
mtu 9216
channel-group 8 mode active
no shutdown
interface Ethernet9/21
description eth9-18.Dr-Core-VDC
switchport
switchport mode trunk
mtu 9216
channel-group 8 mode active
no shutdown
interface Ethernet9/23
description PPP-Fidelity eth9-23.-Sw2-main-OTV
mtu 9100
no ip redirects
ip address 10.175.1.6/30
ip igmp version 3
no shutdown
otv site-identifier 0000.0000.0002
Resolution
You would need to create another OTV VDC at DR site and have the trunks carry the same exended Vlans to the OTV VDC 2. The OTV internal interfaces carry the VLANs to be extended and the OTV site VLAN (used within the data center to provide multihoming). The overlay interfaces encapsulate Layer 2 frames in IP unicast or multicast packets and are logical multiaccess, multicast-capable interfaces. The join interfaces are point-to-point routed interfaces that are used by the OTV edge devices to join the overlay network. The same OTV VDCs can be used by multiple VDCs deployed at the aggregation tier, as well as by other Layer 2 switches connected to the OTV VDCs. This is done by configuring multiple OTV overlays. It’s important to note that the extended VLANs within these multiple overlays should not overlap.
Related Information
Troubleshooting OTV Adjacency
Troubleshooting ARP issues across OTV
