NEXUS 7k doesn't encapsulate on Tunnel GRE

[giuseppe.tetti1]

Dear all,

we have following NEXUS:

NEXUS# sh ver
Cisco Nexus Operating System (NX-OS) Software
Software
  BIOS:      version 2.12.0
  kickstart: version 8.2(1)
  system:    version 8.2(1)
 Hardware
  cisco Nexus7000 C7009 (9 Slot) Chassis ("Supervisor Module-2")
  Intel(R) Xeon(R) CPU         with 12296612 kB of memory.

Configured with a tunnel GRE as follows:

NEXUS# sh int tun 55
Tunnel55 is up
    Description: tunnel per MEB
    Admin State: up
    Internet address is 10.250.250.14/30  
    MTU 1476 bytes, BW 9 Kbit
    Tunnel protocol/transport GRE/IP
    Tunnel source 10.250.1.61, destination 10.50.100.200

What we experience is that communications from NEXUS go to Tunnel55, while communication coming from another device belonging to the same VLAN  doesn’t transit to Tunnel 55 (as you can see follow):

NEXUS# traceroute 10.50.100.1 source vl5

traceroute to 10.50.100.1 (10.50.100.1) from 10.50.160.1 (10.50.160.1), 30 hops max, 40 byte packets

 1  10.250.250.13 (10.250.250.13)  4.41 ms  5.358 ms  3.755 ms

 2  10.50.100.1 (10.50.100.1)  4.321 ms  3.358 ms  3.638 ms

PSP-CORE#

Other-Device#traceroute 10.50.100.1 source vlan 5
Type escape sequence to abort.
Tracing the route to 10.50.100.1
VRF info: (vrf in name/id, vrf out name/id)
  1 10.50.160.1 2 msec 3 msec 3 msec
  2  *  *  *
  3  *  *  *
  4  *  *  *
  5  *  *  *
  6  *  *  *
  7  *  *  *
  8 10.50.100.1 7 msec 4 msec 4 msec

The question is, do we need to enable something on NEXUS that we don’t, or need to get some particular license ?

Thank you for your kind help.

[balaji.bandi]

Can you post the tunnel config? 

PSP-CORE -  we see the traceroute completed right? what is the issue ?

you need to check on the next hop same traceroute see how the routing taking place - 10.50.160.1 

[giuseppe.tetti1]

Sorry PSP-CORE is a mistake, don't care about it. Consider just "NEXUS" and "Other-Device" device.

Here is a part of NEXUS config:

feature telnet

feature interface-vlan

feature hsrp

feature tunnel

feature lldp

interface Vlan5

  description vlan MEB

  no shutdown

  ip address 10.50.160.1/24

interface Vlan799

  no shutdown

  no ip redirects

  ip address 10.250.1.61/28

interface Tunnel55

  ip address 10.250.250.14/30

  tunnel source 10.250.1.61

  tunnel destination 10.50.100.200

  description tunnel per MEB

  mtu 1400

  no shutdown

  tunnel path-mtu-discovery

ip route 0.0.0.0/0 10.250.1.49

ip route 10.50.0.0/16 Tunnel55

ip route 10.50.100.0/24 Tunnel55

ip route 10.50.100.200/30 10.250.1.49

ip route 10.50.100.248/32 10.250.1.49

[MHM Cisco World]

traceroute 10.50.100.1 source vl5 <<- how you use same destination in traceroute in both side ?? 

[giuseppe.tetti1]

I have shown just one side of the tunnel, the source side. VLAN5 is the sorce of traffic to be encapsuleted in tunnel 55.

[MHM Cisco World]

but why th same destination appear in both side ?? 

ip route 10.50.0.0/16 Tunnel55

ip route 10.50.100.0/24 Tunnel55

so you can ping from NSK 10.50.0.0/16 or 10.50.100.0/24 using source that reachable from other side. 
VLAN5 (SVI IP) is add as srtatic route toward tunnel in other side

[giuseppe.tetti1]

As you can see, reachbility to 10.50.0.0 from vlan5 work fine from NEXUS by Tunnel55:

NEXUS# traceroute 10.50.100.1 source vl5

traceroute to 10.50.100.1 (10.50.100.1) from 10.50.160.1 (10.50.160.1), 30 hops max, 40 byte packets

 1  10.250.250.13 (10.250.250.13)  4.41 ms  5.358 ms  3.755 ms

 2  10.50.100.1 (10.50.100.1)  4.321 ms  3.358 ms  3.638 ms

The problem is reachability of 10.50.0.0 from any other device on VLAN 5 that use NEXUS as default-gateway, as you can see below:

Other-Device#traceroute 10.50.100.1 source vlan 5
Type escape sequence to abort.
Tracing the route to 10.50.100.1
VRF info: (vrf in name/id, vrf out name/id)
  1 10.50.160.1 2 msec 3 msec 3 msec
  2  *  *  *
  3  *  *  *
  4  *  *  *
  5  *  *  *
  6  *  *  *
  7  *  *  *
  8 10.50.100.1 7 msec 4 msec 4 msec

Trace route go to 10.50.100.1 but hops are not the tunnel55 ones. So it seems that traffic fron vlan5 does not encapsulate on GRE Tunnel.

[MHM Cisco World]

Are other device use same ip sunbet for vlan 5

[MHM Cisco World]

I run lab and I success to ping from R1 in VLAN1 and R3 connect via router port to NSK. 
can you check if other side using 
tunnel mode gre ip <<- under the tunnel config ??

[giuseppe.tetti1]

Here is config of the other site, just see tunnel 55:

interface Tunnel4
 description tunnel per LDP in conformita' alla VLAN ERG
 ip address 10.250.250.1 255.255.255.252
 ip pim sparse-dense-mode
 tunnel source 172.18.8.250
 tunnel destination 10.110.255.1
!
interface Tunnel5
 description tunnel per MEB in conformita' alla VLAN ERG
 ip address 10.250.250.5 255.255.255.252
 tunnel source 10.50.100.200
 tunnel destination 10.110.255.2
!
interface Tunnel15
 description tunnel per LDP in conformita' alla VLAN ERG
 ip address 10.250.250.9 255.255.255.252
 tunnel source 10.55.101.1
 tunnel destination 10.55.12.1
!
interface Tunnel55
 description tunnel per MEB in conformita' alla VLAN ERG
 ip address 10.250.250.13 255.255.255.252
 ip mtu 1400
 tunnel source 10.50.100.200
 tunnel destination 10.250.1.61
!
!
interface FastEthernet1
 switchport access vlan 5
!
interface FastEthernet2
 switchport access vlan 4
!
interface FastEthernet3
 switchport access vlan 15
!
interface Vlan1
 ip address 10.1.10.68 255.255.0.0
 shutdown
 standby 50 ip 10.1.10.60
 standby 50 priority 99
 standby 50 preempt
!
interface Vlan4
 description tunnel per LDP in conformita' alla VLAN ERG
 ip address 172.18.12.251 255.255.252.0 secondary
 ip address 172.18.12.250 255.255.252.0 secondary
 ip address 172.18.8.251 255.255.252.0
 ip pim sparse-dense-mode
 standby 4 ip 172.18.8.250
 standby 4 priority 99
 standby 4 preempt
!
interface Vlan5
 description interfaccia per Meb
 ip address 10.50.100.201 255.255.255.0
 standby 5 ip 10.50.100.200
 standby 5 priority 99
 standby 5 preempt
!
interface Vlan15
 description tunnel per OCP in conformita' alla VLAN OCP
 ip address 10.55.101.3 255.255.255.0
 standby 15 ip 10.55.101.1
 standby 15 priority 99
 standby 15 preempt
!
ip route 10.0.0.0 255.0.0.0 10.50.100.252
ip route 10.50.144.0 255.255.240.0 Tunnel5
ip route 10.50.160.0 255.255.255.0 Tunnel55
ip route 10.55.0.0 255.255.0.0 172.18.12.161
ip route 10.55.12.0 255.255.255.0 Tunnel15
ip route 10.55.12.1 255.255.255.255 10.1.40.32
ip route 10.55.12.249 255.255.255.255 10.1.10.11
ip route 10.110.255.0 255.255.255.0 10.1.40.32
ip route 10.110.255.1 255.255.255.255 172.16.8.164
ip route 10.110.255.2 255.255.255.255 10.50.100.252
ip route 10.250.1.61 255.255.255.255 10.50.100.252
ip route 172.16.0.0 255.252.0.0 172.18.12.161
ip route 172.20.0.0 255.255.0.0 Tunnel4
ip route 172.20.255.249 255.255.255.255 FastEthernet1
ip route 188.94.133.139 255.255.255.255 10.50.100.252
ip route 195.188.150.133 255.255.255.255 10.50.100.252
!
!
no ip http server
no ip http secure-server
ip pim bidir-enable
!
!        
!
!
!
control-plane
!
!
line con 0
 no modem enable
line aux 0
line vty 0 4
 privilege level 15
 password xxxxxxx
 login
!
scheduler max-task-time 5000
!
webvpn context Default_context
 ssl authenticate verify all
 !
 no inservice
!
end

[MHM Cisco World]

NSK# ethanalyzer local interface inband capture-filter "host x.x.x.x or host y.y.y.y"
run ethanalyzer and do traceroute again may be we get hint why the traffic drop 

[giuseppe.tetti1]

Here what you are requested:

Castel_Fusano#traceroute 10.50.100.1 source vlan 6  (10.50.190.5)
Type escape sequence to abort.
Tracing the route to 10.50.100.1
VRF info: (vrf in name/id, vrf out name/id)
  1 10.50.190.1 3 msec 3 msec 3 msec
  2  *  *  *
  3  *  *  *
  4  *  *  *
  5  *  *  *
  6  *  *  *
  7  *  *  *
  8 10.50.100.1 11 msec 5 msec *

NEXUS# ethanalyzer local interface inband capture-filter "host 10.50.190.5"
Capturing on inband
2023-03-09 01:24:33.534726 10.50.190.5 -> 10.50.100.1 UDP 60 Source port: 51825 Destination port: traceroute
2 2023-03-09 01:24:33.535246 10.50.190.1 -> 10.50.190.5 ICMP 70 Time-to-live exceeded (Time to live exceeded in transit)
2023-03-09 01:24:41.550019 10.50.190.5 -> 10.50.100.1 UDP 60 Source port: 51826 Destination port: 33435
2023-03-09 01:24:41.550500 10.50.190.1 -> 10.50.190.5 ICMP 70 Time-to-live exceeded (Time to live exceeded in transit)
2023-03-09 01:24:41.552928 10.50.190.5 -> 10.50.100.1 UDP 60 Source port: 51827 Destination port: 33436
6 2023-03-09 01:24:41.553357 10.50.190.1 -> 10.50.190.5 ICMP 70 Time-to-live exceeded (Time to live exceeded in transit)
2023-03-09 01:24:44.398220 00:87:64:51:b3:41 -> d4:e8:80:e5:20:f7 ARP 60 Who has 10.50.190.5? Tell 10.50.190.1
8 2023-03-09 01:24:44.400774 d4:e8:80:e5:20:f7 -> 00:87:64:51:b3:41 ARP 60 10.50.190.5 is at d4:e8:80:e5:20:f7

[MHM Cisco World]

but why you use VLAN6 not VLAN5 ???

[giuseppe.tetti1]

Here, from vlan5:

Castel_Fusano#traceroute 10.50.100.1 source vlan 5 (10.50.160.5)
Type escape sequence to abort.
Tracing the route to 10.50.100.1
VRF info: (vrf in name/id, vrf out name/id)
  1 10.50.160.1 2 msec 3 msec 3 msec
  2  *  *  *
  3  *  *  *
  4  *  *  *
  5  *  *  *
  6  *  *  *
  7  *  *  *
  8 10.50.100.1 7 msec 4 msec 4 msec

NEXUSE# ethanalyzer local interface inband capture-filter "host 10.50.160.5"
Capturing on inband
2023-03-13 05:44:33   10.50.160.5 -> 10.50.100.1  UDP 60 Source port: 51825  Destination port: traceroute
2 2023-03-13 05:44:33   10.50.160.1 -> 10.50.160.5  ICMP 70 Time-to-live exceeded (Time to live exceeded in transit)
2023-03-13 05:44:41   10.50.160.5 -> 10.50.100.1  UDP 60 Source port: 51826  Destination port: 33435
2023-03-13 05:44:41   10.50.160.1 -> 10.50.160.5  ICMP 70 Time-to-live exceeded (Time to live exceeded in transit)
2023-03-13 05:44:41   10.50.160.5 -> 10.50.100.1  UDP 60 Source port: 51827  Destination port: 33436
6 2023-03-13 05:44:41   10.50 .160.1 -> 10.50.160.5  ICMP 70 Time-to-live exceeded (Time to live exceeded in transit)
2023-03-13 05:44:44  00:87:64:51:b3:41 -> d4:e8:80:e5:20:f7 ARP 60 Who has 10.50.160.5?  Tell 10.50 .160.1
8 2023-03-13 05:44:44  d4:e8:80:e5:20:f7 -> 00:87:64:51:b3:41 ARP 60 10.50.160.5 is at d4:e8:80:e5:20:f7

Anyway, do you think we can esclude any issue due to lack of licenses or service to be activated  ?

[MHM Cisco World]

believe me I try every last three days run cases to match your condition and I failed, 
license, I dont think so, if it need license then how direct SVI can trace route without issue. 
what I get from your ethanalzyer ?
2023-03-13 05:44:44  00:87:64:51:b3:41 -> d4:e8:80:e5:20:f7 ARP 60 Who has 10.50.160.5?  Tell 10.50 .160.1
8 2023-03-13 05:44:44  d4:e8:80:e5:20:f7 -> 00:87:64:51:b3:41 ARP 60 10.50.160.5 is at d4:e8:80:e5:20:f7

are two or more SVI share same MAC ? can you check this point.