Yea, the Nexus stuff is - Page 2

Kemal Zuko · ‎03-11-2015

Hello Again,

On our new test network that consists of two Nexus 6004's and two Nexus 9372's that run vPC between the two. The network VLANs live on the Nexus 6004's and extend down to 9K's via vPC links

On one of our 9K's we have a host in

switchport access vlan 680

Simple right? :) well there is nothing simple about this network. From the nexus 6004 we can ping the host that sits behind vlan 680 but when we try to ping from the Nexus 9372 (leaf switch) that is directly connected to the host we get the following:

NExus9K# ping 172.16.8.199
PING 172.16.8.199 (172.16.8.199): 56 data bytes
ping: sendto 172.16.8.199 64 chars, No route to host
Request 0 timed out
ping: sendto 172.16.8.199 64 chars, No route to host
Request 1 timed out
ping: sendto 172.16.8.199 64 chars, No route to host
Request 2 timed out
ping: sendto 172.16.8.199 64 chars, No route to host
Request 3 timed out
ping: sendto 172.16.8.199 64 chars, No route to host
Request 4 timed out

--- 172.16.8.199 ping statistics ---
5 packets transmitted, 0 packets received, 100.00% packet loss

This switch is strictly L2

here is some config for reference

Nexus 6004 Primary

vlan 450
name P2P_VRF_SVI_DMZ
vlan 451
name P2P_VRF_SVI_Inside
vlan 600
name DMZ
vlan 652
name Management
vlan 680
name Inside
vrf context DMZ
vrf context Inside
vrf context management
ip route 0.0.0.0/0 172.16.52.1
vrf context peer-keepalive
vpc domain 99
role priority 1
peer-keepalive destination 10.200.50.2 source 10.200.50.1 vrf peer-keepalive
delay restore 120

interface Vlan1

interface Vlan450
description DMZ P2P to ASA
no shutdown
vrf member DMZ
ip address 172.16.230.1/29
ip router eigrp 100
no ip passive-interface eigrp 100

interface Vlan451
description Inside p2p to ASA
no shutdown
vrf member Inside
ip address 172.16.230.9/29
ip router eigrp 100
no ip passive-interface eigrp 100

interface Vlan600
description DMZ
no shutdown
vrf member DMZ
ip address 172.16.0.2/22
ip router eigrp 100
hsrp 2
authentication text test1
preempt
priority 250
ip 172.16.0.1

interface Vlan651

interface Vlan680
description Inside Network
no shutdown
vrf member Inside
ip address 172.16.8.2/22
ip router eigrp 100
hsrp 1
authentication text test
preempt
priority 250
ip 172.16.8.1

interface port-channel99
description vPC Etherchannel
switchport mode trunk
switchport trunk allowed vlan 600,680
spanning-tree port type network
vpc peer-link

interface port-channel102
description vPC to Nexus 9372
switchport mode trunk
switchport trunk allowed vlan 600,680
vpc 102

interface Ethernet1/1
description vPC Peer Link 1.1
switchport mode trunk
switchport trunk allowed vlan 600,680
speed auto
channel-group 99 mode active

interface Ethernet1/7
description vPC Peer Link 1.7 to Nexus 9372 PRI
switchport mode trunk
switchport trunk allowed vlan 600,680
speed auto
channel-group 102 mode active

interface Ethernet2/1
description vPC Peer Link 2.1
switchport mode trunk
switchport trunk allowed vlan 600,680
speed auto
channel-group 99 mode active

interface Ethernet2/7
description vPC Peer Link 2.1 to Nexus SEC
switchport mode trunk
switchport trunk allowed vlan 600,680
speed auto
channel-group 102 mode active

interface Ethernet8/1
description keep-alive peer-link to ALNSWI02
no switchport
vrf member peer-keepalive
ip address 10.200.50.1/30

interface Ethernet8/2
description Uplink to ASA
switchport mode trunk
switchport trunk allowed vlan 450-451

interface Ethernet8/9
description EIGRP PORT
switchport mode trunk
switchport trunk allowed vlan 450-451

interface mgmt0
vrf member management
ip address 172.16.52.3/23
line console
line vty
boot kickstart bootflash:/n6000-uk9-kickstart.7.0.1.N1.1.bin
boot system bootflash:/n6000-uk9.7.0.1.N1.1.bin
router eigrp 100
autonomous-system 100
vrf DMZ
autonomous-system 100
router-id 172.16.0.1
default-information originate
vrf Inside
autonomous-system 100
router-id 172.16.230.9
default-information originate
poap transit

---------------------

Primary Leaf Nexus 9372

vlan 1,600,652,680
vlan 600
name DMZ
vlan 652
name Managment
vlan 680
name Inside

vrf context management
ip route 0.0.0.0/0 172.16.52.1
vrf context peer-keepalive
vpc domain 101
role priority 1
peer-keepalive destination 10.200.50.6 source 10.200.50.5 vrf peer-keepalive

interface Vlan1

interface Vlan652
no shutdown

interface port-channel101
switchport mode trunk
switchport trunk allowed vlan 600,680
spanning-tree port type network
vpc peer-link

interface port-channel102
switchport mode trunk
switchport trunk allowed vlan 600,680
vpc 102

sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 99
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po99 up 600,680

vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
102 Po102 up success success 600,680

interface Ethernet1/16
description HOST <<<<<<<<<<<<<<<<<<<<<<<<<<<< This is the host that we cant reach<<<<<<
switchport access vlan 680

interface Ethernet1/17
description SERVER1
switchport mode trunk
switchport trunk allowed vlan 600,680

interface Ethernet1/46
description keep-alive peer-link to ALNSWI04
no switchport
vrf member peer-keepalive
ip address 10.200.50.5/30
no shutdown

interface Ethernet1/47
description vPC Peer Link 1.47
switchport mode trunk
switchport trunk allowed vlan 600,680
channel-group 101 mode active

interface Ethernet1/48
description vPC Peer Link 2.48
switchport mode trunk
switchport trunk allowed vlan 600,680
channel-group 101 mode active

interface Ethernet1/49
description vPC Link 1.49 to Nexus 6004 PRI
switchport mode trunk
switchport trunk allowed vlan 600,680
channel-group 102 mode active

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53
description vPC Link 1.53 to Nexus 6004 SEC
switchport mode trunk
switchport trunk allowed vlan 600,680
channel-group 102 mode active

interface Ethernet1/54

interface mgmt0
vrf member management
ip address 172.16.52.5/23
line console
line vty
boot nxos bootflash:/n9000-dk9.6.1.2.I3.1.bin

sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 101
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po101 up 600,680

vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- ------ ------------
102 Po102 up success success 600,680

----------------------------------------------------------------------------------------------------------------

So I confused on the ping status from the Leaf switch when it says "no route to host"

Any help is appreciated

Thank you

Bilal Nawaz · ‎03-13-2015

I said yesterday that its something on the ASA. hehe.

Please rate useful posts & remember to mark any solved questions as answered. Thank you.

Jon Marshall · ‎03-13-2015

Yes you did :-)

Jon

Kemal Zuko · ‎03-13-2015

Yea, the Nexus stuff is definetley a different beast ...

Jon Marshall · ‎03-13-2015

Kemal

Actually if that is the issue you can't allow it on the trunk link because it would be a vPC vlan.

I don't think you can extend that vPC vlan back to the ASA unless you connect the ASA using a vPC in which case you would need to not run EIGRP and use HSRP with static routes.

Or you could create another matching pair as you did with vlans 450 and 451 and vlans 600 and 680.

You would then create SVIs for vlan 652 on the Nexus switches and use a new non vPC vlan in the same VRF to extend it back to the ASA.

Jon

Bilal Nawaz · ‎03-12-2015

Hello Kemal,

your output indicates that you have no route to the host from the N9K. You should have route via N6K because its directly connected. Can you ping from the gateway i.e. Nexus 6K?

If so then its working as expected, if you want the N9K to be able to ping, it needs a route - it needs to know how to get to the host in vlan 680. You can see if the N9K has a route by doing "show ip route 172.16.8.199"

Your vpc config seems ok from what i've seen.

Bilal

Please rate useful posts & remember to mark any solved questions as answered. Thank you.

Kemal Zuko · ‎03-12-2015

Hello Bilal,

Yes I can ping the host from the Nexus 6004, and I can also ping the host from my ASA which hangs off the 6004. That tells me that the packet flow works, however I am a little confused with the Nexus lineup on why everything needs a route, even though the vlan 680 is extended from the Nexus 6004 (Gateway) to the Nexus 9372.

There is no route in the N9372 other then the management.

Bilal Nawaz · ‎03-12-2015

Kemal, lets put ourselves in the shoe's of the Nexus 9372. If we want to get to 172.16.8.199, we always do a lookup of some sort - route lookup. Do I know how to get to x.x.x.x? In this case from a layer 3 point of view, answer is no, the N9K does not know how to get there, because there is no route.

From a layer 2 perspective, yes by all means it is carrying that VLAN, you'll be able to see layer 2 information like mac address's for hosts on that vlan, it is providing a means for transit traffic on that segment. But for this scenario we do not have a layer 3 interface with an IP in the 680 vlan. Quite opposite on the N6K, we do have an interface in vlan 680, everything to your ASA is routed, they have routes to the destination network. The ping works.

To get the ping to work on N9K, you would have to put in a route (static) or make the N9K part of your dynamic routing protocol (if you are using one) to learn about the route to the vlan 680.

Hope this helps

Bilal

Please rate useful posts & remember to mark any solved questions as answered. Thank you.

Kemal Zuko · ‎03-12-2015

Makes sense of course. I see how the Nexus switches are different from the old IOS switch.

If I create a static route of some sort am I taking the Switch out of L2 mode?

And would I just create a static route pointing to the core gateway?

ip route 172.16.8.0 255.255.252.0 172.16.8.1 <<<< this one being the gateway?

Jon Marshall · ‎03-12-2015

Kemal

Do you have an SVI on the 9372 for management.

If you do then the gateway is the IP of the SVI on the Nexus switches for that vlan.

On IOS for a L2 switch you used to configure a default gateway but I believe with NXOS even for a L2 switch you configure a route.

Jon

Kemal Zuko · ‎03-12-2015

Hi Jon,

No I don't have an SVI for MGMT. I am using the MGMT interface for management.

vrf context management
ip route 0.0.0.0/0 172.16.52.1

interface mgmt0
vrf member management
ip address 172.16.52.5/23

Bilal Nawaz · ‎03-12-2015

Sorry Kemal, can you try one thing for me please?

On your nexus 9K do this:

ping 172.16.8.199 vrf management

Tell me if it works.

Please rate useful posts & remember to mark any solved questions as answered. Thank you.

Kemal Zuko · ‎03-12-2015

hi Bilal,

No I cant. Here is the output

vrf context management
ip route 0.0.0.0/0 172.16.52.1

interface mgmt0
vrf member management
ip address 172.16.52.5/23

NK9 # ping 172.16.8.199 vrf management
PING 172.16.8.199 (172.16.8.199): 56 data bytes
Request 0 timed out
Request 1 timed out
Request 2 timed out
Request 3 timed out
Request 4 timed out

--- 172.16.8.199 ping statistics ---
5 packets transmitted, 0 packets received, 100.00% packet loss

Bilal Nawaz · ‎03-12-2015

Kemal, does the host 172.16.52.1 have a route to 172.16.8.199. What is 172.16.52.1?

May I also ask if it is a real requirement for an "access switch" to reach this server on vlan 680 :)

Please rate useful posts & remember to mark any solved questions as answered. Thank you.

Kemal Zuko · ‎03-12-2015

172.16.52.1 is our ASA. It handles all the MGMT stuff.

We would like to have that feature in case of troubleshooting...

Bilal Nawaz · ‎03-12-2015

So I think your policy on the ASA is blocking the ping? Take a look at the logs on the ASA and ping the same time, you should see the traffic if a deny rule is set for logging.

i would say that your troubleshooting is better off on the N6K because there's less hops to go through.

Please rate useful posts & remember to mark any solved questions as answered. Thank you.

Host behind vPC not responding