Okay, I didn't realise you

Kemal Zuko · ‎03-11-2015

Hello Again,

On our new test network that consists of two Nexus 6004's and two Nexus 9372's that run vPC between the two. The network VLANs live on the Nexus 6004's and extend down to 9K's via vPC links

On one of our 9K's we have a host in

switchport access vlan 680

Simple right? :) well there is nothing simple about this network. From the nexus 6004 we can ping the host that sits behind vlan 680 but when we try to ping from the Nexus 9372 (leaf switch) that is directly connected to the host we get the following:

NExus9K# ping 172.16.8.199
PING 172.16.8.199 (172.16.8.199): 56 data bytes
ping: sendto 172.16.8.199 64 chars, No route to host
Request 0 timed out
ping: sendto 172.16.8.199 64 chars, No route to host
Request 1 timed out
ping: sendto 172.16.8.199 64 chars, No route to host
Request 2 timed out
ping: sendto 172.16.8.199 64 chars, No route to host
Request 3 timed out
ping: sendto 172.16.8.199 64 chars, No route to host
Request 4 timed out

--- 172.16.8.199 ping statistics ---
5 packets transmitted, 0 packets received, 100.00% packet loss

This switch is strictly L2

here is some config for reference

Nexus 6004 Primary

vlan 450
name P2P_VRF_SVI_DMZ
vlan 451
name P2P_VRF_SVI_Inside
vlan 600
name DMZ
vlan 652
name Management
vlan 680
name Inside
vrf context DMZ
vrf context Inside
vrf context management
ip route 0.0.0.0/0 172.16.52.1
vrf context peer-keepalive
vpc domain 99
role priority 1
peer-keepalive destination 10.200.50.2 source 10.200.50.1 vrf peer-keepalive
delay restore 120

interface Vlan1

interface Vlan450
description DMZ P2P to ASA
no shutdown
vrf member DMZ
ip address 172.16.230.1/29
ip router eigrp 100
no ip passive-interface eigrp 100

interface Vlan451
description Inside p2p to ASA
no shutdown
vrf member Inside
ip address 172.16.230.9/29
ip router eigrp 100
no ip passive-interface eigrp 100

interface Vlan600
description DMZ
no shutdown
vrf member DMZ
ip address 172.16.0.2/22
ip router eigrp 100
hsrp 2
authentication text test1
preempt
priority 250
ip 172.16.0.1

interface Vlan651

interface Vlan680
description Inside Network
no shutdown
vrf member Inside
ip address 172.16.8.2/22
ip router eigrp 100
hsrp 1
authentication text test
preempt
priority 250
ip 172.16.8.1

interface port-channel99
description vPC Etherchannel
switchport mode trunk
switchport trunk allowed vlan 600,680
spanning-tree port type network
vpc peer-link

interface port-channel102
description vPC to Nexus 9372
switchport mode trunk
switchport trunk allowed vlan 600,680
vpc 102

interface Ethernet1/1
description vPC Peer Link 1.1
switchport mode trunk
switchport trunk allowed vlan 600,680
speed auto
channel-group 99 mode active

interface Ethernet1/7
description vPC Peer Link 1.7 to Nexus 9372 PRI
switchport mode trunk
switchport trunk allowed vlan 600,680
speed auto
channel-group 102 mode active

interface Ethernet2/1
description vPC Peer Link 2.1
switchport mode trunk
switchport trunk allowed vlan 600,680
speed auto
channel-group 99 mode active

interface Ethernet2/7
description vPC Peer Link 2.1 to Nexus SEC
switchport mode trunk
switchport trunk allowed vlan 600,680
speed auto
channel-group 102 mode active

interface Ethernet8/1
description keep-alive peer-link to ALNSWI02
no switchport
vrf member peer-keepalive
ip address 10.200.50.1/30

interface Ethernet8/2
description Uplink to ASA
switchport mode trunk
switchport trunk allowed vlan 450-451

interface Ethernet8/9
description EIGRP PORT
switchport mode trunk
switchport trunk allowed vlan 450-451

interface mgmt0
vrf member management
ip address 172.16.52.3/23
line console
line vty
boot kickstart bootflash:/n6000-uk9-kickstart.7.0.1.N1.1.bin
boot system bootflash:/n6000-uk9.7.0.1.N1.1.bin
router eigrp 100
autonomous-system 100
vrf DMZ
autonomous-system 100
router-id 172.16.0.1
default-information originate
vrf Inside
autonomous-system 100
router-id 172.16.230.9
default-information originate
poap transit

---------------------

Primary Leaf Nexus 9372

vlan 1,600,652,680
vlan 600
name DMZ
vlan 652
name Managment
vlan 680
name Inside

vrf context management
ip route 0.0.0.0/0 172.16.52.1
vrf context peer-keepalive
vpc domain 101
role priority 1
peer-keepalive destination 10.200.50.6 source 10.200.50.5 vrf peer-keepalive

interface Vlan1

interface Vlan652
no shutdown

interface port-channel101
switchport mode trunk
switchport trunk allowed vlan 600,680
spanning-tree port type network
vpc peer-link

interface port-channel102
switchport mode trunk
switchport trunk allowed vlan 600,680
vpc 102

sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 99
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po99 up 600,680

vPC status
----------------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
------ ----------- ------ ----------- -------------------------- -----------
102 Po102 up success success 600,680

interface Ethernet1/16
description HOST <<<<<<<<<<<<<<<<<<<<<<<<<<<< This is the host that we cant reach<<<<<<
switchport access vlan 680

interface Ethernet1/17
description SERVER1
switchport mode trunk
switchport trunk allowed vlan 600,680

interface Ethernet1/46
description keep-alive peer-link to ALNSWI04
no switchport
vrf member peer-keepalive
ip address 10.200.50.5/30
no shutdown

interface Ethernet1/47
description vPC Peer Link 1.47
switchport mode trunk
switchport trunk allowed vlan 600,680
channel-group 101 mode active

interface Ethernet1/48
description vPC Peer Link 2.48
switchport mode trunk
switchport trunk allowed vlan 600,680
channel-group 101 mode active

interface Ethernet1/49
description vPC Link 1.49 to Nexus 6004 PRI
switchport mode trunk
switchport trunk allowed vlan 600,680
channel-group 102 mode active

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53
description vPC Link 1.53 to Nexus 6004 SEC
switchport mode trunk
switchport trunk allowed vlan 600,680
channel-group 102 mode active

interface Ethernet1/54

interface mgmt0
vrf member management
ip address 172.16.52.5/23
line console
line vty
boot nxos bootflash:/n9000-dk9.6.1.2.I3.1.bin

sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 101
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po101 up 600,680

vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- ------ ------------
102 Po102 up success success 600,680

----------------------------------------------------------------------------------------------------------------

So I confused on the ping status from the Leaf switch when it says "no route to host"

Any help is appreciated

Thank you

Madhukrishnan Gopinathan Nair · ‎03-12-2015

Hi Kemal,

I have not looked at all config, however a quick thought..If the switch is pure L2, you may not be able to ping it unless you have an SVI for that vlan present in it.

Thanks,

Madhu.

Kemal Zuko · ‎03-12-2015

Hi Madhu,

Are you saying that I may need the SVI created and configured for an IP address?

That SVI already lives upstream on the core Nexus 6004. I am not 100% sure how this would work.

Madhukrishnan Gopinathan Nair · ‎03-12-2015

Dear Kemal,

I am not an expert on n9k, but what is the issue here?

are you able to ping the host in vlan 680 from n6k ? What i was saying you may not be able to ping the host from L2 only switch since it does not have any route to it.

Thanks,

M

glen.grant · ‎03-13-2015

If the ASA is the gateway for 172.16.52.3 then the ASA would have to have a connection to the n6k and statements pointing to everything that is on that N6k if you need to get to those networks. When pinging from the 9k then you have to specify which vrf you are pinging which in your case looks to be the management vrf . On the nexus cisco plays loose with the ip route statements while its a route statement it is basically just a default gateway statement for a L2 switch . It works the same way on the L2 N5k's that we use. In our case though we chose to use a SVI and an address that is allowed down the VPC's .

Kemal Zuko · ‎03-13-2015

Yes,

The ASA is the gateway for all devices that have a MGMT IP. and correct it is directly uplinked to the n6k which are running EIGRP between the two.

Jon Marshall · ‎03-13-2015

Kemal

Is 172.16.52.1 the ASA we have been talking about in the other thread ?

And is it vlan 652 that is used for this IP subnet ?

And finally are you using the same link to the ASA that is also used for vlans 450 and 451 ?

If so you need to allow vlan 652 on that trunk link because currently it isn't.

Jon

Kemal Zuko · ‎03-13-2015

Jon,

Yes that is the same setup as we were discussing in the vPC thread.

Yes, it is vlan 652 that is being used for the ip subnet.

Yes, Using the same link to the ASA that is also using the transit vlans 450 and 451

Jon Marshall · ‎03-13-2015

Okay but you are not allowing that vlan on the trunk link.

But as per my last post you can't because it is a vPC vlan.

So you need to -

1) create SVIs and configure HSRP for vlan 652

2) create a new non vPC vlan together with the SVIs and allow it on the dedicated trunk link between the switches and on the link connecting to the ASA

3) place the new vlan and vlan 652 into their own VRF

Jon

Kemal Zuko · ‎03-13-2015

Jon,

So what you are saying is, move the MGMT connection from MGMT port on the Nexus to a SVI?

Vlan 652 is Management vlan

Jon Marshall · ‎03-13-2015

At the moment can you ping 172.16.52.1 from either N6K switch ?

Jon

Kemal Zuko · ‎03-13-2015

Yes I can from all 4 Nexus switches

SWI01# ping 172.16.52.1 vrf MANagement
PING 172.16.52.1 (172.16.52.1): 56 data bytes
64 bytes from 172.16.52.1: icmp_seq=0 ttl=254 time=0.616 ms
64 bytes from 172.16.52.1: icmp_seq=1 ttl=254 time=0.494 ms
64 bytes from 172.16.52.1: icmp_seq=2 ttl=254 time=0.519 ms
64 bytes from 172.16.52.1: icmp_seq=3 ttl=254 time=0.516 ms
64 bytes from 172.16.52.1: icmp_seq=4 ttl=254 time=0.575 ms

--- 172.16.52.1 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 0.494/0.544/0.616 ms

Here is the setup for Management

ASA MGMT Interface 172.16.52.0/23 connects to a L2 Cisco 2960 MGMT switch

From there, there is an SVI 652 configured for 172.16.52.0/23 with the ip default-gateway 172.16.52.1 which is the ASA

From the MGMT switch (cisco 2960) port 1/0/43 goes to the MGMT on the Cisco Nexus 6004 MGMT intrface

interface GigabitEthernet1/0/43
description Uplink to Nexus 9372 PRI
switchport access vlan 652
switchport mode access

On the Nexus 6004 here is the MGMT config

vrf context management
ip route 0.0.0.0/0 172.16.52.1

interface mgmt0
vrf member management
ip address 172.16.52.3/23

Jon Marshall · ‎03-13-2015

Okay, I didn't realise you had a totally separate connection on the ASA for management.

You won't be able to do the ping then as far as I am aware.

The management interface on the ASA will not pass through traffic from one interface to another.

So when you ping from the 9372 the traffic arrives on the management interface but the ASA routing table shows the route for 172.16.8.0/22 via the inside interface.

And it can't pass that traffic through from the management interface to the inside interface.

So it's nothing to do with your switch configurations and there is nothing you can do to make this work, as far as I am aware, because of the way the management interface on the ASA works.

Jon

Kemal Zuko · ‎03-13-2015

Yea, that is what I thought.

Ultimately we could take the Managment interfaces on the n9K's and move them to a regular port, IP the sVI 652 for an IP address, apply the vlan to the interface and I think that might do it.

At this point this is very minor but it's something that has been bothering me.

Jon Marshall · ‎03-13-2015

It may be worth considering.

A lot people don't use the management port for this very reason.

Bear in mind that if you wanted to access your switches remotely then it won't work because your IP would not be from the 172.16.52.x IP subnet so the ASA won't pass that traffic.

You would need to be in the same IP subnet.

You may be better off doing what I suggested earlier as that way all traffic goes via the inside interface of the ASA and then you would be able to access your switch IPs remotely.

You can still control who could get that access via the ASA.

Jon

Host behind vPC not responding