Strange Issue with VLAN routing

[martin.daley@eventura.com]

I have a very strange VLAN routing issue and struggling to get my head around what the issue could be.

Topology as follows.

Cisco Nexus Core and a series of Cisco Catalyst Edge/Access Layer switches.

The access layer switches are connected via trunk port channel interfaces to the nexus core.

Connected to the Nexus core is my Firewall. There are a series of interfaces directly connected for various different VLANS.

I have 2 example VLANS vlan 222 and 224.

The firewall interface on VLAN222 is on 192.168.222.1 and is a /24 network

The firewall interface on VLAN224 is on 192.168.224.1 and is a /23 network.

I am using static routing and have configured a vlan interface IP on the nexus of 192.168.222.2 and 192.168.224.2 respectively for each vlan. The nexus can ping both firewall interfaces no problem and also the other way round, so i am confident this bit is working correcttly

I have also configured a VLAN interface on the access layer switches on the respective vlans. example 192.168.222.5 for vlan 222 and 192.168.224.5 for vlan 224.

As mentioned I have trunked both vlans to the Catalyst access layer switches and from the nexus i am able to ping 192.168.222.5 no problem. If I try to to the same thing for 192.168.224.4 i get no response

Both status and protocol are up/up on the access layer switch.

If i try to locally ping the VLAN 222 interface from the access layer switch itself it responds to pings. If I try to do the same for the VLAN 224 interface it does not respond.

below ip route snip shows both vlans are directly connected.

C 192.168.206.64 is directly connected, Vlan206
O E2 192.168.70.0/24 [110/20] via 10.10.99.9, 6d20h, Vlan444
O E2 192.168.184.0/24 [110/20] via 10.10.99.9, 6d20h, Vlan444
C 192.168.222.0/24 is directly connected, Vlan222
192.168.207.0/26 is subnetted, 2 subnets
O E2 192.168.207.128 [110/20] via 10.10.99.9, 6d20h, Vlan444
O E2 192.168.207.0 [110/20] via 10.10.99.9, 6d20h, Vlan444
O E2 192.168.71.0/24 [110/20] via 10.10.99.9, 6d20h, Vlan444
S* 0.0.0.0/0 [1/0] via 192.168.0.235
C 192.168.224.0/23 is directly connected, Vlan224

So i guess my issue is that for some reason I can route the 222 network no problem but for some reason unbeknown to me, this 224 vlan just won't play ball. This is obviously preventing devices connected to the VLAN from accessing the default gateway and ultimately the internet.

Interface IP-Address OK? Method Status Protocol
Vlan1 unassigned YES NVRAM administratively down down
Vlan2 unassigned YES NVRAM up down
Vlan11 unassigned YES NVRAM up down
Vlan12 unassigned YES NVRAM up down
Vlan13 unassigned YES NVRAM up down
Vlan30 unassigned YES NVRAM up down
Vlan205 unassigned YES NVRAM up down
Vlan206 192.168.206.65 YES NVRAM up up
Vlan207 192.168.207.65 YES NVRAM up down
Vlan208 192.168.208.65 YES NVRAM up down
Vlan222 192.168.222.5 YES manual up up
Vlan224 192.168.224.4 YES manual up up
Vlan251 192.168.251.65 YES NVRAM up up
Vlan444 10.10.99.10 YES NVRAM up up

From comparing the configs, both vlans look to be configured identically so I can't for the lift of me work out why one would work fine but the other doesn't. The fact one locally pings and then the other one doesn't is also suspect i think?

Any help/pointers would be much appreciated!

[MHM Cisco World]

hard to answer but I have two idea here, 
1- are ASA is transparent mode ? if yes then the VLAN is change ID when pass through ASA 
2- If VLAN is not allow in trunk in away between the ping source and destination then frame can drop.

[martin.daley@eventura.com]

hi, thanks for the response, to confirm, there is no firewalling between the Nexus and the Catalyst Access Layer, they are directly connected. With regards to firewall, this is on the perimeter and is a Fortinet appliance. The issue I seem to have is some bizarre anomaly on the access layer catalysts that isnt playing ball. its very odd how it works for one VLAN but not the other.

[MHM Cisco World]

but still second point I mention before, do you check the VLAN allow in trunk ?

[martin.daley@eventura.com]

Nexus related port config as follows

interface Vlan224
no shutdown
ip address 192.168.224.2/23

interface port-channel44
description *** LINK TO REC1-3560-01 ***
switchport mode trunk
switchport trunk allowed vlan 1,222,224,229,444,701
speed 1000
bandwidth 1000000

interface Ethernet1/17
description REC2-3560-01
switchport mode trunk
switchport trunk allowed vlan 1,222,224,229,444,701
speed 1000
channel-group 44 mode active

sho ip route

192.168.222.0/24, ubest/mbest: 1/0, attached
*via 192.168.222.2, Vlan222, [0/0], 11w0d, direct
192.168.222.2/32, ubest/mbest: 1/0, attached
*via 192.168.222.2, Vlan222, [0/0], 11w0d, local
192.168.224.0/23, ubest/mbest: 1/0, attached
*via 192.168.224.2, Vlan224, [0/0], 4w5d, direct
192.168.224.0/24, ubest/mbest: 1/0
*via 10.10.99.45, Vlan436, [110/20], 2w5d, ospf-1, type-2
192.168.224.2/32, ubest/mbest: 1/0, attached
*via 192.168.224.2, Vlan224, [0/0], 4w5d, local

It looks like its learning an incorrect route via ospf but surely a directly connected interface should override any routing protocol as it will be metric 0?

Catalyst Access Layer Switch

---------------------------------

interface GigabitEthernet0/1
description *** Gi 0/1 UPLINK TO WEST-NEXUS-01 ETH 1/17 ***
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,222,224,229,444,701
switchport mode trunk
channel-protocol lacp
channel-group 44 mode passive

interface Port-channel44
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,222,224,229,444,701
switchport mode trunk

C 192.168.222.0/24 is directly connected, Vlan222
192.168.207.0/26 is subnetted, 2 subnets
O E2 192.168.207.128 [110/20] via 10.10.99.9, 6d22h, Vlan444
O E2 192.168.207.0 [110/20] via 10.10.99.9, 6d22h, Vlan444
O E2 192.168.71.0/24 [110/20] via 10.10.99.9, 6d22h, Vlan444
S* 0.0.0.0/0 [1/0] via 192.168.0.235
C 192.168.224.0/23 is directly connected, Vlan224

REC2-3560-01#ping 192.168.224.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.224.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/4/9 ms
REC2-3560-01#ping 192.168.224.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.224.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

[MHM Cisco World]

sorry but many VLAN and two layers SW and FW, 
can you  draw the topology ?

[balaji.bandi]

Just Looking at the information. is the nexus is vPC or Standalone Nexus core ?

as asked you need to make a small diagram

For testing shutdown one of the Link on the port-channel from nexus to access switch and test test the ping.