cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1848
Views
25
Helpful
23
Replies

ESXi -> Cisco 3850 -> Upstream Router routing Not Working

hbuchal
Beginner
Beginner

Please see attached diagram.

I currently have "router on the stick" setup and I am moving to SVIs on Cisco 3850 stack. I have moved VLAN100 as a start.  I can ping each of the directly connected devices (i.e. 3850 and 2911 router). I can't seem to ping a VM on vlan 100 from the router and vice versa. Here is what is working what is not working.

Working in both directions

VM (172.16.100.51) <-> GW on SVI (172.16.100.254) 

VM (172.16.100.51) <-> Another SVI (172.16.230.254)

VM (172.16.100.51) <-> L3 Int on 3850 (10.2.2.2)

L3 int on 3850 (10.2.2.2) <-> L3 int on 2911 (10.2.2.1)

SVI on 3850 (172.16.100.254) <-> L3 int on 2911 (10.2.2.1)

Not Working in either direction:

VM (172.16.100.51) <-> L3 interface on 2911 (10.2.2.1)

VM (172.16.100.51) <-> Anything else NOT routed on 3850

I have following routes on 2911 and 3850.

3850:
ip route 0.0.0.0 0.0.0.0 10.2.2.1

2911:

ip route 172.16.100.0 255.255.255.0 10.2.2.2

ip route 172.16.230.0 255.255.255.0 10.2.2.2

So In theory anything coming from 172.16.100.51 not local to 3850 should be forwarded to 10.2.2.1 since it's default route on 3850.

I suspect this to be a licensing issue. I do have IP Base feature set license on 3850 stack.I have verified it using show license and show version commands.

As per this Cisco FAQ, http://www.cisco.com/c/en/us/products/collateral/switches/catalyst-3850-series-switches/qa_c67-722110.html, routing should be working as I don't have more than 16 static routes and I am only using basic L3 routing features.

I am at a loss here. What's going on? Can someone please confirm?

I had bought WS-C3850-24T-S,

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/3e/release_notes/OL3262101.html#pgfId-950711

thinking I would be able to use SVIs and keep all traffic from going to upstream routers as our older switches were only L2.

It looks like an upgrade to IP Services feature set is possible,

https://cisco3850.wordpress.com/2015/04/22/licensing-for-cisco-catalyst-3850-series-switches/.

Do I need to upgrade the image as well or can I just switch the license using the built-in commands described here,

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/3se/system_management/configuration_guide/b_sm_3se_3850_cg/b_sm_3se_3850_cg_chapter_0100.html#concept_83A11E6B66E349A0A9090DBD37F28602

I hope I don't have to reboot the switches as this setup is currently using this stack as core and distribution.

Any help is appreciated.

Thanks

1 Accepted Solution

Accepted Solutions

Reza Sharifi
Hall of Fame Expert Hall of Fame Expert
Hall of Fame Expert

Turning off and on "IP routing" did it?

View solution in original post

23 Replies 23

Reza Sharifi
Hall of Fame Expert Hall of Fame Expert
Hall of Fame Expert

You don't need to upgrade the IOS or the license, as this is very basic setup and should work just fine with IP Base license.  IP services license is needed for advanced features like, VRF, PBR, BGP, etc.. and you are not running any of these protocols. Also, your setup seems to be correct.  Can you post "sh run" from both the 3850 and 2911?

HTH

I have attached both configs. 

I also will be using IP SLA for tracking default static route on 3850 for failing over GWR2. Is it part of PBR on 3850? I won't be using route maps just straight forward ICMP tracking of upstream IPs on the WAN switch. Here is a similar example:

http://www.firewall.cx/cisco-technical-knowledgebase/cisco-routers/813-cisco-router-ipsla-basic.html

Thanks

Reza Sharifi
Hall of Fame Expert Hall of Fame Expert
Hall of Fame Expert

Hi,

Can you turn on "ip routing" on the 3850 and test again?

HTH

Not sure how to enable ip routing on 3850. but it looks like it's enabled by default.

Running the command below only gives me the option to purge routes on an interface

Core#ip routing protocol purge interface

Routing Table

Core#sh ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
+ - replicated route, % - next hop override

Gateway of last resort is 10.2.2.1 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 10.2.2.1
10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 10.2.2.0/24 is directly connected, GigabitEthernet1/0/24
L 10.2.2.2/32 is directly connected, GigabitEthernet1/0/24
172.16.0.0/16 is variably subnetted, 4 subnets, 2 masks
C 172.16.100.0/24 is directly connected, Vlan100
L 172.16.100.254/32 is directly connected, Vlan100
C 172.16.230.0/24 is directly connected, Vlan230
L 172.16.230.254/32 is directly connected, Vlan230

default-router is not listed in running config but shows up here

Core#sh running-config | i rout
default-router 172.16.100.254
ip route 0.0.0.0 0.0.0.0 10.2.2.1

Based on this doc, http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/3-2_0_se/routing/configuration_guide/b_rt_32se_3850_cg/b_rt_32se_3850_cg_chapter_010.html#ID176 

IP routing should be enabled when L3 interface or SVI or L3 PC is created.

I appreciate you prompt help but this issue is mind boggling.

I was thinking about removing all references of 172.16.100.x from GWR1 NAT and IPSec ACLs but having those should not cause issues with local routing.

it's a simple setup. I have done SVIs on older 3750 switches and new Nexus ones without any issues.

Reza Sharifi
Hall of Fame Expert Hall of Fame Expert
Hall of Fame Expert

From the output you provided, I think ip routing is enabled.

The next thing I was going to suggest was to remove the ALC from the router and test.  If it starts working than there is an ACL blocking communication.

HTH

I removed all ACL, NAT, VPN related entries for 172.16.100.0/24 w/o any luck.

GWR1#ping
Protocol [ip]:
Target IP address: 172.16.100.51 (VM)
Repeat count [5]:
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Source address or interface: 10.2.2.1 (2911 Downstream)
Type of service [0]:
Set DF bit in IP header? [no]:
Validate reply data? [no]:
Data pattern [0xABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.100.51, timeout is 2 seconds:
Packet sent with a source address of 10.2.2.1
.....
Success rate is 0 percent (0/5)

I even enabled debugging on 2911 and 3850 but to no avail. nothing prints on console when i ran traceroute or pings to 172.16.100.51 <->10.2.2.1

GWR1#debug ip routing static route 172.16.100.0 255.255.255.0

Core#debug ip routing static route 0.0.0.0 0.0.0.0 

Next I am going to connect a host directly to VLAN 100 access port on 3850 and test it to see if it's a VMware issue and not Cisco. I doubt it.

Reza Sharifi
Hall of Fame Expert Hall of Fame Expert
Hall of Fame Expert

Yes, try it with a laptop as an access port.  Although I don't think this is a VM issue as you are able to ping the gateway (3850) and vice versa, but testing with a laptop would not hurt.

HTH

I just did. please see my reply earlier. we seemed to have stepped on each other while replying :-)

Reza Sharifi
Hall of Fame Expert Hall of Fame Expert
Hall of Fame Expert

Yes, we did.

From the switch can you ping 10.2.2.1 and from the router 10.2.2.2 (directly connected)?

Yes. Both simple ping and extended from directly correctly interfaces' IP addresses.

GWR1#ping 10.2.2.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.2.2.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/4 ms

Core#ping 10.2.2.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.2.2.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/10 ms

Found a similar thread.

https://supportforums.cisco.com/discussion/12630316/local-subnet-not-able-see-remote-networks-passed-3850

They had to enable IP Services

----------------------------------------------------------------

Jon, sorry for the delay!!

what i found was the new 3850's come from the factory with a "universal" IOS and

what determines which services are these lines when you do sh ver:

License Level: Ipservices
License Type: Permanent
Next reload license Level: Ipservices

 

once i ran license right-to-use activate ipservices all acceptEULA

IPservices was enabled.

 

Thanks for your asistance!!

------------------------------------------------------------------------------

Does it require us a reboot of the stack?

Reza Sharifi
Hall of Fame Expert Hall of Fame Expert
Hall of Fame Expert

Usually for the new license to take effect you need to reboot the switch.  So, I would say rebooting the stack should do it. I am still puzzled why you need IP services for this simple static routing but glad to know its working.

HTH 

I don't know if it will work. I am still puzzled why it won't work with Ip base.

I am going to have to schedule a reboot of the stack as this is production setup.

I will change the license and reboot. I will let you know if it worked.

From this thread, it looks like I can enable IP Services w/o first buying a license.

https://supportforums.cisco.com/discussion/12760546/rtu-license-3850-ip-base-ip-service

If everything works, I will buy a license.

Thanks

IT DID NOT WORK.....

Core#sh license right-to-use
 Slot#  License name   Type     Count   Period left
----------------------------------------------------------
 1      ipservices   permanent     N/A   Lifetime

License Level on Reboot: ipservices


 Slot#  License name   Type     Count   Period left
----------------------------------------------------------
 2      ipservices   permanent     N/A   Lifetime

License Level on Reboot: ipservices

3850 is not letting any transient routing.

- I disabled proxy arp

- I created 2 SVIs and tried to ping host addresses in both direction;
   172.16.230.51 <--> 172.16.100.51 (IT Doesn't WORK)

What is going on. This is crazy...I have never seen anything like this. Is there any low level command I can use to see what the hell is the routing engine doing. This is bizarre.

I have gone through every single document here,

http://www.cisco.com/c/en/us/support/switches/catalyst-3850-series-switches/products-installation-and-configuration-guides-list.html

but nothing to seem to indicate what's making 3850 block routing between hosts and to upstream devices.

Please help. I want to do L3 routing on 3850 and between SVIs.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Recognize Your Peers