C3750 Routing issue

[Phil Curwood]

Hi Guys,

We have a core stack of two C3750 switches performing inter VLAN routing within one of our DC's.
We only use static routes on the devices, no routing protocol instances exist and I'm having a strange problem.

VLAN10 SVI ip address 172.18.10.254
VLAN30 SVI ip address 172.18.30.254

sh ip route

Gateway of last resort is 172.18.200.1 to network 0.0.0.0

     172.18.0.0/16 is variably subnetted, 5 subnets, 2 masks
C       172.18.200.0/29 is directly connected, Vlan200
C       172.18.30.0/24 is directly connected, Vlan30
C       172.18.31.0/24 is directly connected, Vlan31
C       172.18.12.0/24 is directly connected, Vlan12
C       172.18.10.0/24 is directly connected, Vlan10
S    172.20.0.0/16 [1/0] via 10.99.100.3
S    172.22.0.0/16 [1/0] via 10.99.100.20
     172.30.0.0/24 is subnetted, 1 subnets
S       172.30.10.0 [1/0] via 10.99.100.3
S    192.168.200.0/24 [1/0] via 10.99.100.40
C    192.168.140.0/24 is directly connected, Vlan140
     10.0.0.0/24 is subnetted, 2 subnets
C       10.10.10.0 is directly connected, Vlan1
C       10.99.100.0 is directly connected, Vlan100
S    192.168.6.0/24 [1/0] via 10.99.100.40
S    192.168.100.0/24 [1/0] via 10.99.100.40
S    192.168.3.0/24 [1/0] via 10.99.100.40
S*   0.0.0.0/0 [1/0] via 172.18.200.1

When I run a tracert to 192.168.201.245 from a machine on VLAN10 the response is as below (which is as I'd expect):

C:\Users\administrator.ONDEMANDIT>tracert -d 192.168.201.245

Tracing route to 192.168.201.245 over a maximum of 30 hops

  1     2 ms     3 ms     2 ms  172.18.10.254
  2     1 ms    <1 ms    <1 ms  83.138.56.1
  3     *        *     ^C

When I run a tracert to 192.168.201.245 from a machine on VLAN30 the response is as below (which is wrong):

C:\Users\administrator.ONDEMANDIT>tracert -d 192.168.201.245

Tracing route to 192.168.201.245 over a maximum of 30 hops

  1     2 ms     2 ms     5 ms  172.18.30.254
  2     8 ms     7 ms     7 ms  10.10.20.12
  3  ^C

The version of IOS is C3750-IPBASEK9-M, Version 12.2(55)SE9

There is no PBR in place, so can anyone give me any pointers on this as I'm at a loss as to how this is routing differently.

Thanks.

Robbie.

[Mark Malone]

It looks like you have no specific statics set in that routing table for vlan 10 or 30 just a default route pointing to 172.18.200.1 , if that's correct that would suggest this device is not doing the routing the 172.18.200.1 is for those vlans , so I would be looking at that device to see what's in place there regarding routing of those vlan subnets

[Phil Curwood]

Thanks for your responses guys.

172.18.200.1 is indeed an ASA and 83.138.56.1 is the IP of the default route to our ISP so this is the route I would expect and indeed want the traffic to take.
We have a BT WAN link connected to the core switches on VLAN 100 and the IP address range 10.10.20.0/24 sits beyond this on a clients site so the traffic somehow is being routed along this WAN link.

Hope that helps shed some more light on this.

[Jon Marshall]

Presumably 192.168.201.0/24 does not exist anywhere in your network ?

Can you do the traceroutes again and post a "sh ip redirects" from your switch.

Jon

[Phil Curwood]

yes - I am trying to get a site to site VPN working from the ASA for the 192.168.201.0 /24 network but the traffic isn't hitting the ASA.

The cache on the switch is empty:

#sh ip redirects
Default gateway is 172.18.10.254

Host               Gateway           Last Use    Total Uses  Interface
ICMP redirect cache is empty

[Jon Marshall]

Okay this is really puzzling.

Does 192.168.201.0/24 exist on your network anywhere ?

Jon

[Phil Curwood]

No, absolutely not - that's whats completely confusing about it all.

192.168.201.0 /24 is one of our clients remote sites. We did used to route traffic to this network over our WAN in to our clients head office, then they routed this over their own WAN to the remote site.
The remote site has moved and they're done away with their WAN link and just have an internet connection now hence me creating a site to site VPN.
So, our core switches used to have a route on them sending traffic to 192.168.201.0 /24 network via 10.99.100.40 which is our WAN switch on the clients head office site.
But now I've removed that route and fully expected the traffic to use the default route from the core switch to the ASA and on to the internet.
So traffic sourced from VLAN10 takes this route fine, traffic sourced from VLAN 30 doesn't!

[Jon Marshall]

Something in the routing table is confusing.

You said you had a BT WAN link connected to your switch on vlan 100 (10.99.100.0/24) and the 10.10.20.0/24 network sat beyond that.

And there is no entry in the routing table for the 10.10.20.0/24 subnet.

But your traceroute shows 10.10.20.12 as a next hop IP after the local SVI IP address.

What is the BT device with an IP of 10.99.100.x ?

Jon

[Phil Curwood]

There isn't any BT device as such, it's a wires only Etherflow circuit so it's a Layer 2 link but we're routing across it using the 10.99.100.0/24 network as the 'routing' network.
We have circa 8 tails in to the Etherflow service from BT with a mixture of Cisco 887 routers and Cisco C3750 switches being the terminating devices. We just have static routes on all the devices.

[Jon Marshall · Accepted solution]

So if there is no firewall in the path then you should see the next hop IP in your traceroute.

Which means somehow your 3750 is sending a packet to 10.10.20.12 when it has no path to it other than the default route.

This client with the 192.168.201.x network that you are connecting to with a VPN.

Have you created the tunnel on the ASA yet ?

Jon

[Phil Curwood]

Got it!
We have a VPN failover (if the WAN went down) so there is (was) VPN config on the ASA already which was set to tunnel the traffic destined to 192.168.201.0 over the VPN to the head office site, then this would route the traffic over their WAN accordingly!
I've just removed this element of the config (as it's no longer needed) and the traceroute is now showing as taking the correct path. It was indeed taking the correct path anyway (i.e. being routed via the ASA) but was being sent to the head office network over the VPN but this appeared as if it was going over our WAN link!

Jon - thank you very much for your assistance.

[Jon Marshall]

I was wondering if it was going via the ASA but that there was a tunnel for it but I still couldn't work out how it ended up where it did :-)

Glad you got it sorted.

Jon

[Jon Marshall]

Just to add to Mark's reply.

Going off your routing table both those traceroutes look wrong because the next hop IP for both should be 172.18.200.1.

Is that device a firewall ?

Jon