Cisco 3560 interVLAN routing, with DHCP, and ip sla

[ianbrown553803657]

Good day all,

   I have been searching for some time now on how to configure this. The situation is; I have cable internet as a primary and I have added recently 4G LTE as a back up solution (when I loose power, I loose cable internet). I have a Cisco 1921 router and a Cisco 3560 Switch. The 4G LTE modem is also a router. 

What I am looking to do is use the 1921 for Nat'ing the cable internet, since I get one public IP from them. Then use the 3560 for IP SLA, which i had working until i wiped the switch out of frustration. I also want to use the switch for multiple VLAN's, four to be exact all with their own DHCP pool.

Here's the hang up, I can't get the VLANs to route to the internet. I have tried different ip routes, pointing the default router to the default gateway.

Am i over complicating this by having the 1921 between the switch and cable modem? Do i need to turn off nat on the 4G LTE modem? Do i need to configure router osfp or eigrp on the switch and router to make this work?

I don't have a config to share, as i wiped the switch (like i said earlier). I am reloading the config and as soon as it's up i will share it.

Thank you all in advance for your help.

[Jon Marshall]

Difficult to say without configs but things to check - 

1) ip routing must be enabled on the switch and you need a default route (not default gateway) on the switch pointing to the LAN interface of the router

2) the router also needs to know how to reach the IP subnets on the switch so for each subnet you need to add a route to the router pointing to whichever IP on the switch is in the same subnet as the LAN interface on the router.

3) you will need to setup NAT for all the IP subnets on the router so they can go out to the internet. 

Quick test, if from a client in one of the vlans you can ping the LAN interface of the router your routing is fine, concentrate on NAT but if you can't ping the LAN interface you need to do 1) and 2) from above. 

Get that connectivity working before you start worrying about the IP SLA part. 

Jon

[paul driver]

Hello
Sound like a router on a stick setup would be applicable, Even if your cable modem is already performing NAT you can still use nat yourself for your lan clients.

Pease see attached file for a possibe example:

[ianbrown553803657]

Wow, thank you guys for the timely replies. I really appreciate the help. 

Would using a router on a stick config be a waste of a 3560, or would it be overkill? I am leaning that way.

This is the config that is on the switch. The intervlan works once a device is connected to that vlan. I can ping from the switch to the internet, but can not ping from any vlan to the internet. I know it's something simple i am missing. Should vlan 1 be active?

Switch>enable
Switch#show run
Building configuration...

Current configuration : 5636 bytes
!
version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Switch
!
boot-start-marker
boot-end-marker
!
!
!
!
no aaa new-model
system mtu routing 1500
ip routing
ip dhcp excluded-address 172.24.16.1 172.24.16.9
ip dhcp excluded-address 172.24.24.1 172.24.24.9
ip dhcp excluded-address 192.168.40.1 192.168.40.2
ip dhcp excluded-address 10.10.1.1
ip dhcp excluded-address 10.10.1.9
!
ip dhcp pool The_Compound
network 172.24.16.0 255.255.248.0
default-router 172.24.16.1
dns-server 8.8.8.8
!
ip dhcp pool The_Compound_IoT
network 172.24.24.0 255.255.248.0
default-router 172.24.24.1
dns-server 8.8.8.8
!
ip dhcp pool Cameras
network 192.168.40.0 255.255.255.240
default-router 192.168.40.1
dns-server 8.8.8.8
!
ip dhcp pool Work
network 10.10.1.0 255.255.255.248
default-router 10.10.1.1
dns-server 8.8.8.8
!
ip dhcp pool Cisco_Phone
network 10.10.1.8 255.255.255.248
default-router 10.10.1.9
dns-server 8.8.8.8
!
!
!
track 1 ip sla 1 reachability
!
!
crypto pki trustpoint TP-self-signed-4071831808
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-4071831808
revocation-check none
rsakeypair TP-self-signed-4071831808
!
!
crypto pki certificate chain TP-self-signed-4071831808
certificate self-signed 01
3082023F 308201A8 A0030201 02020101 300D0609 2A864886 F70D0101 04050030
31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 34303731 38333138 3038301E 170D3933 30333031 30303031
32365A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649
4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D34 30373138
33313830 3830819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281
8100F03C BD5E54C8 9C798E59 3382A82C 4A016B9C 87188263 F0B355BD A6494E76
7A73BE98 1D3617A1 F3E13FAC 1965FED0 7B03AED6 624C45BA 5543C733 1C521C60
577A5FD5 2128FAE1 1B4916F7 8A9F6D29 DA75624B 2D8ECBA1 8D604719 25F3EBF0
3028A542 56983BC7 72AC3897 0413DD49 79337370 174A64AF 57523C5E 07E98270
C8610203 010001A3 67306530 0F060355 1D130101 FF040530 030101FF 30120603
551D1104 0B300982 07537769 7463682E 301F0603 551D2304 18301680 14ED36DD
C82FEAE4 E458AFE0 D758978A E207C1C9 42301D06 03551D0E 04160414 ED36DDC8
2FEAE4E4 58AFE0D7 58978AE2 07C1C942 300D0609 2A864886 F70D0101 04050003
81810080 339E262A A6CACC9E E5FC1C08 9D03FEDC D17DA4DD 4D9392A6 08ABE78C
B584064D F8EE5321 DD211514 BDAB8B6E 1AA06A2E 63FC055A B0F17B19 498F4664
DA4F923E B0EFE4C8 6B051E33 51AF3536 060D0CF2 43B84B92 D8418CDF 93B9AA55
78E7916F E59FFFF6 D1B419D0 75CAA3F3 64EA5FF0 2053B65E BBD65CFB 22F74B2D C94361
quit
spanning-tree mode pvst
spanning-tree extend system-id
!
!
!
!
vlan internal allocation policy ascending
!
!
!
interface FastEthernet0
no ip address
no ip route-cache cef
no ip route-cache
no ip mroute-cache
shutdown
!
interface GigabitEthernet0/1
description Link to Primary ISP
no switchport
ip address dhcp
!
interface GigabitEthernet0/2
description Link to Backup ISP
no switchport
ip address dhcp
!
interface GigabitEthernet0/3
switchport trunk allowed vlan 10,20
!
interface GigabitEthernet0/4
!
interface GigabitEthernet0/5
!
interface GigabitEthernet0/6
!
interface GigabitEthernet0/7
!
interface GigabitEthernet0/8
!
interface GigabitEthernet0/9
!
interface GigabitEthernet0/10
!
interface GigabitEthernet0/11
!
interface GigabitEthernet0/12
!
interface GigabitEthernet0/13
!
interface GigabitEthernet0/14
!
interface GigabitEthernet0/15
!
interface GigabitEthernet0/16
!
interface GigabitEthernet0/17
!
interface GigabitEthernet0/18
!
interface GigabitEthernet0/19
!
interface GigabitEthernet0/20
!
interface GigabitEthernet0/21
!
interface GigabitEthernet0/22
!
interface GigabitEthernet0/23
switchport access vlan 50
switchport mode access
!
interface GigabitEthernet0/24
!
interface GigabitEthernet1/1
!
interface GigabitEthernet1/2
!
interface GigabitEthernet1/3
!
interface GigabitEthernet1/4
!
interface TenGigabitEthernet1/1
!
interface TenGigabitEthernet1/2
!
interface Vlan1
no ip address
shutdown
!
interface Vlan10
description The_Compound
ip address 172.24.16.1 255.255.248.0
!
interface Vlan20
description The_Compound_IoT
ip address 172.24.24.1 255.255.248.0
!
interface Vlan40
description Cameras
ip address 192.168.40.1 255.255.255.240
!
interface Vlan50
description Work
ip address 10.10.1.1 255.255.255.248
!
interface Vlan99
ip address 192.168.99.2 255.255.255.0
!
interface Vlan150
description VoiP
ip address 10.10.1.9 255.255.255.248
!
router ospf 10
log-adjacency-changes
network 10.10.1.0 0.0.0.7 area 0
network 10.10.1.8 0.0.0.7 area 0
network 172.24.16.0 0.0.7.255 area 0
network 172.24.24.0 0.0.7.255 area 0
network 192.168.40.0 0.0.0.15 area 0
!
ip default-gateway 192.168.20.1
ip classless
ip route 0.0.0.0 0.0.0.0 192.168.20.1 track 1
ip route 0.0.0.0 0.0.0.0 192.168.0.1 100
ip route 10.10.1.0 255.255.255.248 192.168.20.1
ip route 10.10.1.8 255.255.255.248 192.168.99.1
ip route 172.24.16.0 255.255.248.0 192.168.99.1
ip route 172.24.24.0 255.255.248.0 192.168.99.1
ip route 192.168.20.0 255.255.255.0 0.0.0.0
ip route 192.168.40.0 255.255.255.240 192.168.99.1
ip route 192.168.99.0 255.255.255.0 192.168.20.1
ip route 192.168.99.0 255.255.255.0 192.168.0.1 100
ip http server
ip http secure-server
!
ip access-list standard no_ip
permit any
!
ip sla 1
icmp-echo 98.2.224.1
timeout 1000
threshold 1500
frequency 3
ip sla schedule 1 life forever start-time now
!
!
line con 0
line vty 0 4
login
line vty 5 15
login
!
end

Switch#

[Richard Burts]

I am perhaps a bit confused about the topology of this network. There is a 1921 router and a 3560 switch. The 1921 is to do nat and the switch for IP SLA. I assumed that logically the switch would connect to the router and the router would connect to the ISPs. But the configuration posted is pretty clear that the switch connects to the ISPs. Getting that to work would be pretty complicated and I would suggest that the best thing to do would be to change the topology.

In looking at the posted configuration I see several issues.

- first and most important it appears that end stations are connected to the switch and the ISPs are connected to the switch. But there is no address translation in the configuration.

- also there are 2 default routes configured (a primary and a floating static)

ip route 0.0.0.0 0.0.0.0 192.168.20.1 track 1
ip route 0.0.0.0 0.0.0.0 192.168.0.1 100

But where are these next hop addresses? Logically they are the addresses in subnets of G0/1 and G0/2, which are the ISPs. If that is the case then certainly the biggest issue is that no traffic is forwarded to the 1921 for address translation.

[ianbrown553803657]

I think your confusion was caused by some ambiguity in the description of my topology. 

My primary ISP is the cable company. The cable modem is connected to the 1921. The CM issues a public ip (no nat) then I have the 1921 configured for nat. The 3560 does not do nat. The secondary ISP is an LTE wireless modem that has nat built in. This is why both ISPs are connected to the 3560. The IP SLA works great on the switch. I just don't have traffic passing from the vlans to the ISP ports. 

Am I over complicating this by trying to use one device to handle the majority of the routing and the IP SLA? Should I be using a router on a stick config, or is what I am doing reasonable and I am missing something small?

Thank you all for your help and suggestions so far.

[Richard Burts]

There is certainly some ambiguity in the description of your topology. So far the only detailed information we have is the posted config of the switch. So I would ask for some clarification of these points:

- what is G0/1 physically connected to? The comment identifies it as " Link to Primary ISP" Is it to the ISP or is it to the 1921?

- what is G0/2 physically connected to? The comment identifies it as " Link to Backup ISP" Is it to the backup ISP or is it to the 1921?

- your primary static default route specifies 192.168.20.1 as the next hop. What device is this part of?

- your floating static default route specifies 192.168.0.1 as the next hop. What device is this part of?

- at one point you tell us that "The cable modem is connected to the 1921"  

and at another point you tell us that "This is why both ISPs are connected to the 3560." How can both of these statements be correct?

- you tell us that "The secondary ISP is an LTE wireless modem that has nat built in." But you have not told us where the secondary ISP is physically connected. Can you clarify this?

[ianbrown553803657]

"Link to Primary ISP" G0/1 is connected to the 1921 which is connected to the cable modem. The cable modem does not have NAT, it issues a public IP. The 1921 is configured to issue a private IP (192.168.20.1)

"Link to Backup ISP" G0/2 is connected to a Netgear Orbi LTE Modem with built in router and Wifi. That device also has two ethernet ports, which connects to the 3560. That device issues a private IP (192.168.0.1)

The static route 192.168.20.1 is the default router for the Primary ISP, through the 1921 and out to the internet through the cable modem

The floating static route 192.168.0.1 is the default router on the Backup ISP, through the Netgear Orbi. That route also has a longer administrative distance assigned for the failover config (IP SLA).

The cable modem is connected to the 1921 strictly for the purposes of NAT, since the 3560 doesn't do NAT. 

Since failover is configured on the 3560, port G0/1 and G0/2 are not set as switchports, the are routed ports.

Hope this clarifies.

Thank you

[Richard Burts]

Thanks for the clarification. I had been under the impression that since the original post describes the 4G LTE as a back up that the original network (through the 1921 to the cable Internet) was working. But as I read through the discussion I believe that is not the case (I can ping from the switch to the internet, but can not ping from any vlan to the internet.) In this case we need to look not only at the 3560 but also at the 1921. Would you post the config of the 1921?

As I look into the switch config I do have some comments and questions:

- the switch config has a single port assigned to vlan 50 while all other switch ports are in vlan 1. What are you testing with? What port in what vlan?

- if G0/1connects to the 1921 why does it use dhcp? It seems to me that this adds complexity to the configuration and what is the benefit of doing this. A simple IP address on each side would be much more simple, and quite effective.

- I see that OSPF is configured with network statements for 5 of the subnets configured on the switch. But I do not see a network statement for the subnet connecting the switch to the 1921. I had assumed that you were running OSPF between the switch and the router. But that does not seem to be the case. What else in your network is running OSPF?

- what is the purpose of this static route

ip route 10.10.1.0 255.255.255.248 192.168.20.1

this subnet is locally connected. So why have a static route for it with the next hop being the 1921?

- what is the purpose of these static routes

ip route 10.10.1.8 255.255.255.248 192.168.99.1
ip route 172.24.16.0 255.255.248.0 192.168.99.1
ip route 172.24.24.0 255.255.248.0 192.168.99.1

ip route 192.168.40.0 255.255.255.240 192.168.99.1

These subnets are all locally connected. So why have static routes for them, and what device is at the next hop address in vlan 99?

- this static route is quite odd

ip route 192.168.20.0 255.255.255.0 0.0.0.0

The subnet appears to be the subnet connecting to the 1921. So why have a static route for it? And the next hop of 0.0.0.0 essentially means it can not be forwarded.

- why have these floating static routes for a subnet that is locally connected

ip route 192.168.99.0 255.255.255.0 192.168.20.1
ip route 192.168.99.0 255.255.255.0 192.168.0.1 100

- you have 6 subnets configured on the switch. In the event that you fail over to the Netgear is it configured to perform address translation for all 6 subnets? Does the Netgear have routes configured for those 6 subnets?

- I dont know that it relates to the issues about access to Internet but what does this trunk connect to?

interface GigabitEthernet0/3
switchport trunk allowed vlan 10,20

[ianbrown553803657]

This is all becoming clearer and clearer. If you couldn't tell I am still learning IOS. I think I have most of the basics, it's just combining them to make it all work the way I know it can.

To answer your questions

I have one switchport assigned to VLAN 50 and all the other assigned to VLAN 1 because I hadn't gotten that far with assigning the ports. I wanted to test the config and I picked VLAN 50 on G0/23 for testing. Once a device is connected the port comes up and then VLAN 50 comes up too. I did assign G0/4 (not in the attached config) to VLAN 30 and I can ping between VLANs. I have VLAN 1 shut in the config.

Nor sure why I set up DHCP on the 1921. I see your point a simple static address between the 3560 and the 1921 would work, and save some time.

I am running OSPF for intervaln routing on the 3560 only. Looking through the documents I have, my understanding was that I had to broadcast the VLANs.

The other static routes I added to see if that would pass the data from each VLAN DHCP. At the time i added them I knew they wouldn't work, but I wanted to test it anyway.

You ask a very valid question about the Netgear that I hadn't thought of. That is not conifgured for the 6 subnets, however if all the traffic is being routed to the routed ports G0/1 and G0/2 wouldn't it pass on the one ip address ultimately being on VLAN 1? Or did I just describe NAT? I think I just found the issue.

[Richard Burts]

I am glad that we are making progress. I have several points:

- first and most important to enable inter vlan routing on the switch all you need to do is to put "ip routing" into the configuration. You only need OSPF if you want to advertise your subnets to some other device or if you want to learn other subnets from some other device. So remove the configuration of OSPF. 

- next is that you do not need static routes for connected subnets. So remove all the static routes except for the static default route and the floating static default route.

- you can certainly forward traffic from your test machine in vlan 50 to Netgear over the routed link with its IP address. The issue is when Netgear wants to send a response to your test machine how will it know how to reach it? That is why Netgear needs routes for your subnets. 

- a related but separate issue is the need for address translation on the Netgear. When you fail over to the Netgear you will be sending traffic with private IP addressing. To forward it to the Internet Netgear will need to perform address translation for each of your subnets. 

[ianbrown553803657]

Wouldn't i want the traffic to default to VLAN 1 to leave the switch? That way the 2 routers (The 1921 and the Netgear) don't really need to know what VLAN it came from and is returning to.

Basically I want the switch to do some packet prioritization, so the Cisco IP phone on Voice VLAN to take priority (with out doing a bunch of QOS). Then send all the traffic to the internet using either of the available paths (Primary or Backup ISP).

Does that make sense?

[Richard Burts]

Actually you do not want to use vlan 1 for traffic to leave the switch. You want to use the routed ports G0/1 or G0/2 to leave the switch. And when that traffic gets to the 1921 or the Netgear the source address 172.24.16.62 or 172.24.24.128 or whatever reflects the vlan it came from and is the subnet that 1921 or Netgear need to be able to route to in order to send responses to your PCs etc.

[ianbrown553803657]

Awesome, thank you for all the help.

The VLANs need to be created on the routers (the 1921 and the Netgear) and they will route the traffic appropriately? At this point would this be a router on a stick config, kind of?

Would the interface on the router be G0/1.10, G0/1.20 ect?

[Richard Burts]

I do not want to be overly picky, but do want to be very clear about terminology: you do not need to create vlans on the 1921 or on the Netgear. You want to create routes for those subnets of the vlans for the 1921 and for the Netgear. And you want to configure address translation for those subnets on the 1921 and on the Netgear.

router on a stick is a possibility. To evaluate this possibility we need to think about where routing will be done for vlan to vlan traffic and where routing will be done for vlan to Internet traffic. You could choose to configure the interface of the switch connecting to the 1921 as a trunk carrying all of the vlans, and configure the interface on the 1921 connecting to the switch with vlan subinterfaces and IP addresses, and do the vlan to vlan routing on the 1921. In that case you would probably turn off ip routing on the switch and have it operate as simply a layer 2 switch. This would work pretty well for vlan to vlan traffic. But it would be a bit complicated for the vlan to Internet traffic. You would want to do the IP SLA on the 1921. You could configure the primary static default route and the floating static default route for failover on the 1921. In thinking about failover it occurs to me that you might want to create a new vlan on the switch and connect the Netgear in that new vlan. You would carry that new vlan on the trunk along with the other vlans. You would configure the vlan subinterface on the 1921 for the new vlan with an IP address in the subnet used by Netgear. That way the floating static default route could just use the IP of Netgear as the next hop. And it would allow you to configure the 1921 to do address translation for both the cable modem and for Netgear.

When I started writing the explanation about router on a stick I was thinking that it was a bit complicated and probably not worth it. But as I worked through the logic of how it might work (and especially realizing the advantage of the floating static default route to use the Netgear as the next hop) I think it is actually a very attractive alternative. I am not clear what that does to the objective of being able to prioritize voice traffic, etc. You could, of course, stay with your original plan to have the switch do the inter vlan routing, do the IP SLA, and do the failover routing. But in terms of routing between vlans and routing to the Internet I think router on a stick is a good alternative.