Setup Cisco 3650 with a working UniFi Gateway Router

[R3spirat0r]

Hi All,

Hopefully this can be done. I have tirelessly looked through the forums and tried to see if others can do the same and tried different configs but no go.  Here is what I am trying to do.

I have a USG Router with 2xWAN ports and 1xLAN port WAN1 = ETH0 = 192.168.1.2/24 receiving internet from another Linksys router (192.168.1.1) connected to a Comcast. This is working just fine, no issues.

LAN1 = ETH1 = 10.0.0.0/24 and the USG is managing dhcp and routing for this network with IP Address 10.0.0.1.

UniFi gateway WiFi access point connected on LAN1 of this port with POE Injector. 

On this as well I have a seperate VLAN55 with a seperate 192.168.55.0/24 network. Works just fine.  

This as well is working fine.

On the USG Router there is a WAN2/LAN2 port for a seperate network.  ETH2: 192.168.2.0/24.  DHCP managed by USG at 192.168.2.1. When I plugin a PC to this, I get an IP address on this subnet and can connect to the internet just fine. 

I have a Catalyst 3650 Layer 3 Switch that I would like to integrate as a test.  Purpose: Have the layer 3 switch manage its own internal network i.e., DHCP and VLANS and have the USG to route just the internet traffic in and out.  I would like for the 3650 to have for example:

Int gi1/0/1 connected to ETH2 on the USG.

##Do you recommend switch port access or trunk?  I have tried trunk but the USG Routers ETH2 does not support trunking.  I would need to buy a USG L3 Switch to get this method to work##   

Int VLAN 100

IP address 192.168.100.1 255.255.255.0

No shut

IP DHCP pool pool100

network 192.168.100.0 255.255.255.0

I tried helplessly to get this to work using up routes and putting everything on VLAN1; using the USG for dhcp and after hours, I just wiped the config clean on the 3650. It has the latest iOS Gibraltar. It’s blank on a clean slate. 

Any ideas or suggestions?  Thank you all in advance for your time.

[Seb Rupik · Accepted solution]

Hi there,

Does the USG support routed interfaces? In light of not supporting trunk ports I think it is a better solution to create a point-to-point link with routed interfaces at either end:

USG:Eth2  <----> Gi1/0/1:3560

!
int gi1/0/1
  no switchport
  ip address 169.254.0.2 255.255.255.252
  no shut
!

configure the USG Eth2 interface with 169.254.0.1/30 .

The go ahead and configure the other VLANs and DHCP pools on the 3560. Keep the VLAN subnets contiguous so that they can be summarised. Then use the summary route on the USG to direct traffic to the 3560 for those subnets, eg:

!!USG router
ip route 192.168.100.0 255.255.252.0 169.254.0.2
!

then on the 3560 configure a default route towards the USG:

!
ip route 0.0.0.0 0.0.0.0 169.254.0.1
!

cheers,

Seb.

[R3spirat0r]

Okay I am trying this out and we are off to a good start.  If this works, I am sending you $ for a few Beers, trust me I am good for it!  It is comforting to know that I am not the smartest person in the room!  I did not even think about routing using the 169.254.x.x apipa!!! Brilliant. 

So far this is the running config and I am able to ping eth2 on the USG from the Switch, which is huge progress!

=========================================================

Cisco3650#ping 169.254.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 169.254.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/2/2 ms
=========================================================

Also, when I plug-in a PC to gi1/0/2 it receives an IP Address: 192.168.100.100 and can ping 169.254.0.2 with 100% success. 

Now, I still dont have internet on the connected PC on gi1/0/2 of the Cisco Switch.   So that is the next outstanding issue. 

Here is the running config:

Current configuration : 9061 bytes
!
! Last configuration change at 00:52:32 UTC Sun Jun 28 2020
!
version 16.12
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
! Call-home is enabled by Smart-Licensing.
service call-home
platform punt-keepalive disable-kernel-core
!
hostname Cisco3650
!
!
vrf definition Mgmt-vrf
!
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family

no aaa new-model
switch 1 provision ws-c3650-24ps
!
!
!
!
call-home
! If contact email address in call-home is configured as sch-smart-licensing@cisco.com
! the email address configured in Cisco Smart License Portal will be used as contact email address to send SCH notifications.
contact-email-addr sch-smart-licensing@cisco.com
profile "CiscoTAC-1"
active
destination transport-method http
no destination transport-method email
ip routing
!
!
!
!
!
ip dhcp excluded-address 192.168.100.1 192.168.100.100
ip dhcp excluded-address 192.168.100.200 192.168.100.255
!
ip dhcp pool pool100

network 192.168.100.0 255.255.252.0
default-router 192.168.100.1
dns-server 192.168.100.1

!
!
!
diagnostic bootup level minimal
!
spanning-tree mode rapid-pvst
spanning-tree extend system-id
memory free low-watermark processor 79502
!
!
redundancy
mode sso
!
!
!
!
!
transceiver type all
monitoring
!
!
class-map match-any system-cpp-police-topology-control
description Topology control

class-map match-any system-cpp-police-sw-forward
description Sw forwarding, L2 LVX data, LOGGING
class-map match-any system-cpp-default
description EWLC control, EWLC data, Inter FED
class-map match-any system-cpp-police-sys-data
description Learning cache ovfl, High Rate App, Exception, EGR Exception, NFL SAMPLED DATA, RPF Failed
class-map match-any system-cpp-police-punt-webauth
description Punt Webauth
class-map match-any system-cpp-police-l2lvx-control
description L2 LVX control packets
class-map match-any system-cpp-police-forus
description Forus Address resolution and Forus traffic
class-map match-any system-cpp-police-multicast-end-station
description MCAST END STATION
class-map match-any system-cpp-police-multicast
description Transit Traffic and MCAST Data
class-map match-any system-cpp-police-l2-control
description L2 control
class-map match-any system-cpp-police-dot1x-auth
description DOT1X Auth
class-map match-any system-cpp-police-data
description ICMP redirect, ICMP_GEN and BROADCAST
class-map match-any system-cpp-police-stackwise-virt-control
description Stackwise Virtual
class-map match-any non-client-nrt-class
class-map match-any system-cpp-police-routing-control
description Routing control and Low Latency
class-map match-any system-cpp-police-protocol-snooping
description Protocol snooping
class-map match-any system-cpp-police-dhcp-snooping

description Protocol snooping
class-map match-any system-cpp-police-dhcp-snooping
description DHCP snooping
class-map match-any system-cpp-police-system-critical
description System Critical and Gold Pkt
!
policy-map system-cpp-policy
!
!
!
!
!

interface GigabitEthernet0/0
vrf forwarding Mgmt-vrf
ip address 192.168.100.250 255.255.255.0
negotiation auto
!
interface GigabitEthernet1/0/1
no switchport
ip address 169.254.0.2 255.255.255.252
!
interface GigabitEthernet1/0/2
switchport mode access
!
interface GigabitEthernet1/0/3
!
interface GigabitEthernet1/0/4
!
interface GigabitEthernet1/0/5
!
interface GigabitEthernet1/0/6
!
interface GigabitEthernet1/0/7
!
interface GigabitEthernet1/0/8
!
interface GigabitEthernet1/0/9
!
interface GigabitEthernet1/0/10
!
interface GigabitEthernet1/0/11
!
interface GigabitEthernet1/0/12

!
interface GigabitEthernet1/0/13
!
interface GigabitEthernet1/0/14
!
interface GigabitEthernet1/0/15
!
interface GigabitEthernet1/0/16
!
interface GigabitEthernet1/0/17
!
interface GigabitEthernet1/0/18
!
interface GigabitEthernet1/0/19
!
interface GigabitEthernet1/0/20
!
interface GigabitEthernet1/0/21
!

interface GigabitEthernet1/0/22
!
interface GigabitEthernet1/0/23
!
interface GigabitEthernet1/0/24
!
interface GigabitEthernet1/1/1
!
interface GigabitEthernet1/1/2
!
interface GigabitEthernet1/1/3
!
interface GigabitEthernet1/1/4

interface Vlan1
ip address 192.168.100.1 255.255.252.0
!
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
ip route 0.0.0.0 0.0.0.0 169.254.0.1
!
!
ip access-list extended 101
10 permit ip 10.0.0.0 0.255.255.255 any
20 permit ip 192.0.0.0 0.255.255.255 any

!
!
control-plane
service-policy input system-cpp-policy
!
!
line con 0
stopbits 1
line aux 0
stopbits 1
line vty 0 4
password 
login
line vty 5 15
password 
login
!

!
!
!
!
!
!
!
end

[Seb Rupik]

Hello again,

You have two possible issues. The first will be NAT. The USG will already be NAT'ing the subnets which are directly connected to it, ie, the subnets in 10.0.0.0/8 range, but you have now added subnets in the 192.168.0.0/16 range which are being routed on the 3560. You need to configure the USG to perform source NAT on devices in the 192.168.0.0/16 range.

The second possible problem you may encounter involves spoofing. The USG will only expect to see packets with a source IP of 169.254.0.0/30 on Eth2, anything else (ie, 192.168.0.0/16) it will think is spoofed traffic and may drop it. I say may, as it depends on how the USG is configured, or whether it performs the check at all.

Let us know how you get on.

cheers,

Seb.

[R3spirat0r]

Hi Seb, 

Yesterday I was looking at the config on the USG and I noticed the problem.

The static route to the 192.168.100.0 was configured wrong.  Originally you had mentioned to static route 192.168.100.0/30 over to the next hop on 169.254.0.2.

I changed the network subnet mask to /22 (i.e., 192.168.100.0/22) and I immediately gained an internet connection!  

Fantastic!!!  After the slight adjustment, your method worked.

Do you see any limitations with this type of setup?  For example, port forwarding or any other type of potential security issue or network limitation with this type of setup in a production environment?

Thank you Seb!

[Seb Rupik]

The route originally specified was a /22 :

!!USG router
ip route 192.168.100.0 255.255.252.0 169.254.0.2
!

The setup you are creating is a pretty standard production setup, with DMZ/ guest wireless functions being routed on a firewall and a further connection to an internal 'core' switch. Using a Layer3 switch for inter-VLAN routing is preferable to a firewall as it will have greater performance.

If you believe it has been, please mark your question as answered.

cheers,

Seb.

[Richard Burts]

I agree with Seb that if you want to connect the 3650 to the USG that it should be an access port on the 3650 to the USG. And if you want to configure additional vlans on the 3650 you could use the access port connecting to USG as a routed link (transit link between switch and USG). But I believe that there are some problems in getting that to work. Would you be able to get the USG to route to additional subnets on its LAN2? Would the USG be able to do address translation for additional subnets? I am not familiar with that router and so am not sure. But it sounds to me like the USG supports a single subnet connected through LAN2.

[R3spirat0r]

Hi Richard, 

Thank you for your response!  I just posted an update.  So far the transit link is working, just dont have internet.

Yes, Here is the USG specs and data sheet attached.  I would believe maybe it could support it?  I can add additional networks and specify the WAN2/LAN2 port. 

USG
Dimensions 135 x 135 x 28.3 mm
(5.32 x 5.32 x 1.11")
Weight 366 g
(12.9 oz)
Networking Interfaces
Serial Console Port
Data Ports
(1) RJ45 Serial Port
(3) 10/100/1000 Ethernet Ports
Max. Power Consumption 7W
Power Supply 12VDC, 1A Power Adapter (Included)
Supported Voltage Range 9 to 24VDC
LEDs
System
Serial Console Port
Data Ports
Status
Power
Speed/Link/Activity
Layer 3 Forwarding Performance
Packet Size: 64 Bytes
Packet Size: 512 Bytes or Larger
1,000,000 pps
3 Gbps (Line Rate)
Processor Dual-Core 500 MHz, MIPS64 with Hardware
Acceleration for Packet Processing
System Memory 512 MB DDR2 RAM
On-Board Flash Storage 2 GB
Rackmount Yes
Operating Temperature -10 to 45° C (14 to 113° F)
Operating Humidity 10 to 90% Noncondensing
Certifications CE, FCC, IC