L3 Switch Routing Issue?

[Zaleon]

Hey All,

   I'm trying to setup a new home network with some new gear I got that consist of a Cisco C819 ISR, ASA 5506-X and 3650 switch. I had no problems with the C819 or the ASA 5506-X, but I can't seem to ping anything other than VLAN10, which is why I suspect its a simple routing issue I overlooked.

I've setup 5 sub-interfaces on the ASA as VLAN10, 20, 30, 40 and 50. I can ping both VLAN and internet addresses from the ASA console. I setup the same VLANs on the 3650 switch starting with VLAN 10, I assigned it an IP address and DHCP pool, but the only addresses I can ping are 10.10.1.x addresses. Any help would be much appreciated. My two configs are listed below:

ASA 5506-X Config:

: Saved

:
: Serial Number: 
: Hardware:   ASA5506, 4096 MB RAM, CPU Atom C2000 series 1250 MHz, 1 CPU (4 cores)
:
ASA Version 9.9(1)3
!
hostname HOME-FW1
domain-name home.internal
enable password 
names

!
interface GigabitEthernet1/1
 description WAN
 nameif outside
 security-level 0
 ip address 10.55.1.2 255.255.255.0
!
interface GigabitEthernet1/2
 no nameif
 no security-level
 no ip address
!
interface GigabitEthernet1/2.10
 description "Main VLAN"
 vlan 10
 nameif MAIN
 security-level 100
 ip address 10.10.1.1 255.255.255.0
!
interface GigabitEthernet1/2.20
 description "Media/Streaming VLAN"
 vlan 20
 nameif MEDIA
 security-level 95
 ip address 10.20.1.1 255.255.255.0
!
interface GigabitEthernet1/2.30
 description "WiFi Guest Network"
 vlan 30
 nameif GUEST
 security-level 80
 ip address 192.168.5.1 255.255.255.0
!
interface GigabitEthernet1/2.40
 description "Cameras and NVR"
 vlan 40
 nameif CAMERAS
 security-level 85
 ip address 10.40.1.1 255.255.255.0
!
interface GigabitEthernet1/2.50
 vlan 50
 nameif SECURITY
 security-level 90
 ip address 10.50.1.1 255.255.255.0
!
interface GigabitEthernet1/3
 shutdown
 no nameif
 no security-level
 no ip address
!
interface GigabitEthernet1/4
 shutdown
 no nameif
 no security-level
 no ip address
!
interface GigabitEthernet1/5
 shutdown
 no nameif
 no security-level
 no ip address
!
interface GigabitEthernet1/6
 shutdown
 no nameif
 no security-level
 no ip address
!
interface GigabitEthernet1/7
 shutdown
 no nameif
 no security-level
 no ip address
!
interface GigabitEthernet1/8
 shutdown
 no nameif
 no security-level
 no ip address
!
interface Management1/1
 management-only
 nameif management
 security-level 100
 ip address 10.5.1.1 255.255.255.0
!
ftp mode passive
clock timezone GMT -6
dns server-group DefaultDNS
 domain-name home.internal
object network MAIN-PAT
 subnet 10.10.0.0 255.255.255.0
object network MEDIA-PAT
 subnet 10.20.0.0 255.255.255.0
object network GUEST-PAT
 subnet 192.168.5.0 255.255.255.0
object network CAMERAS-PAT
 subnet 10.40.0.0 255.255.255.0
object network SECURITY-PAT
 subnet 10.50.0.0 255.255.255.0
access-list sfr_redirect extended permit ip any any
pager lines 24
mtu management 1500
mtu outside 1500
mtu MAIN 1500
mtu MEDIA 1500
mtu GUEST 1500
mtu CAMERAS 1500
mtu SECURITY 1500
icmp unreachable rate-limit 1 burst-size 1
no asdm history enable
arp timeout 14400
no arp permit-nonconnected
arp rate-limit 16384
!
object network MAIN-PAT
 nat (MAIN,outside) dynamic interface
object network MEDIA-PAT
 nat (MEDIA,outside) dynamic interface
object network GUEST-PAT
 nat (GUEST,outside) dynamic interface
object network CAMERAS-PAT
 nat (CAMERAS,outside) dynamic interface
object network SECURITY-PAT
 nat (SECURITY,outside) dynamic interface
route outside 0.0.0.0 0.0.0.0 10.55.1.1 1
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 sctp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
timeout conn-holddown 0:00:15
timeout igp stale-route 0:01:10
user-identity default-domain LOCAL
aaa authentication ssh console LOCAL
aaa authentication enable console LOCAL
aaa authentication login-history
http server enable
http 10.5.1.5 255.255.255.255 management
no snmp-server location
no snmp-server contact
no service password-recovery
service sw-reset-button
crypto ipsec security-association pmtu-aging infinite
crypto ca trustpool policy
telnet timeout 5
ssh stricthostkeycheck
ssh 10.10.1.15 255.255.255.255 MAIN
ssh timeout 5
ssh version 2
ssh key-exchange group dh-group1-sha1
console timeout 5

threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
ntp server 132.163.96.1
ntp server 132.163.96.2
ntp server 129.6.15.29
ntp server 129.6.15.28 prefer
ssl cipher default low
ssl cipher tlsv1 low
ssl cipher tlsv1.1 low
ssl cipher tlsv1.2 low
ssl cipher dtlsv1 low
dynamic-access-policy-record DfltAccessPolicy
username  password  privilege 15
!
class-map sfr
 match access-list sfr_redirect
class-map inspection_default
 match default-inspection-traffic
!
!
policy-map type inspect dns migrated_dns_map_1
 parameters
  message-length maximum client auto
  message-length maximum 512
  no tcp-inspection
policy-map global_policy
 class inspection_default
  inspect dns migrated_dns_map_1
  inspect ftp
  inspect h323 h225
  inspect h323 ras
  inspect ip-options
  inspect netbios
  inspect rsh
  inspect rtsp
  inspect skinny
  inspect esmtp
  inspect sqlnet
  inspect sunrpc
  inspect tftp
  inspect sip
  inspect xdmcp
 class sfr
  sfr fail-open
policy-map type inspect dns migrated_dns_map_2
 parameters
  message-length maximum client auto
  message-length maximum 512
  no tcp-inspection
!
service-policy global_policy global
prompt hostname context
no call-home reporting anonymous
Cryptochecksum:2f3927d85027fa3394dc3571043a766f
: end

3650 Switch Config:

Current configuration : 9042 bytes
!
! Last configuration change at 13:44:56 UTC Tue Mar 6 2018
!
version 16.3
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
!
hostname HOME-SW1
!
!
vrf definition Mgmt-vrf
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv6
 exit-address-family
!
enable secret 5 .
!
no aaa new-model
switch 1 provision ws-c3650-24ps
!
!
!
!
ip routing
!
!
!
ip dhcp excluded-address 10.10.1.1 10.10.1.40
ip dhcp excluded-address 10.10.1.241 10.10.1.255
!
ip dhcp pool VLAN10
 network 10.10.1.0 255.255.255.0
 default-router 10.10.1.1
 dns-server 208.67.222.222 208.67.220.220
!
!
!
!
!
!
!
!
!
crypto pki trustpoint TP-self-signed-765948992
 enrollment selfsigned
 subject-name cn=IOS-Self-Signed-Certificate-765948992
 revocation-check none
 rsakeypair TP-self-signed-765948992
!
!
crypto pki certificate chain TP-self-signed-765948992
 certificate self-signed 01
        quit
!
license boot level ipbasek9
diagnostic bootup level minimal
spanning-tree mode rapid-pvst
spanning-tree extend system-id
!
!
username  privilege 15 password 0 
!
redundancy
 mode sso
!
!
!
class-map match-any system-cpp-police-topology-control
  description Topology control
class-map match-any system-cpp-police-sw-forward
  description Sw forwarding, SGT Cache Full, LOGGING
class-map match-any system-cpp-default
  description DHCP snooping, show forward and rest of traffic
class-map match-any system-cpp-police-sys-data
  description Learning cache ovfl, Crypto Control, Exception, EGR Exception, NFL SAMPLED DATA, Gold Pkt, RPF Failed
class-map match-any system-cpp-police-punt-webauth
  description Punt Webauth
class-map match-any system-cpp-police-forus
  description Forus Address resolution and Forus traffic
class-map match-any system-cpp-police-multicast-end-station
  description MCAST END STATION
class-map match-any system-cpp-police-multicast
  description Transit Traffic and MCAST Data
class-map match-any system-cpp-police-l2-control
  description L2 control
class-map match-any system-cpp-police-dot1x-auth
  description DOT1X Auth
class-map match-any system-cpp-police-data
  description ICMP_GEN and BROADCAST
class-map match-any system-cpp-police-control-low-priority
  description ICMP redirect and general punt
class-map match-any system-cpp-police-wireless-priority1
  description Wireless priority 1
class-map match-any system-cpp-police-wireless-priority2
  description Wireless priority 2
class-map match-any system-cpp-police-wireless-priority3-4-5
  description Wireless priority 3,4 and 5
class-map match-any non-client-nrt-class
class-map match-any system-cpp-police-routing-control
  description Routing control
class-map match-any system-cpp-police-protocol-snooping
  description Protocol snooping
!
policy-map port_child_policy
 class non-client-nrt-class
  bandwidth remaining ratio 10
policy-map system-cpp-policy
 class system-cpp-police-data
  police rate 200 pps
 class system-cpp-police-sys-data
  police rate 100 pps
 class system-cpp-police-sw-forward
  police rate 1000 pps
 class system-cpp-police-multicast
  police rate 500 pps
 class system-cpp-police-multicast-end-station
  police rate 2000 pps
 class system-cpp-police-punt-webauth
 class system-cpp-police-l2-control
 class system-cpp-police-routing-control
  police rate 1800 pps
 class system-cpp-police-control-low-priority
 class system-cpp-police-wireless-priority1
 class system-cpp-police-wireless-priority2
 class system-cpp-police-wireless-priority3-4-5
 class system-cpp-police-topology-control
 class system-cpp-police-dot1x-auth
 class system-cpp-police-protocol-snooping
 class system-cpp-police-forus
 class system-cpp-default
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0
 vrf forwarding Mgmt-vrf
 ip address 10.5.1.2 255.255.255.0
 negotiation auto
!
interface GigabitEthernet1/0/1
 switchport access vlan 10
 switchport mode access
!
interface GigabitEthernet1/0/2
!
interface GigabitEthernet1/0/3
!
interface GigabitEthernet1/0/4
!
interface GigabitEthernet1/0/5
!
interface GigabitEthernet1/0/6
!
interface GigabitEthernet1/0/7
!
interface GigabitEthernet1/0/8
!
interface GigabitEthernet1/0/9
!
interface GigabitEthernet1/0/10
!
interface GigabitEthernet1/0/11
!
interface GigabitEthernet1/0/12
!
interface GigabitEthernet1/0/13
!
interface GigabitEthernet1/0/14
!
interface GigabitEthernet1/0/15
!
interface GigabitEthernet1/0/16
!
interface GigabitEthernet1/0/17
!
interface GigabitEthernet1/0/18
!
interface GigabitEthernet1/0/19
!
interface GigabitEthernet1/0/20
!
interface GigabitEthernet1/0/21
!
interface GigabitEthernet1/0/22
!
interface GigabitEthernet1/0/23
!
interface GigabitEthernet1/0/24
!
interface GigabitEthernet1/1/1
 switchport mode trunk
!
interface GigabitEthernet1/1/2
!
interface GigabitEthernet1/1/3
!
interface GigabitEthernet1/1/4
!
interface Vlan1
 no ip address
!
interface Vlan10
 ip address 10.10.1.2 255.255.255.0
!
ip default-gateway 10.55.1.2
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
!
ip route 0.0.0.0 0.0.0.0 10.55.1.2
!
ip access-list extended AutoQos-4.0-wlan-Acl-Bulk-Data
 permit tcp any any eq 22
 permit tcp any any eq 465
 permit tcp any any eq 143
 permit tcp any any eq 993
 permit tcp any any eq 995
 permit tcp any any eq 1914
 permit tcp any any eq ftp
 permit tcp any any eq ftp-data
 permit tcp any any eq smtp
 permit tcp any any eq pop3
ip access-list extended AutoQos-4.0-wlan-Acl-MultiEnhanced-Conf
 permit udp any any range 16384 32767
 permit tcp any any range 50000 59999
ip access-list extended AutoQos-4.0-wlan-Acl-Scavanger
 permit tcp any any range 2300 2400
 permit udp any any range 2300 2400
 permit tcp any any range 6881 6999
 permit tcp any any range 28800 29100
 permit tcp any any eq 1214
 permit udp any any eq 1214
 permit tcp any any eq 3689
 permit udp any any eq 3689
 permit tcp any any eq 11999
ip access-list extended AutoQos-4.0-wlan-Acl-Signaling
 permit tcp any any range 2000 2002
 permit tcp any any range 5060 5061
 permit udp any any range 5060 5061
ip access-list extended AutoQos-4.0-wlan-Acl-Transactional-Data
 permit tcp any any eq 443
 permit tcp any any eq 1521
 permit udp any any eq 1521
 permit tcp any any eq 1526
 permit udp any any eq 1526
 permit tcp any any eq 1575
 permit udp any any eq 1575
 permit tcp any any eq 1630
 permit udp any any eq 1630
 permit tcp any any eq 1527
 permit tcp any any eq 6200
 permit tcp any any eq 3389
 permit tcp any any eq 5985
 permit tcp any any eq 8080
!
!
!
control-plane
 service-policy input system-cpp-policy
!
!
no vstack
!
line con 0
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 password 
 login
line vty 5 15
 password 
 login
!
!
wsma agent exec
!
wsma agent config
!
wsma agent filesys
!
wsma agent notify
!
!
ap dot11 airtime-fairness policy-name Default 0
ap group default-group
ap hyperlocation ble-beacon 0
ap hyperlocation ble-beacon 1
ap hyperlocation ble-beacon 2
ap hyperlocation ble-beacon 3
ap hyperlocation ble-beacon 4
end

The show ip route/show route from both devices:

Switch:

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        10.10.1.0/24 is directly connected, Vlan10
L        10.10.1.2/32 is directly connected, Vlan10


ASA:

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2, V - VPN
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, + - replicated route
Gateway of last resort is 10.55.1.1 to network 0.0.0.0

S*       0.0.0.0 0.0.0.0 [1/0] via 10.55.1.1, outside
C        10.10.1.0 255.255.255.0 is directly connected, MAIN
L        10.10.1.1 255.255.255.255 is directly connected, MAIN
C        10.20.1.0 255.255.255.0 is directly connected, MEDIA
L        10.20.1.1 255.255.255.255 is directly connected, MEDIA
C        10.40.1.0 255.255.255.0 is directly connected, CAMERAS
L        10.40.1.1 255.255.255.255 is directly connected, CAMERAS
C        10.50.1.0 255.255.255.0 is directly connected, SECURITY
L        10.50.1.1 255.255.255.255 is directly connected, SECURITY
C        10.55.1.0 255.255.255.0 is directly connected, outside
L        10.55.1.2 255.255.255.255 is directly connected, outside
C        192.168.5.0 255.255.255.0 is directly connected, GUEST
L        192.168.5.1 255.255.255.255 is directly connected, GUEST

[thiland]

What is your topology?  Where are you sourcing the pings from and what IPs are you trying to ping?

Assuming:

Node > 3650 > ASA > 819 > Internet

Your switch seems to be the issue.

Is this the link to the ASA?

interface GigabitEthernet1/0/1
 switchport access vlan 10
 switchport mode access

Always add descriptions to your interfaces like "ASA G1/2" to make life easier.  Assuming it is, you need to make it an 802.1Q trunk so it can carry the VLAN tags the ASA is looking for.

This is only used when the switch is running in L2 mode (e.g., "no ip routing").

ip default-gateway 10.55.1.2

You're specifying a next hop IP that the switch doesn't have direct knowledge about (not a directly connected network).  This is called a recursive route and should not be used in this situation (it doesn't have a route to 10.55.1.2 anyway which is why it doesn't show in the RIB).  Make the next hop 10.10.1.1.

ip route 0.0.0.0 0.0.0.0 10.55.1.2 name DEFAULT

[Zaleon]

Your assumption is the correct topology. The source of the pings is the 3650 switch. I can ping 10.10.1.1 (ASA), 10.10.1.2 (Switch) and 10.10.1.41 (Laptop) which is the first DHCP ip handed out to my laptop connect to switch port GigabitEthernet1/0/1.

The link to the ASA is GigabitEthernet1/1/1 on the switch which is marked as trunk port and is connected to port GigabitEthernet1/2 on the ASA. The 10.55.1.2 address is assigned to port GigabitEthernet1/1 on the ASA and connects to the C819 router.

So am I needing to add a static route for each vlan on the switch back to the .1 address on the ASA?

[thiland]

Since the laptop switchport VLAN is in the same VLAN as the gateway (ASA) and your laptop is pointing at the ASA as the default gateway, your switch is not participating in any routing decisions for the laptop.

Your laptop has the correct gateway which is good.

The switch just needs a default route to 10.10.1.1, but but that's a side issue.

If you're trying to ping the ASA VLAN interface IPs from your laptop, you should be able to ping the directly connected ASA interface (10.10.1.1).  Unlike a router, ASAs do not allow pinging of other interfaces except the one that you're on.

I'm still unclear which source device you're on and what IP(s) you're trying reach, but hopefully that helps.

[Zaleon]

Well the main issue is I can't connect to nor ping anything on the Internet from the switch or my laptop connected to the switch on VLAN10 which is the 10.10.1.x subnet. I haven't even added my other 4 subnets to the switch because if one won't work the others won't as well.

The issue could be on the ASA, If I console into the ASA, I can ping 8.8.8.8 and 8.8.4.4 directly from the console. However, if I plug my laptop into the ASA's GigabitEthernet1/2 where all the sub interfaces are setup, I can't ping or connect to anything. I assume this is because I can't assign a VLAN to my laptop directly.

If I move my laptop back to the switch, I can ping everything on the 10.10.1.x network, but nothing else. I am aware the ASA will block pings/connection to other VLANs but why can't I get to the Internet?

[thiland · Accepted solution]

On your ASA change the object for MAIN to 10.10.1.0 so it matches the correct source network.

object network MAIN-PAT
 subnet 10.10.0.0 255.255.255.0

 Also validate the other rules (routing, ACL, NAT, VPN, etc.) by using packet tracer from the command line which will tell you if a flow phase is incorrect.

packet-tracer input MAIN icmp 10.10.1.10 8 0 8.8.8.8

[Zaleon]

Ugh, I knew it was probably something simple I missed. Sometimes it helps to have someone look at things for sure. It was that 1 number off in the object subnet that was causing all the issues. I have corrected the other subnets as well. Its all working as expected now. Thanks for your solution Thiland!