Cisco c9200 with c8200 network chaos

[hayden]

Hello,

Apologies for the not very specific subject title. Im working on a production system, which many mistakes have been made today with compounding hardware issues on top of it. Im very stress rn, will have to go back tomorrow to make sure my revert worked since it didnt look like it did completely before i left.

So, we installed a c9200-48p-4l? Switch a few months ago, and left a normal gamer/consumer router handling routing and dhcp. No problems other than the ubiquiti system not playing ball. We got the router in the other day and have just attempted an install. Now let the chaos rain.

No vlan subnets were implemented previously, so on the switch. We have now setup vlans 10 workshop (.1.0), 20 desktops (.2.0), 30 phones (.3.0) , 40 cameras, 50 wifi.

Dhcp pools created on switch.

Network 192.168. 255.255.255.0

Default router (.x.1)

Dns-server 1.1.1.1 8.8.8.8

Vlan interfaces with ips made.

Ip add (.x.1) 255.255.255.0

Cool.

Interfaces which have the ip phones which are mostly sip-t54w have been setup with:Switchport voice vlan 30

Switchport Portsec max 2

Switchport mode access

Switchport access vlan 20

Auto qos something? Ill make a comment when i find it in the saved config i got.

Interface 48 (con to router)

No switchport

Ip add 192.167.0.2

Ip route 0.0.0.0 0.0.0.0 192.168.0.1

Router was given

Interface 2 (con to switch)

Ip address 192.168.0.1

Nat inside

Interface 1 (isp)

Ip add (public ip)

Nat outside

Ip default-gateway (gateway)

Nat inside source list 5 pool internetconnection overload

Ip access-list standard 5

Permit 192.168.2.0 0.0.0.255

Permit 192.168.3.0 0.0.0.255

Permit any (added lated to see why it wasnt work)

Okay so.

Heres a quick run down of whats happened and why this post is a thing.

We setup the switch and got the dhcp working easily. Both phones and pcs had IPs. Problem was getting internet access.

Eventually sorted it for the pcs. Again acl. Now phones didnt work. Acl was fine. But i forgot to give it to the nat.

Specified in the nat the correct acl.

Phones immediately registered. (They were all, "accounts not registered").

PCs immediately lost dhcp. No idea why. I removed the voice vlan from one interface and plugged my pc directly into the switch. Absolutely nada.

I spent a good hour trying to figure it out. Maybe another ACL somewhere, an ip route i might have missed or deleted. Maybe a miss typed or accidently deleted a nat. (But its dhcp gone) No idea why. Its still there, vlan interface, and the dhcp pool. But nothing on pcs.

Im thinking the voice vlan is taking over the entire port and not letting through the access port data. How can i fix this?

Anyone have thoughts?

[Flavio Miranda]

@hayden 

 I know you tried add only the configuration that you though matters but you could share the whole show running config and fi that would be easier to help.

"Ip add 192.167.0.2" this I believe is a typo

-----------------------------------

"Ip access-list standard 5

Permit 192.168.2.0 0.0.0.255

Permit 192.168.3.0 0.0.0.255"

You did not add the interfaces vlan configuration from the switch but for this ACL to match the traffic for natting, it should contemplate this two networks exactly.

192.168.2.0 255.255.255.0

192.168.3.0 255.255.255.0

Anything different from that wouldn´t work. Permit ip any any for NAT is not good.

Another problem I see is, you are calling the pool name on the NAT statement, but you have only one network on the NAT pool, right?

probably only one network is doing NAT.

You should be creating  3 NAT statement to contemplate all the DHCP pools.

Where the devices is getting DHCP from? Does it depend on the NAT?  IT seems the PCs lost DHCP connectivity when you fixed the phone, I dont believe the voice vlan could be a problem. Voice vlan on cisco switch is necessary if you have PCs behind phones. It is your scenario? what about if you add the phone directly on the cable and the PC direct in the cable?

[hayden]

Hello @Flavio Miranda,

Thanks for responding. Sorry about the errors. I free hand wrote that from my phone in the car haha...

I'll type everything out here, the config excluding the public or secret stuff was removed.

So the premise is. The switch handles all DHCP servers and inter-vlan routing. (I do plan to block the vlans from communication with one another after we get the devices actually connected first.

The switch is connected on port 48 to the router on a layer 3 port so there's no vlan stuff happening on the router, only the NAT. Makes configuring simpler I thought... The router does NAT to public IP and happy days. I did forget to add the other pools, sorry. Below is exactly what I got from the config before I removed it all back to vlan 1 to get them back going again.

My trouble shooting steps, I think i touched on in my post.- I connected the switch directly to my PC on an affected port. I also tried on an affected PC that had a working phone.

I really hope this helps, and its not just that stupid dhcp line, but that should theoretically block the 192.168.2.0 network from getting IP's too.

Here's what I got.

Switch config

ip routing
!
ip name-server 1.1.1.1 8.8.8.8
ip dhcp excluded-address 192.168.1.0
ip dhcp excluded-address 192.168.1.255 255.255.255.255 (Okay let me know if I'm crazy. This line right here. Might be it......... But that would also block 192.168.3.0.. from being generated??/ would it not )
!
ip dhcp pool desktops
network 192.168.2.0 255.255.255.0
default-router 192.168.2.1
dns-server 1.1.1.1 8.8.8.8
lease infinite
!
ip dhcp pool phones
network 192.168.3.0 255.255.255.0
default-router 192.168.3.1
dns-server 1.1.1.1 8.8.8.8
lease infinite
!
ip dhcp pool management
network 192.168.99.0 255.255.255.0
default-router 192.168.99.1
!
ip dhcp pool workshop
network 192.168.1.0 255.255.255.0
default-router 192.168.1.1
dns-server 1.1.1.1 8.8.8.8
lease infinite
!
ip dhcp pool WIFI
network 192.168.5.0 255.255.255.0
default-router 192.168.5.1
dns-server 1.1.1.1 8.8.8.8

policy-map AutoQos-4.0-Output-Policy
class AutoQos-4.0-Output-Priority-Queue
priority level 1 percent 30
class AutoQos-4.0-Output-Control-Mgmt-Queue
bandwidth remaining percent 10
queue-limit dscp cs2 percent 80
queue-limit dscp cs3 percent 90
queue-limit dscp cs6 percent 100
queue-limit dscp cs7 percent 100
queue-buffers ratio 10
class AutoQos-4.0-Output-Multimedia-Conf-Queue
bandwidth remaining percent 10
queue-buffers ratio 10
class AutoQos-4.0-Output-Trans-Data-Queue
bandwidth remaining percent 10
queue-buffers ratio 10
class AutoQos-4.0-Output-Bulk-Data-Queue
bandwidth remaining percent 4
queue-buffers ratio 10
class AutoQos-4.0-Output-Scavenger-Queue
bandwidth remaining percent 1
queue-buffers ratio 10
class AutoQos-4.0-Output-Multimedia-Strm-Queue
bandwidth remaining percent 10
queue-buffers ratio 10
class class-default
bandwidth remaining percent 25
queue-buffers ratio 25
policy-map AutoQos-4.0-Trust-Cos-Input-Policy
class class-default
set cos cos table AutoQos-4.0-Trust-Cos-Table

interface GigabitEthernet1/0/3
switchport access vlan 20
switchport mode access

switchport voice vlan 30
switchport port-security maximum 2
switchport port-security aging time 10
ip access-group Desktops in
trust device cts
auto qos trust cos
spanning-tree portfast
service-policy input AutoQos-4.0-Trust-Cos-Input-Policy
service-policy output AutoQos-4.0-Output-Policy

(the interface i tested my laptop on) its in the middle of a cluster of the affected interfaces.

interface GigabitEthernet1/0/6
switchport access vlan 20
switchport mode access
switchport port-security maximum 2
switchport port-security aging time 10
auto qos trust cos

interface GigabitEthernet1/0/48 ( to Router )
no switchport
ip address 192.168.0.2 255.255.255.0

interface Vlan10
description Workshop
ip address 192.168.1.251 255.255.255.0 (not a typo, workshop has weird static ips all through it)
ip access-group Workshop in
!
interface Vlan20
description Desktops
ip address 192.168.2.1 255.255.255.0
ip access-group Desktops in
!
interface Vlan30
description Phones
ip address 192.168.3.1 255.255.255.0
ip access-group Phones in
!
interface Vlan50
description WIFI
ip address 192.168.5.1 255.255.255.0
ip access-group WIFI in
!
interface Vlan99
description management
ip address 192.168.99.1 255.255.255.0
!
ip default-gateway 192.168.0.1
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 192.168.0.1
!
ip access-list standard Desktops
10 permit 192.168.2.0 0.0.0.255
20 permit any (only because we lost all connection, to see if the acl was blocking access to the access port)
ip access-list standard Phones
10 permit 192.168.3.0 0.0.0.255
20 permit any (only because we lost all connection, to see if the acl was blocking access to the access port)
ip access-list standard WIFI
10 permit 192.168.5.0 0.0.0.255
ip access-list standard Workshop
10 permit 192.168.1.0 0.0.0.255

Router Config.

ip name-server 1.1.1.1 8.8.8.8

interface GigabitEthernet0/0/0
ip address (ISP Address)
ip nat outside
negotiation auto
!
interface GigabitEthernet0/0/1
ip address 192.168.0.1 255.255.255.0
ip nat inside
negotiation auto

ip default-gateway (ISP gateway)

ip nat pool InternetConnection (ISP address) (ISP address) netmask (we were given a .254 netmask, but a singular address in the 252 range was all it would allow. it works so I'm not going to say no)

ip nat pool Inside 192.168.2.1 192.168.2.254 netmask 255.255.255.0
ip nat pool InsideWifi 192.168.5.1 192.168.5.254 netmask 255.255.255.0
ip nat pool InsidePhones 192.168.3.1 192.168.3.254 netmask 255.255.255.0

ip nat inside source list 5 pool InternetConnection overload

ip route 0.0.0.0 0.0.0.0 (ISP default gateway)

ip route 192.168.1.0 255.255.255.0 192.168.0.2
ip route 192.168.2.0 255.255.255.0 192.168.0.2
ip route 192.168.3.0 255.255.255.0 192.168.0.2
ip route 192.168.4.0 255.255.255.0 192.168.0.2
ip route 192.168.5.0 255.255.255.0 192.168.0.2

ip access-list standard 5
10 permit 192.168.2.0 0.0.0.255
20 permit 192.168.3.0 0.0.0.255
30 permit 192.168.5.0 0.0.0.255

ntp server pool.ntp.org
ntp server 0.au.pool.ntp.org

[balaji.bandi]

Switch :

ip dhcp excluded-address 192.168.1.255 255.255.255.255 - this should be corrected, if not DHCP also allocate Gateway ip .1 so it will have duplicate
ip dhcp excluded-address 192.168.1.1 - 10 or specific IP 192.168.1.251 so on

If switch doing Layer3 routing remove this :
no ip default-gateway 192.168.0.1

Until all working - remove SVI ACL ip access-group XXXX in (once all working you can implement more security)

interface Vlan10
description Workshop
ip address 192.168.1.251 255.255.255.0 - if you like 251 as IP configured, then you need to exclude ip in DHCP and DHCP default route ip need to change from .1 to .251)

Router :

since lot of information not correct - i would try below for testing :

ip nat inside source list 5 GigabitEthernet0/0/0 overload

You already have route command so you don't need below command

ip default-gateway (ISP gateway)

[hayden]

Thank you @balaji.bandi,

Ill be heading back there in the next few hours.

Ill update and let you know if I get it working.

I appreciate the help you two.

@Flavio Miranda, If you think its possibly caused by something else, feel free to chime in.

Cheers,

[Flavio Miranda]

@hayden 

 Besides the recommendation you got, I would recommend you to, at the beginning, dont use port-security and ACL on the interface vlans.

 Make everything work as plan as it can be to get confidence on what you are doing and , once everything is working fine, go step by step and add security.

Security features like port-security and ACL sometimes have negative effect and end up adding unncessary complexity one something apparently simple.

[paul driver]

Hello
First of what stands out -  how is the router aware of your new switch SVIs?
I do not see any static or dynamic  (or have you just omitted this)

Also suggest:
Note SVI acl logic
IN= traffic originating from within the vlan
OUT=  external traffic towards the vlan

Use extended acl s(sip/dip) for SVIs & NAT ( remove the svi acls for now and test reachabiltiy 
Specify a lease time (8rs) wired (2hrs max wifi) for you dhcp scopes also exclude ONLY the svi/rtr addressing


no Ip access-list standard 5
no ip default-gateway 192.168.0.1
no ip route 0.0.0.0 0.0.0.0 192.168.0.1

ip access-list extended NAT
permit ip  192.168.2.0 0.0.0.255 any
etc...
( obviously change your NAT statement to call the above new ACL)

ip route 0.0.0.0 0.0.0.0 gigabitEthernet1/0/48 192.168.0.1

[hayden]

Thank you @paul driver , @Flavio Miranda, @balaji.bandi,

TLDR: got the network working by removing my dhcp exclude mistake. Ubiquiti systems can be hit and miss. I think I understand networking, at least a little. There's a Yealink ACL and DHCP question at the bottom.

An update from my side, I appreciate all the help. Posting here let me actually recap and de-stress letting me see my issues regardless. Thanks for your time.

I removed the accidental

ip dhcp exclude-address 192.168.1.255 255.255.255.254 (I feel like I wrote this thinking one thing but typing another.)

This immediately got those PC's up and running. Then I spent a good 5 hours trying to get a Ubiquiti Dream Machine connected through the switch...... no internet .... Factory reset it, restored from a backup and connected all of their access points, and voila, internet. Idk guys, networking is hard sometimes. Probably isn't setup correctly at this stage but it was 8pm, pretty sure it was trunked on vlan 50 from the switch. Maybe the reset just let it setup correctly or something.

@paul driverThe route doesn't need to know about the vlan SVI's as they are running on a layer 3 interface, I'm pretty new to networking, but that's what I've understood anyways. All the config is detailed in the above post, other than my secrets etc haha... The router isn't running on-the-stick, instead it is simply just routing and doing NAT to any traffic this is incoming from the g0/0/1 interface (192.168.0.2). The only thing the router knows is that to get to the networks of the vlans, is through 192.168.0.2. Above there's a list of ip routes, detailing this, I could have specified it more elegantly, but for anyone who looks at the config later, thats probably the most clear way of doing it I reckon.

I'll be there tomorrow so I might actually apply the changes if they don't disrupt service much... here's hoping anyways.

Also I think you mixed up the ACL definitions, but that's okay.  https://www.cisco.com/c/en/us/support/docs/security/ios-firewall/23602-confaccesslists.html#toc-hId-802643119 (Define In, Out, Inbound, Outbound, Source, and Destination), I fact checked myself when I read your post. This will save you the trouble if you wanted to double check too. I think of it as -

this access list only allows this traffic out of the VLAN 

this access list only allows this traffic in to the VLAN

The port security at the time was disabled, it looks scary looking at my config, but its only enabled when you see 'switchport port-security'. Honestly, thankfully I didn't, because looking around there were random switch behind cabinets. A whole mess. Anyways..

ACL for YeaLinks question.

I did end up removing all the ACLs and slowly blocking all the traffic from 192.168.3.0 to the others networks. I haven't fine tuned it so at the end it still has 'permit ip any any'. I purposefully permitted the ports I found in Yealink's documentation and within their own network. But still nothing no internet with that. So 'any any' it was. I might have to connect to that network manually and do some Wireshark sniffing to figure it out. But that might take some time. If you guys have any tips for that, that would be awesome.

This is general question about DHCP scopes. 

Quick background, I excluded a few addresses on my vlan 10 (.1.x network) since that had some statically set ip's around their workspace. I tried to statically set them again within the DHCP pool, but clearly... haha, probably knew this. Because the dhcp pool doesn't include those addresses, it doesn't let you statically set the IPs. So my question is... how do you statically set a devices with say 192.168.1.3 if that has been excluded. Or is that why it's not recommended.

Thank you incredible people who probably don't have imposter syndrome!

[paul driver]

Hello
The acl logic FOR svis I posted is correct - you need to remember these are logical interfaces  not physical which are performing your inter-vlan routing 
As for the rtr not needing to know about the internal subnets you have additionally created again this IS a requirement for how would the rtr know how to route to them an as such return traffic?

Anyway  i’m glad to hear you’ve got it initially working - i do believe you need to clean up your cfg a little to optimise though.

Edited:

---

@hayden wrote:

. So my question is... how do you statically set a devices with say 192.168.1.3 if that has been excluded. Or is that why it's not recommended.

---

ip dhcp pool PC1
host x.x.x.x y.y.y.y. 
hardware-address <mac dress of host>

[Flavio Miranda]

"Thank you incredible people who probably don't have imposter syndrome!"

 I dont dont know anyone who dont have it. It is part of the game.

God job.