Cisco Sx550x - access lists suddenly stops working

[mferk11]

Hi,

does anybody have problems with access lists on new Cisco Sx350 and SX550X devices. Everything works ok for 2-3 monts and then ACL suddenly stops working and acts as everything is denied. Is this access list to much for small business devices ?

It si very strange, because it all works and then just stops, you also cannot se logs in real time. When I turn off access list on VLAN logs keeps comming for hours, and there are no access lists turned on...  

ip access-list extended #########name##########
permit udp 0.0.0.0 0.0.0.0 bootpc 255.255.255.255 0.0.0.0 bootps ace-priority 10
permit udp any any any 67-68 ace-priority 20
permit tcp 10.203.180.0 0.0.3.255 domain any any ace-priority 30
permit tcp 10.203.180.0 0.0.3.255 any any domain ace-priority 40
permit udp 10.203.180.0 0.0.3.255 domain any any ace-priority 50
permit udp 10.203.180.0 0.0.3.255 any any domain ace-priority 60
permit icmp 10.203.180.0 0.0.3.255 any echo-reply any ace-priority 70
permit icmp 10.203.180.0 0.0.3.255 any echo-request any ace-priority 80
permit ip 10.203.180.0 0.0.3.255 172.28.88.0 0.0.0.255 ace-priority 90
permit tcp 10.203.180.0 0.0.3.255 9100 any any ace-priority 100
permit tcp 10.203.180.0 0.0.3.255 any any 9100 ace-priority 110
permit ip 10.203.180.0 0.0.3.255 10.7.7.30 0.0.0.0 ace-priority 120
permit ip 10.203.180.0 0.0.3.255 10.7.7.122 0.0.0.0 ace-priority 130
permit ip 10.203.180.0 0.0.3.255 10.0.0.119 0.0.0.0 ace-priority 140
permit ip 10.203.180.0 0.0.3.255 10.0.0.59 0.0.0.0 ace-priority 150
permit ip 10.203.180.0 0.0.3.255 10.0.0.64 0.0.0.0 ace-priority 160
permit ip 10.203.180.0 0.0.3.255 10.0.0.214 0.0.0.0 ace-priority 180
permit ip 10.203.180.0 0.0.3.255 10.0.0.115 0.0.0.0 ace-priority 190
permit ip 10.203.180.0 0.0.3.255 10.0.0.153 0.0.0.0 ace-priority 200
permit ip 10.203.180.0 0.0.3.255 10.0.0.47 0.0.0.0 ace-priority 210
permit ip 10.203.180.0 0.0.3.255 10.0.0.34 0.0.0.0 ace-priority 220
permit ip 10.203.180.0 0.0.3.255 10.0.0.148 0.0.0.0 ace-priority 230
permit ip 10.203.180.0 0.0.3.255 10.0.0.24 0.0.0.0 ace-priority 240
permit ip 10.203.180.0 0.0.3.255 10.0.0.32 0.0.0.0 ace-priority 250
permit ip 10.203.180.0 0.0.3.255 10.0.0.81 0.0.0.0 ace-priority 270
permit ip 10.203.180.0 0.0.3.255 10.0.0.96 0.0.0.0 ace-priority 280
permit ip 10.203.180.0 0.0.3.255 10.0.0.104 0.0.0.0 ace-priority 290
permit ip 10.203.180.0 0.0.3.255 10.0.0.116 0.0.0.0 ace-priority 300
permit ip 10.203.180.0 0.0.3.255 10.0.0.133 0.0.0.0 ace-priority 310
permit ip 10.203.180.0 0.0.3.255 10.0.0.210 0.0.0.0 ace-priority 320
permit ip 10.203.180.0 0.0.3.255 10.0.0.233 0.0.0.0 ace-priority 330
permit ip 10.203.180.0 0.0.3.255 10.203.180.0 0.0.3.255 ace-priority 340
deny tcp 10.203.180.0 0.0.3.255 any any telnet ace-priority 350
deny tcp 10.203.180.0 0.0.3.255 any any 22 ace-priority 360
deny udp 10.203.180.0 0.0.3.255 any any 22 ace-priority 370
deny tcp 10.203.180.0 0.0.3.255 any any 3389 ace-priority 380 log-input
deny udp 10.203.180.0 0.0.3.255 any any 3389 ace-priority 390 log-input
permit ip 10.203.180.0 0.0.3.255 10.0.0.8 0.0.0.0 ace-priority 400
permit ip 10.203.180.0 0.0.3.255 10.0.0.9 0.0.0.0 ace-priority 410
permit ip 10.203.180.0 0.0.3.255 10.0.0.10 0.0.0.0 ace-priority 420
permit ip 10.203.180.0 0.0.3.255 10.0.0.11 0.0.0.0 ace-priority 430
permit ip 10.203.180.0 0.0.3.255 10.0.0.12 0.0.0.0 ace-priority 440
permit ip 10.203.180.0 0.0.3.255 10.0.0.13 0.0.0.0 ace-priority 450
permit ip 10.203.180.0 0.0.3.255 10.0.0.14 0.0.0.0 ace-priority 460
permit ip 10.203.180.0 0.0.3.255 10.0.0.15 0.0.0.0 ace-priority 470
permit ip 10.203.180.0 0.0.3.255 10.0.0.18 0.0.0.0 ace-priority 480
permit ip 10.203.180.0 0.0.3.255 10.0.0.20 0.0.0.0 ace-priority 490
permit ip 10.203.180.0 0.0.3.255 10.0.0.23 0.0.0.0 ace-priority 495
permit ip 10.203.180.0 0.0.3.255 10.0.0.40 0.0.0.0 ace-priority 500
permit ip 10.203.180.0 0.0.3.255 10.0.0.41 0.0.0.0 ace-priority 510
permit ip 10.203.180.0 0.0.3.255 10.0.0.43 0.0.0.0 ace-priority 520
permit ip 10.203.180.0 0.0.3.255 10.0.0.44 0.0.0.0 ace-priority 530
permit ip 10.203.180.0 0.0.3.255 10.0.0.46 0.0.0.0 ace-priority 540
permit ip 10.203.180.0 0.0.3.255 10.0.0.66 0.0.0.0 ace-priority 550
permit ip 10.203.180.0 0.0.3.255 10.0.0.68 0.0.0.0 ace-priority 560
permit ip 10.203.180.0 0.0.3.255 10.0.0.69 0.0.0.0 ace-priority 570
permit ip 10.203.180.0 0.0.3.255 10.0.0.74 0.0.0.0 ace-priority 580
permit ip 10.203.180.0 0.0.3.255 10.0.0.131 0.0.0.0 ace-priority 590
permit ip 10.203.180.0 0.0.3.255 10.0.0.134 0.0.0.0 ace-priority 600
permit ip 10.203.180.0 0.0.3.255 10.0.0.135 0.0.0.0 ace-priority 610
permit ip 10.203.180.0 0.0.3.255 10.0.0.136 0.0.0.0 ace-priority 620
permit ip 10.203.180.0 0.0.3.255 10.0.0.147 0.0.0.0 ace-priority 630
permit ip 10.203.180.0 0.0.3.255 10.0.0.150 0.0.0.0 ace-priority 640
permit ip 10.203.180.0 0.0.3.255 10.0.0.194 0.0.0.0 ace-priority 650
permit ip 10.203.180.0 0.0.3.255 10.0.0.205 0.0.0.0 ace-priority 660
permit ip 10.203.180.0 0.0.3.255 10.0.0.42 0.0.0.0 ace-priority 670
permit ip 10.203.180.0 0.0.3.255 10.0.0.241 0.0.0.0 ace-priority 680
permit ip 10.203.180.0 0.0.3.255 10.0.0.25 0.0.0.0 ace-priority 690
permit ip 10.203.180.0 0.0.3.255 10.0.0.242 0.0.0.0 ace-priority 700
permit ip 10.203.180.0 0.0.3.255 10.0.0.243 0.0.0.0 ace-priority 710
permit ip 10.203.180.0 0.0.3.255 10.0.0.244 0.0.0.0 ace-priority 720
permit ip 10.203.180.0 0.0.3.255 10.0.0.248 0.0.0.0 ace-priority 730
permit ip 10.203.180.0 0.0.3.255 10.0.0.249 0.0.0.0 ace-priority 740
permit ip 10.203.180.0 0.0.3.255 10.0.0.246 0.0.0.0 ace-priority 745
permit ip 10.203.180.0 0.0.3.255 10.100.0.0 0.0.255.255 ace-priority 750
permit ip 10.203.180.0 0.0.3.255 10.150.0.0 0.0.255.255 ace-priority 760
permit ip 10.203.180.0 0.0.3.255 10.160.0.0 0.0.255.255 ace-priority 770
permit ip 10.203.180.0 0.0.3.255 10.162.0.0 0.0.255.255 ace-priority 780
permit ip 10.203.180.0 0.0.3.255 10.163.0.0 0.0.255.255 ace-priority 790
permit ip 10.203.180.0 0.0.3.255 10.164.0.0 0.0.255.255 ace-priority 800
permit ip 10.203.180.0 0.0.3.255 10.165.0.0 0.0.255.255 ace-priority 810
permit ip 10.203.180.0 0.0.3.255 172.17.0.0 0.0.255.255 ace-priority 820
deny ip 10.203.180.0 0.0.3.255 10.0.0.0 0.255.255.255 ace-priority 830 log-input
deny ip 10.203.180.0 0.0.3.255 192.168.0.0 0.0.255.255 ace-priority 840 log-input
deny ip 10.203.180.0 0.0.3.255 172.16.0.0 0.15.255.255 ace-priority 850 log-input
permit ip 10.203.180.0 0.0.3.255 any ace-priority 860
exit

[balaji.bandi]

How is the CPU usage ? Do you see any Logs ?

Suggest to check any Latest firmware and upgrade and check  ?

Other suggestion for testing minimized the ACL shorter version and check also ?

[mferk11]

Hi,

CPU usage is normal…

So you think that those ACLs are to big for small business series ?

We have latest firmware for that switch:

2.5.9.16

Regards,

Nejc

[mferk11]

When this happens I also tested ACL with:

permit tcp any any any any ace-priority 10 log-input
permit udp any any any any ace-priority 20 log-input
permit ip any any ace-priority 30 log-input
exit

And it was still everything blocked on this VLAN

Only restart of CORE switches helps.

[balaji.bandi]

Only restart of CORE switches helps.

CORE switch any bigger model or 350 only ?

i would not advice log anytime these devices.

I know it is security concern, what happens when there is no ACL at all ?

[Karsten Iwen]

Whatever this problem is. You really should consider doing this filtering on a device that is built for this: A firewall.

[mferk11]

Hi, please tell me. How is the firewall responsible for internal VLAN-routing and implementing access between local IP addresses/subnets. That is not a job for a firewall, that is a job for a CORE switch. 

[Karsten Iwen]

If you need extensive filtering between VLANs, the VLANs should be terminated on a firewall capable of handling the required throughput. A switch is not a suitable device for this task as it has no insight into the upper layers and is typically not stateful.

[mferk11]

Yeah, thats why Cisco Implemented ISE, to not have downloadable ACLs. Its better to do it on firewall ... 

[Karsten Iwen]

Downloadable ACLs are a great tool. But Switch resources always have to be kept in mind. ACLs are implemented in hardware, and these resources are limited. On paper, the SG550X supports "up to" 2000 ACEs, but I wouldn't suspect to be able to go to this limit.

[mferk11]

Yeh, it is very funny, because we have so SG300 and ACLs on it, bigger than this one and everything works ok. But ACLs on those never Cisco switches doesn't work ok.... I know that they need catalyst 9300 minimum for CORE switch but those thing that worked on older small business switches doesn't work ok on Sx350 and Sx550x switches...