Re: Simple Nexus Access List Question

hayesa · ‎06-01-2018

Hello,

I am running nxos 7.0(3)I7(3) and I'm confused on why I can't get a simple access list to work.

Maybe it's my unfamiliarity with the Nexus OS.

I am trying to block ICMP ping requests with the following:

ip access-list TEST
10 deny icmp any any

interface Ethernet1/1
ip port access-group TEST in
no switchport
ip address x.x.x.x/x
ip router eigrp xxxxx
no shutdown

I'm having absolutely no luck. Any ideas?

-Austin

dperezoquendo · ‎06-01-2018

Hello,

Try "ip access-group TEST in". Port ACL's only work on L2 interfaces. For more information, refer to the following link: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/4_1/nx-os/security/configuration/guide/sec_nx-os-cfg/sec_ipacls.html#wp1583685

hayesa · ‎06-01-2018

Thanks for the reply.

I'm getting the same issue. The "ip access-group TEST in" command is not working either.

ip access-list TEST
10 deny icmp any any

interface Ethernet1/1
ip access-group TEST in
no switchport
ip address x.x.x.x/x
ip router eigrp xxxx
no shutdown

dperezoquendo · ‎06-01-2018

Hmm, I am unsure. I just tested this on my network device and it blocked incoming ICMPs just fine. I assume there are more to the ACL?

hayesa · ‎06-01-2018

Unfortunately no. This is a brand new Nexus 3K with very basic configurations, not in production. I was prepping for deployment and got to configuring ACLs and noticed they weren't working as expected.

This is the only ACL on the box.

Reza Sharifi · ‎06-01-2018

Keep in mind that when you apply an ACL inbound (in) it only blocks IPs/hosts connected to that interface.

If you want to block traffic coming from another interface, the ACL needs to be in "out" direction.

Also, since you don't have any permit line, the ACL you posted will deny everything as there is an implicit deny at the end of every access-list.

HTH

hayesa · ‎06-01-2018

The switch has only one active port, the one I posted. So all traffic is traversing that link.
And you bring up another good point. Applying any ACL to a port should deny all traffic without a permit statement.

My setup is as follows:
I'm configuring via SSH from a networked PC. (This is the PC I'm pinging from)
The Nexus switch is connected with only 1 port to my Core Switch (6880) on an Access Switchport.
The Nexus switch has only a few VLANs configured, a default route to the mentioned Core Switch, a configured loopback, and some Radius configs.
The configs I posted are unaltered aside from the IP and EIGRP values.

I'm really not sure what the problem is.

dperezoquendo · ‎06-01-2018

Just to clarify, you have "permit ip any any" or something equivalent at the end of your TEST ACL. Also, when you try to ping the E1/1 interface from another device, it goes through. Is this correct?
If so, I can't think of anything else at the moment. Other than the port/router ACLs defined in the link I provided earlier, ACLs work the same way in the NX-OS as in IOS. I had also tested it on one of my 5k's whose port configuration was very similar to yours and it passed. :(

Peter Paluch · ‎06-02-2018

Hello,

Indeed, the behavior is surprising - the other friends here had very good points.

May I suggest an additional experiment? Please modify your ACL as follows:

ip access-list TEST
  statistics per-entry
  deny icmp any any
  permit ip any any

Then install it on your e1/1:

interface Ethernet1/1
  ip access-group TEST in

Then try pinging this interface, or anything beyond this interface (the deny icmp any any should block it), and then please post the outputs of the following commands - we need to understand if the ACL was properly applied and whether it registered any matches:

show ip access-list TEST
show hardware access-list interface e1/1 input config
show hardware access-list interface e1/1 input entries detail

Thank you!

Best regards,
Peter

hayesa · ‎06-04-2018

Hi Peter,

I've added the "permit ip any any" & "statistics per-entry" to the ACL and have posted the output of the show commands below. I'm definitely getting matches on my pings. I kept rerunning "sh ip access-lists TEST" and was seeing an incremental increase of matches with the pings. However, I am still getting responses on my PC.

DR_3K_1(config-acl)# sh ip access-lists TEST

IP access list TEST
statistics per-entry
10 deny icmp any any [match=17]
20 permit ip any any [match=175]

DR_3K_1(config-acl)# show hardware access-list interface e1/1 input config

slot 1
=======


Policy id: 4, Type: RACL, Protocol: IPv4 Name: TEST

deny icmp 0.0.0.0/0.0.0.0 0.0.0.0/0.0.0.0
permit ip 0.0.0.0/0.0.0.0 0.0.0.0/0.0.0.0
deny ip 0.0.0.0/0.0.0.0 0.0.0.0/0.0.0.0 *

DR_3K_1(config-acl)# show hardware access-list interface e1/1 input entries detail

slot 1
=======


Flags: F - Fragment entry E - Port Expansion
D - DSCP Expansion M - ACL Expansion
T - Cross Feature Merge Expansion
N - NS Transit B - BCM Expansion C - COPP


INSTANCE 0x0
---------------

Tcam 2 resource usage:
----------------------
LBL C = 0x1
Bank 0
------
IPv4 Class
Policies: RACL(TEST)
Netflow profile: 0
Netflow deny profile: 0
Entries:
[Index] Entry [Stats]
---------------------
[0x0000:0x0001:0x0001] deny icmp 0.0.0.0/0 0.0.0.0/0 [17]
[0x0003:0x0004:0x0004] permit ip 0.0.0.0/0 0.0.0.0/0 [277]
[0x0004:0x0005:0x0005] deny ip 0.0.0.0/0 0.0.0.0/0 [0]


L4 protocol cam entries usage: none

No mac protocol cam entries are in use

Peter Paluch · ‎06-04-2018

Hi Austin,

This is getting very suspicious. May I ask you for a set of more extensive outputs if it is possible to share them?

### Begin

terminal length 0

show version

show module

show running-config

show policy-map interface control-plane

show mac address-table

show ip arp

show ip arp detail

show ip route

show forwarding route

terminal no length

### End

Also, would it be possible for you to run Wireshark on your PC, record the pings as they are responded to even despite the ACL, and share that Wireshark capture file here as well?

Thank you!

Best regards,
Peter

hayesa · ‎06-04-2018

I can provide you with some of it.

DR_3K_1# sh ver
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (C) 2002-2018, Cisco and/or its affiliates.
All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under their own
licenses, such as open source. This software is provided "as is," and unless
otherwise stated, there is no warranty, express or implied, including but not
limited to warranties of merchantability and fitness for a particular purpose.
Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or
GNU General Public License (GPL) version 3.0 or the GNU
Lesser General Public License (LGPL) Version 2.1 or
Lesser General Public License (LGPL) Version 2.0.
A copy of each such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://opensource.org/licenses/gpl-3.0.html and
http://www.opensource.org/licenses/lgpl-2.1.php and
http://www.gnu.org/licenses/old-licenses/library.txt.

Software
BIOS: version 04.17
NXOS: version 7.0(3)I7(3)
BIOS compile time: 09/12/2017
NXOS image file is: bootflash:///nxos.7.0.3.I7.3.bin
NXOS compile time: 2/12/2018 13:00:00 [02/12/2018 14:13:48]

Hardware
cisco Nexus 31108TCV Chassis (Nexus 9000 Series)
Intel(R) Core(TM) i3- CPU @ 2.50GHz with 16400992 kB of memory.
Processor Board ID FOC221210RF

Device name: DR_3K_1
bootflash: 53298520 kB
Kernel uptime is 5 day(s), 3 hour(s), 7 minute(s), 20 second(s)

Last reset
Reason: Unknown
System version: 7.0(3)I7(3)
Service:

plugin
Core Plugin, Ethernet Plugin

Active Package(s):

DR_3K_1# sh module
Mod Ports Module-Type Model Status
--- ----- ------------------------------------- --------------------- ---------
1 54 48x1/10G-T 6x40/100G QSFP28 Ethernet N3K-C31108TC-V active *

Mod Sw Hw Slot
--- ---------------- ------ ----
1 7.0(3)I7(3) 1.1 NA

Mod MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
1 28-ac-9e-fc-06-00 to 28-ac-9e-fc-06-5f FOC221210RF

Mod Online Diag Status
--- ------------------
1 Pass

* this terminal session

DR_3K_1# sh running-config

!Command: show running-config
!Time: Mon Jun 4 13:59:02 2018

version 7.0(3)I7(3)
hostname DR_3K_1
vdc DR_3K_1 id 1
limit-resource vlan minimum 16 maximum 4094
limit-resource vrf minimum 2 maximum 4096
limit-resource port-channel minimum 0 maximum 511
limit-resource u4route-mem minimum 248 maximum 248
limit-resource u6route-mem minimum 96 maximum 96
limit-resource m4route-mem minimum 58 maximum 58
limit-resource m6route-mem minimum 8 maximum 8

feature eigrp
feature interface-vlan
feature vtp
clock timezone EST -5 0
clock summer-time EDT 2 Sun Mar 02:00 1 Sun Nov 02:00 60

no password strength-check
username admin password 5 xxxx role network-admin

banner motd #

"Omitted"

#

no ip domain-lookup
ip domain-name xxxx.com
radius-server key 7 "xxxx"
radius-server host x.x.x.x authentication accounting timeout 2
radius-server host x.x.x.x authentication accounting timeout 2
radius-server host x.x.x.x authentication accounting timeout 2
aaa group server radius RadServer
server x.x.x.x
server x.x.x.x
server x.x.x.x
ip access-list TEST
statistics per-entry
10 deny icmp any any
20 permit ip any any
class-map type qos match-all WAN_ENS_Bandwidth
copp profile strict
snmp-server user admin network-admin auth md5 xxxx priv xxxx localizedkey
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO
ntp server x.x.x.x use-vrf default
ntp source-interface Ethernet1/1
aaa authentication login default group RadServer
aaa authentication login console group RadServer
aaa accounting default group RadServer
aaa authentication login error-enable

"Static Routes Omitted"

no ip source-route
vlan 1,3-4,10-11,172
vlan 3
vlan 4
name Servers
vlan 10
vlan 11
vlan 172
name Vendor

vrf context management
hardware profile portmode 48x10G+6x100G

vlan configuration 4

interface Vlan1

interface Vlan4
ip address x.x.x.x/x
ip router eigrp xxxx

interface Ethernet1/1
ip access-group TEST in
no switchport
ip address x.x.x.x/x
ip router eigrp xxxx
no shutdown

interface Ethernet1/2
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/3
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/4
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/5
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/6
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/7
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/8
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/9
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/10
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/11
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/12
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/13
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/14
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/15
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/16
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/17
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/18
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/19
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/20
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/21
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/22
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/23
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/24
switchport access vlan 4
spanning-tree port type edge

interface Ethernet1/25
switchport access vlan 3
spanning-tree port type edge

interface Ethernet1/26
switchport access vlan 3
spanning-tree port type edge

interface Ethernet1/27
switchport access vlan 3
spanning-tree port type edge

interface Ethernet1/28
switchport access vlan 3
spanning-tree port type edge

interface Ethernet1/29
switchport access vlan 10
spanning-tree port type edge

interface Ethernet1/30
switchport access vlan 10
spanning-tree port type edge

interface Ethernet1/31
switchport access vlan 10
spanning-tree port type edge

interface Ethernet1/32
switchport access vlan 10
spanning-tree port type edge

interface Ethernet1/33
switchport access vlan 10
spanning-tree port type edge

interface Ethernet1/34
switchport access vlan 10
spanning-tree port type edge

interface Ethernet1/35
switchport access vlan 10
spanning-tree port type edge

interface Ethernet1/36
switchport access vlan 10
spanning-tree port type edge

interface Ethernet1/37
switchport access vlan 10
spanning-tree port type edge

interface Ethernet1/38
switchport access vlan 10
spanning-tree port type edge

interface Ethernet1/39
switchport access vlan 172
spanning-tree port type edge

interface Ethernet1/40
switchport access vlan 172
spanning-tree port type edge

interface Ethernet1/41
switchport access vlan 172
spanning-tree port type edge

interface Ethernet1/42
switchport access vlan 172
spanning-tree port type edge

interface Ethernet1/43
switchport access vlan 172
spanning-tree port type edge

interface Ethernet1/44
switchport access vlan 172
spanning-tree port type edge

interface Ethernet1/45
switchport access vlan 172
spanning-tree port type edge

interface Ethernet1/46
switchport access vlan 172
spanning-tree port type edge

interface Ethernet1/47
switchport access vlan 172
spanning-tree port type edge

interface Ethernet1/48
switchport access vlan 172
spanning-tree port type edge

interface Ethernet1/49

interface Ethernet1/50

interface Ethernet1/51

interface Ethernet1/52

interface Ethernet1/53

interface Ethernet1/54

interface mgmt0
vrf member management

interface loopback0
shutdown
ip address x.x.x.x/32
ip router eigrp xxxx
line console
line vty
boot nxos bootflash:/nxos.7.0.3.I7.3.bin
router eigrp xxxx
router-id x.x.x.x

DR_3K_1# sh policy-map interface control-plane
Control Plane

Service-policy input: copp-system-p-policy-strict

class-map copp-system-p-class-l3uc-data (match-any)
match exception glean
set cos 1
police cir 250 pps , bc 32 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-critical (match-any)
match access-group name copp-system-p-acl-bgp
match access-group name copp-system-p-acl-rip
match access-group name copp-system-p-acl-vpc
match access-group name copp-system-p-acl-bgp6
match access-group name copp-system-p-acl-ospf
match access-group name copp-system-p-acl-rip6
match access-group name copp-system-p-acl-eigrp
match access-group name copp-system-p-acl-ospf6
match access-group name copp-system-p-acl-eigrp6
match access-group name copp-system-p-acl-auto-rp
match access-group name copp-system-p-acl-mac-l3-isis
set cos 7
police cir 19000 pps , bc 128 packets
module 1 :
transmitted 474262 packets;
dropped 0 packets;

class-map copp-system-p-class-important (match-any)
match access-group name copp-system-p-acl-hsrp
match access-group name copp-system-p-acl-vrrp
match access-group name copp-system-p-acl-hsrp6
match access-group name copp-system-p-acl-vrrp6
match access-group name copp-system-p-acl-mac-lldp
set cos 6
police cir 3000 pps , bc 128 packets
module 1 :
transmitted 441466 packets;
dropped 0 packets;

class-map copp-system-p-class-openflow (match-any)
match access-group name copp-system-p-acl-openflow
set cos 5
police cir 2000 pps , bc 32 packets
module 1 :
transmitted 5 packets;
dropped 0 packets;

class-map copp-system-p-class-multicast-router (match-any)
match access-group name copp-system-p-acl-pim
match access-group name copp-system-p-acl-msdp
match access-group name copp-system-p-acl-pim6
match access-group name copp-system-p-acl-pim-reg
match access-group name copp-system-p-acl-pim6-reg
match access-group name copp-system-p-acl-pim-mdt-join
set cos 6
police cir 3000 pps , bc 128 packets
module 1 :
transmitted 3 packets;
dropped 0 packets;

class-map copp-system-p-class-multicast-host (match-any)
match access-group name copp-system-p-acl-mld
set cos 1
police cir 2000 pps , bc 128 packets
module 1 :
transmitted 137 packets;
dropped 0 packets;

class-map copp-system-p-class-l3mc-data (match-any)
match exception multicast rpf-failure
match exception multicast dest-miss
set cos 1
police cir 3000 pps , bc 32 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-normal (match-any)
match access-group name copp-system-p-acl-mac-dot1x
match protocol arp
set cos 1
police cir 1500 pps , bc 32 packets
module 1 :
transmitted 3514728 packets;
dropped 1588 packets;

class-map copp-system-p-class-ndp (match-any)
match access-group name copp-system-p-acl-ndp
set cos 6
police cir 1500 pps , bc 32 packets
module 1 :
transmitted 358 packets;
dropped 0 packets;

class-map copp-system-p-class-normal-dhcp (match-any)
match access-group name copp-system-p-acl-dhcp
match access-group name copp-system-p-acl-dhcp6
set cos 1
police cir 300 pps , bc 32 packets
module 1 :
transmitted 2 packets;
dropped 0 packets;

class-map copp-system-p-class-normal-dhcp-relay-response (match-any)
match access-group name copp-system-p-acl-dhcp-relay-response
match access-group name copp-system-p-acl-dhcp6-relay-response
set cos 1
police cir 400 pps , bc 64 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-normal-igmp (match-any)
match access-group name copp-system-p-acl-igmp
set cos 3
police cir 6000 pps , bc 64 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-redirect (match-any)
match access-group name copp-system-p-acl-ptp
set cos 1
police cir 1500 pps , bc 32 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-exception (match-any)
match exception ip option
match exception ip icmp unreachable
match exception ipv6 option
match exception ipv6 icmp unreachable
set cos 1
police cir 50 pps , bc 32 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-exception-diag (match-any)
match exception ttl-failure
match exception mtu-failure
set cos 1
police cir 50 pps , bc 32 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-management (match-any)
match access-group name copp-system-p-acl-ftp
match access-group name copp-system-p-acl-ntp
match access-group name copp-system-p-acl-ssh
match access-group name copp-system-p-acl-http
match access-group name copp-system-p-acl-ntp6
match access-group name copp-system-p-acl-sftp
match access-group name copp-system-p-acl-snmp
match access-group name copp-system-p-acl-ssh6
match access-group name copp-system-p-acl-tftp
match access-group name copp-system-p-acl-https
match access-group name copp-system-p-acl-snmp6
match access-group name copp-system-p-acl-tftp6
match access-group name copp-system-p-acl-radius
match access-group name copp-system-p-acl-tacacs
match access-group name copp-system-p-acl-telnet
match access-group name copp-system-p-acl-radius6
match access-group name copp-system-p-acl-tacacs6
match access-group name copp-system-p-acl-telnet6
set cos 2
police cir 3000 pps , bc 512000 packets
module 1 :
transmitted 41723 packets;
dropped 0 packets;

class-map copp-system-p-class-monitoring (match-any)
match access-group name copp-system-p-acl-icmp
match access-group name copp-system-p-acl-icmp6
match access-group name copp-system-p-acl-traceroute
set cos 1
police cir 300 pps , bc 128 packets
module 1 :
transmitted 12341 packets;
dropped 0 packets;

class-map copp-system-p-class-l2-unpoliced (match-any)
match access-group name copp-system-p-acl-mac-stp
match access-group name copp-system-p-acl-mac-lacp
match access-group name copp-system-p-acl-mac-cfsoe
match access-group name copp-system-p-acl-mac-sdp-srp
match access-group name copp-system-p-acl-mac-l2-tunnel
match access-group name copp-system-p-acl-mac-cdp-udld-vtp
set cos 7
police cir 20000 pps , bc 8192 packets
module 1 :
transmitted 447763 packets;
dropped 0 packets;

class-map copp-system-p-class-undesirable (match-any)
match access-group name copp-system-p-acl-undesirable
match exception multicast sg-rpf-failure
set cos 0
police cir 15 pps , bc 32 packets
module 1 :
transmitted 5 packets;
dropped 0 packets;

class-map copp-system-p-class-fcoe (match-any)
match access-group name copp-system-p-acl-mac-fcoe
set cos 6
police cir 1500 pps , bc 128 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-nat-flow (match-any)
match exception nat-flow
set cos 7
police cir 100 pps , bc 64 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-l3mcv6-data (match-any)
match exception multicast ipv6-rpf-failure
match exception multicast ipv6-dest-miss
set cos 1
police cir 3000 pps , bc 32 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-undesirablev6 (match-any)
match exception multicast ipv6-sg-rpf-failure
set cos 0
police cir 15 pps , bc 32 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map copp-system-p-class-l2-default (match-any)
match access-group name copp-system-p-acl-mac-undesirable
set cos 0
police cir 50 pps , bc 32 packets
module 1 :
transmitted 0 packets;
dropped 0 packets;

class-map class-default (match-any)
set cos 0
police cir 50 pps , bc 32 packets
module 1 :
transmitted 3618431 packets;
dropped 214429 packets;

Peter Paluch · ‎06-04-2018

Austin,

Can you please try configuring ip access-list match-local-traffic in the global configuration mode of this switch, and give it another try?

It seems that we have a limitation on the particular ASIC used by this switch where a Routed (RACL) is not able to override the copy-2-CPU mandated by FIB. The ip access-list match-local-traffic has two consequences: It will obviously make the ACLs apply to locally originated traffic (which we're not really interested in here but that's the primary purpose of the command), and - what is interesting for us - it will place a software copy of ACLs to the path of CPU-bound packets which should make the ACL behave as expected since it will also be executed in software.

Please give this a try and let us know. Thank you!

Best regards,
Peter

hayesa · ‎06-07-2018

I'm going to open a ticket with TAC to see if they can figure this out. Keep the ideas coming though if anyone else thinks they might have a solution.

I tried adding the ip access-list match-local-traffic command but I'm still getting the same behavior. The access list is logging the matches but my pings are still making it through.

If TAC comes up with anything I'll post it here.

Reza Sharifi · ‎06-07-2018

Maybe you have already done this but for testing, can you change the command and add "echo and echo-reply" to the statement?

deny icmp any any echo-reply

Also, have you tried testing this with an SVI instead of a routed port?

HTH