Solved: Re: Nexus 3k route ACL on L3 interface

YFZH · ‎07-25-2023

I'm testing ACL on nexus 3k switches. Very simple setup. Two switches, R1 and R2. Linked with each other over their layer3 interfaces Eth1/1.

Trying to use ACL r1r2 to block the traffic from R1 loopback1 to R2 loopback2 without any luck. No hit showing on the ACL counter. Any ideas are appreciated!

Config as follows:
R1:

ip route 0.0.0.0/0 12.12.12.2
interface Ethernet1/1
  no switchport
  ip address 12.12.12.1/24
  no shutdown
!
interface loopback1
  ip address 1.1.1.1/32

R2:

interface Ethernet1/1
  no switchport
  ip access-group r1r2 in
  ip address 12.12.12.2/24
  no shutdown
!
interface loopback2
  ip address 2.2.2.2/32
!
ip route 0.0.0.0/0 12.12.12.1
!
ip access-list r1r2
  statistics per-entry
  10 deny ip 1.1.1.1/32 2.2.2.2/32
  100 permit ip any any
!

Tried reboot and attching log to the ACL entry. Not seeing any changes. R1 loop1 can still ping R2 loop2, ACL on R2 has no hit, no log.

R1# ping 2.2.2.2 source 1.1.1.1
PING 2.2.2.2 (2.2.2.2) from 1.1.1.1: 56 data bytes
64 bytes from 2.2.2.2: icmp_seq=0 ttl=254 time=3.382 ms
64 bytes from 2.2.2.2: icmp_seq=1 ttl=254 time=2.666 ms
64 bytes from 2.2.2.2: icmp_seq=2 ttl=254 time=2.639 ms
64 bytes from 2.2.2.2: icmp_seq=3 ttl=254 time=2.494 ms
64 bytes from 2.2.2.2: icmp_seq=4 ttl=254 time=2.526 ms

--- 2.2.2.2 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 2.494/2.741/3.382 ms
R1#

Peter Paluch · ‎07-27-2023

Hi @YFZH ,

And yet another update, but this time more serious. I did some internal research, and the

ip access-list match-local-traffic

may have effect on some traffic but not all.

Generally, locally-destined traffic on Nexus 3000 platforms has redirect entries installed to the TCAM to punt that traffic to CPU (if it is destined to one of the local IPs). These redirect TCAM entries are evaluated before the RACL entries, and so the RACL won't get hit.

There is no workaround on this platform possible from the NX-OS itself - this is in fact described in CSCvs40404 I've just found. If you are a

Linux geek

you can possibly enter the bash shell and create iptables rules to block that unwanted locally-destined traffic at the

Linux level

So... in the end... sorry for giving you some false hopes. At least now we know what's going on.

Many thanks to you, and everyone who joined.

Best regards,
Peter

View solution in original post

Flavio Miranda · ‎07-26-2023

Hi @YFZH

Not sure if the output of you command is not showing it but it seems you did not apply the ACL to the interface. Seems to me you only created the ACL.

You should have something like

int lo 2

 ip access-group r1r2 in

Peter Paluch · ‎07-26-2023

Flavio,

The ACL is in fact correctly applied on

R2 Ethernet1/1 inbound

as it should. I am not even sure if applying an ACL on a Loopback interface would ever be effective since packets do not transit Loopback interfaces per se.

What's weird here is that the ACL does not even register hits in the counters as YFZH points out. That makes me wonder if the ACL was properly programmed in hardware - that's why I asked to pull out those extra outputs.

@MHM Cisco World, I suspect that if the RACL TCAM region was not carved, the switch would even reject the ACL right at the time of applying it to an interface, so I would rather wait for more details before trying to configure things blindly.

Best regards,
Peter

MHM Cisco World · ‎07-26-2023

@Peter Paluch I aleady run lab and do TCAM carved same result, so I think RACL is not support in NSK in

gns3vm

other emulator.

MHM Cisco World · ‎07-26-2023

same Lab I use also ACL to VLAN interface and same the traffic not hit the ACL
same lab but this time I dont end the traffic to NSK but to host connect to NSK and apply the ACL to VLAN and it work.
so it seem the issue is the traffic will not hit the ACL for traffic direct to NSK but it hit for traffic pass through the ACL

YFZH · ‎07-26-2023

Thanks, but the goal is to block the traffic on the inbound interface of the device which is Eth1/1. I did have the ACL applied on the in direction of that internface.
See config:

interface Ethernet1/1
  no switchport
  ip access-group r1r2 in
  ip address 12.12.12.2/24
  no shutdown
!

Peter Paluch · ‎07-26-2023

Hello,

This is a little weird indeed.

Would you be so kind to answer a few questions for me?

1) Is this a virtual environment or are those true physical switches?

2) What is the exact NX-OS version and platform?

3) Please share the outputs of the following commands from R2:

show hardware access-list tcam region
show hardware access-list interface e1/1 input entries detail

Best regards,
Peter

YFZH · ‎07-26-2023

Hi Peter, thanks for checking this.
The information above was in CML, so it is virtual. Used

N9Kv

in the LAB. Tried attached the ACL to SVI and it doesn't do anything either which makes me thinking it could be some issue with the virtual LAB itself.

This is all about fix the ACL problem in our production. Tested this again on our real physical environment, a bit different compare with the LAB, we saw some hits, but it didn't block the traffic.
N3K-C3048TP-1GE, Version 7.0(3)I7(7).
The command show hardware

access-list tcam

region isn't supported on the switch.
And show hardware

access-list interface ex/x

input entries detail gives the details of the ACL with some hits. Sorry I had to replaced the real IP for security reason. But as you can see the second deny entry got 8 hits.

R2# show hardware access-list interface e1/47 input entries detail

slot 1
=======


Flags: F - Fragment entry E - Port Expansion
D - DSCP Expansion M - ACL Expansion
T - Cross Feature Merge Expansion
N - NS Transit B - BCM Expansion C - COPP


INSTANCE 0x0
---------------

Tcam 2 resource usage:
----------------------
LBL C = 0x1
Bank 0
------
IPv4 Class
Policies: RACL(ACL_V4_INTERNET_IN)
Netflow profile: 0
Netflow deny profile: 0
Entries:
[Index] Entry [Stats]
---------------------
[0x0000:0x0014:0x0014] deny ip $ipaddr $ipaddr [0]
[0x0001:0x0015:0x0015] deny ip $ipaddr $ipaddr [8]
[0x0002:0x0016:0x0016] permit ip $ipaddr $ipaddr [0]
[0x0003:0x0017:0x0017] permit ip $ipaddr $ipaddr [0]
[0x0004:0x0018:0x0018] permit ip $ipaddr $ipaddr [0]
[0x0005:0x0019:0x0019] permit ip $ipaddr $ipaddr [0]
[0x0006:0x001a:0x001a] permit ip $ipaddr $ipaddr [0]
[0x0007:0x001b:0x001b] permit ip $ipaddr $ipaddr [0]
[0x0008:0x001c:0x001c] permit ip $ipaddr $ipaddr [0]
[0x0009:0x001d:0x001d] permit ip $ipaddr $ipaddr [0]
[0x000a:0x001e:0x001e] permit ip $ipaddr $ipaddr [0]
[0x000b:0x001f:0x001f] permit ip $ipaddr $ipaddr [0]
[0x000c:0x0020:0x0020] permit ip $ipaddr $ipaddr [0]
[0x000d:0x0021:0x0021] permit ip $ipaddr $ipaddr [0]
[0x000e:0x0022:0x0022] permit ip $ipaddr $ipaddr [0]
[0x000f:0x0023:0x0023] permit ip $ipaddr $ipaddr [0]
[0x0010:0x0024:0x0024] permit ip $ipaddr $ipaddr [0]
[0x0011:0x0025:0x0025] permit ip $ipaddr $ipaddr [0]
[0x0012:0x0026:0x0026] permit ip $ipaddr $ipaddr [0]
[0x0013:0x0027:0x0027] permit ip $ipaddr $ipaddr [0]
[0x0014:0x0028:0x0028] permit ip $ipaddr $ipaddr [0]
[0x0015:0x0029:0x0029] permit ip $ipaddr $ipaddr [0]
[0x0016:0x002a:0x002a] permit ip $ipaddr $ipaddr [0]
[0x0017:0x002b:0x002b] permit ip $ipaddr $ipaddr [0]
[0x0018:0x002c:0x002c] permit ip $ipaddr $ipaddr [0]
[0x0019:0x002d:0x002d] permit ip $ipaddr $ipaddr [0]
[0x001a:0x002e:0x002e] permit ip $ipaddr $ipaddr [0]
[0x001b:0x002f:0x002f] permit ip $ipaddr $ipaddr [0]
[0x001c:0x0030:0x0030] permit ip $ipaddr $ipaddr [0]
[0x001d:0x0031:0x0031] permit ip $ipaddr $ipaddr [0]
[0x001e:0x0032:0x0032] permit ip $ipaddr $ipaddr [0]
[0x001f:0x0033:0x0033] permit ip $ipaddr $ipaddr [0]
[0x0020:0x0034:0x0034] permit ip $ipaddr $ipaddr [0]
[0x0021:0x0035:0x0035] permit ip $ipaddr $ipaddr [0]
[0x0022:0x0036:0x0036] permit ip $ipaddr $ipaddr [0]
[0x0023:0x0037:0x0037] permit ip $ipaddr $ipaddr [0]
[0x0024:0x0038:0x0038] permit ip $ipaddr $ipaddr [0]
[0x0025:0x0039:0x0039] permit ip $ipaddr $ipaddr [0]
[0x0026:0x003a:0x003a] permit ip $ipaddr $ipaddr [0]
[0x0027:0x003b:0x003b] permit ip $ipaddr $ipaddr [0]
[0x0028:0x003c:0x003c] permit ip $ipaddr $ipaddr [0]
[0x0029:0x003d:0x003d] permit ip $ipaddr $ipaddr [0]
[0x002a:0x003e:0x003e] permit ip $ipaddr $ipaddr [0]
[0x002b:0x003f:0x003f] permit ip $ipaddr $ipaddr [0]
[0x002c:0x0040:0x0040] permit ip $ipaddr $ipaddr [0]
[0x002d:0x0041:0x0041] permit ip $ipaddr $ipaddr [0]
[0x002e:0x0042:0x0042] permit ip $ipaddr $ipaddr [0]
[0x002f:0x0043:0x0043] permit ip $ipaddr $ipaddr [0]
[0x0030:0x0044:0x0044] permit ip $ipaddr $ipaddr [0]
[0x0031:0x0045:0x0045] permit ip $ipaddr $ipaddr [0]
[0x0032:0x0046:0x0046] permit ip $ipaddr $ipaddr [0]
[0x0033:0x0047:0x0047] permit ip $ipaddr $ipaddr [0]
[0x0034:0x0048:0x0048] permit ip $ipaddr $ipaddr [0]
[0x0035:0x0049:0x0049] permit ip $ipaddr $ipaddr [0]
[0x0036:0x004a:0x004a] permit ip $ipaddr $ipaddr [0]
[0x0037:0x004b:0x004b] permit ip $ipaddr $ipaddr [0]
[0x0038:0x004c:0x004c] permit ip $ipaddr $ipaddr [0]
[0x0039:0x004d:0x004d] permit ip $ipaddr $ipaddr [0]
[0x003a:0x004e:0x004e] permit ip $ipaddr $ipaddr [0]
[0x003b:0x004f:0x004f] permit ip $ipaddr $ipaddr [0]
[0x003c:0x0050:0x0050] permit ip $ipaddr $ipaddr [0]
[0x003d:0x0051:0x0051] permit ip $ipaddr $ipaddr [0]
[0x003e:0x0052:0x0052] permit ip $ipaddr $ipaddr [0]
[0x003f:0x0053:0x0053] permit ip $ipaddr $ipaddr [0]

INSTANCE 0x0
---------------

[0x0040:0x0054:0x0054] permit ip $ipaddr $ipaddr [0]
[0x0041:0x0055:0x0055] permit ip $ipaddr $ipaddr [0]
[0x0042:0x0056:0x0056] permit ip $ipaddr $ipaddr [0]
[0x0043:0x0057:0x0057] permit ip $ipaddr $ipaddr [0]
[0x0044:0x0058:0x0058] permit ip $ipaddr $ipaddr [0]
[0x0045:0x0059:0x0059] permit ip $ipaddr $ipaddr [0]
[0x0046:0x005a:0x005a] permit ip $ipaddr $ipaddr [0]
[0x0047:0x005b:0x005b] permit ip $ipaddr $ipaddr [0]
[0x0048:0x005c:0x005c] permit ip $ipaddr $ipaddr [0]
[0x0049:0x005d:0x005d] permit ip $ipaddr $ipaddr [0]
[0x004a:0x005e:0x005e] permit ip $ipaddr $ipaddr [0]
[0x004b:0x005f:0x005f] permit ip $ipaddr $ipaddr [0]
[0x004c:0x0060:0x0060] permit ip $ipaddr $ipaddr [0]
[0x004d:0x0061:0x0061] permit ip $ipaddr $ipaddr [0]
[0x004e:0x0062:0x0062] permit ip $ipaddr $ipaddr [0]
[0x004f:0x0063:0x0063] permit ip $ipaddr $ipaddr [0]
[0x0050:0x0064:0x0064] permit ip $ipaddr $ipaddr [0]
[0x0051:0x0065:0x0065] permit ip $ipaddr $ipaddr [0]
[0x0052:0x0066:0x0066] permit ip $ipaddr $ipaddr [0]
[0x0053:0x0067:0x0067] permit ip $ipaddr $ipaddr [0]
[0x0054:0x0068:0x0068] permit ip $ipaddr $ipaddr [0]
[0x0055:0x0069:0x0069] permit ip $ipaddr $ipaddr [0]
[0x0056:0x006a:0x006a] permit ip $ipaddr $ipaddr [0]
[0x0057:0x006b:0x006b] permit ip $ipaddr $ipaddr [0]
[0x0058:0x006c:0x006c] permit ip $ipaddr $ipaddr [0]
[0x0059:0x006d:0x006d] permit ip $ipaddr $ipaddr [0]
[0x005a:0x006e:0x006e] permit ip $ipaddr $ipaddr [0]
[0x005b:0x006f:0x006f] permit ip $ipaddr $ipaddr [0]
[0x005c:0x0070:0x0070] permit ip $ipaddr $ipaddr [0]
[0x005d:0x0071:0x0071] permit ip $ipaddr $ipaddr [0]
[0x005e:0x0072:0x0072] permit ip $ipaddr $ipaddr [0]
[0x005f:0x0073:0x0073] permit ip $ipaddr $ipaddr [0]
[0x0060:0x0074:0x0074] permit ip $ipaddr $ipaddr [0]
[0x0061:0x0075:0x0075] permit ip $ipaddr $ipaddr [0]
[0x0062:0x0076:0x0076] permit ip $ipaddr $ipaddr [0]
[0x0063:0x0077:0x0077] permit ip $ipaddr $ipaddr [0]
[0x0064:0x0078:0x0078] permit ip $ipaddr $ipaddr [0]
[0x0065:0x0079:0x0079] permit ip $ipaddr $ipaddr [0]
[0x0066:0x007a:0x007a] permit ip $ipaddr $ipaddr [0]
[0x0067:0x007b:0x007b] permit ip $ipaddr $ipaddr [0]
[0x0068:0x007c:0x007c] permit ip $ipaddr $ipaddr [0]
[0x0069:0x007d:0x007d] permit ip $ipaddr $ipaddr [0]
[0x006a:0x007e:0x007e] permit ip $ipaddr $ipaddr [0]
[0x006b:0x007f:0x007f] permit ip $ipaddr $ipaddr [0]
[0x006c:0x0080:0x0080] permit ip $ipaddr $ipaddr [0]
[0x006d:0x0081:0x0081] permit ip $ipaddr $ipaddr [0]
[0x006e:0x0082:0x0082] permit ip $ipaddr $ipaddr [0]
[0x006f:0x0083:0x0083] permit ip $ipaddr $ipaddr [0]
[0x0070:0x0084:0x0084] permit ip $ipaddr $ipaddr [0]
[0x0071:0x0085:0x0085] permit ip $ipaddr $ipaddr [0]
[0x0072:0x0086:0x0086] permit ip $ipaddr $ipaddr [0]
[0x0073:0x0087:0x0087] permit ip $ipaddr $ipaddr [0]
[0x0074:0x0088:0x0088] permit ip $ipaddr $ipaddr [0]
[0x0075:0x0089:0x0089] permit ip $ipaddr $ipaddr [0]
[0x0076:0x008a:0x008a] permit ip $ipaddr $ipaddr [0]
[0x0077:0x008b:0x008b] permit ip $ipaddr $ipaddr [0]
[0x0078:0x008c:0x008c] permit ip $ipaddr $ipaddr [0]
[0x0079:0x008d:0x008d] permit ip $ipaddr $ipaddr [0]
[0x007a:0x008e:0x008e] permit ip $ipaddr $ipaddr [0]
[0x007b:0x008f:0x008f] permit ip $ipaddr $ipaddr [0]
[0x007c:0x0090:0x0090] permit ip $ipaddr $ipaddr [0]
[0x007d:0x0091:0x0091] permit ip $ipaddr $ipaddr [0]
[0x007e:0x0092:0x0092] permit ip $ipaddr $ipaddr [0]
[0x007f:0x0093:0x0093] permit ip $ipaddr $ipaddr [0]

INSTANCE 0x0
---------------

[0x0080:0x0094:0x0094] permit ip $ipaddr $ipaddr [0]
[0x0081:0x0095:0x0095] permit ip $ipaddr $ipaddr [0]
[0x0082:0x0096:0x0096] permit ip $ipaddr $ipaddr [0]
[0x0083:0x0097:0x0097] permit ip $ipaddr $ipaddr [0]
[0x0084:0x0098:0x0098] permit ip $ipaddr $ipaddr [0]
[0x0085:0x0099:0x0099] permit ip $ipaddr $ipaddr [0]
[0x0086:0x009a:0x009a] permit ip $ipaddr $ipaddr [0]
[0x0087:0x009b:0x009b] permit ip $ipaddr $ipaddr [0]
[0x0088:0x009c:0x009c] permit ip $ipaddr $ipaddr [0]
[0x0089:0x009d:0x009d] permit ip $ipaddr $ipaddr [0]
[0x008a:0x009e:0x009e] permit ip $ipaddr $ipaddr [0]
[0x008b:0x009f:0x009f] permit ip $ipaddr $ipaddr [0]
[0x008c:0x00a0:0x00a0] permit ip $ipaddr $ipaddr [0]
[0x008d:0x00a1:0x00a1] permit ip $ipaddr $ipaddr [0]
[0x008e:0x00a2:0x00a2] permit ip $ipaddr $ipaddr [0]
[0x008f:0x00a3:0x00a3] permit ip $ipaddr $ipaddr [0]
[0x0090:0x00a4:0x00a4] permit ip $ipaddr $ipaddr [0]
[0x0091:0x00a5:0x00a5] permit ip $ipaddr $ipaddr [0]
[0x0092:0x00a6:0x00a6] permit ip $ipaddr $ipaddr [0]
[0x0093:0x00a7:0x00a7] permit ip $ipaddr $ipaddr [0]
[0x0094:0x00a8:0x00a8] permit ip $ipaddr $ipaddr [0]
[0x0095:0x00a9:0x00a9] permit ip $ipaddr $ipaddr [0]
[0x0096:0x00aa:0x00aa] permit ip $ipaddr $ipaddr [0]
[0x0097:0x00ab:0x00ab] permit ip $ipaddr $ipaddr [0]
[0x0098:0x00ac:0x00ac] permit ip $ipaddr $ipaddr [0]
[0x0099:0x00ad:0x00ad] permit ip $ipaddr $ipaddr [0]
[0x009a:0x00ae:0x00ae] permit ip $ipaddr $ipaddr [0]
[0x009b:0x00af:0x00af] permit ip $ipaddr $ipaddr [0]
[0x009c:0x00b0:0x00b0] permit ip $ipaddr $ipaddr [0]
[0x009d:0x00b1:0x00b1] permit ip $ipaddr $ipaddr [0]
[0x009e:0x00b2:0x00b2] permit ip $ipaddr $ipaddr [0]
[0x009f:0x00b3:0x00b3] permit ip $ipaddr $ipaddr [0]
[0x00a0:0x00b4:0x00b4] permit ip $ipaddr $ipaddr [0]
[0x00a1:0x00b5:0x00b5] permit ip $ipaddr $ipaddr [0]
[0x00a2:0x00b6:0x00b6] permit ip $ipaddr $ipaddr [0]
[0x00a3:0x00b7:0x00b7] permit ip $ipaddr $ipaddr [0]
[0x00a4:0x00b8:0x00b8] permit ip $ipaddr $ipaddr [0]
[0x00a5:0x00b9:0x00b9] permit ip $ipaddr $ipaddr [0]
[0x00a6:0x00ba:0x00ba] permit ip $ipaddr $ipaddr [0]
[0x00a7:0x00bb:0x00bb] permit ip $ipaddr $ipaddr [0]
[0x00a8:0x00bc:0x00bc] permit ip $ipaddr $ipaddr [0]
[0x00a9:0x00bd:0x00bd] permit ip $ipaddr $ipaddr [0]
[0x00aa:0x00be:0x00be] permit ip $ipaddr $ipaddr [0]
[0x00ab:0x00bf:0x00bf] permit ip $ipaddr $ipaddr [0]
[0x00ac:0x00c0:0x00c0] permit ip $ipaddr $ipaddr [0]
[0x00ad:0x00c1:0x00c1] permit ip $ipaddr $ipaddr [0]
[0x00ae:0x00c2:0x00c2] permit ip $ipaddr $ipaddr [0]
[0x00af:0x00c3:0x00c3] permit ip $ipaddr $ipaddr [0]
[0x00b0:0x00c4:0x00c4] permit ip $ipaddr $ipaddr [0]
[0x00b1:0x00c5:0x00c5] permit ip $ipaddr $ipaddr [0]
[0x00b2:0x00c6:0x00c6] permit ip $ipaddr $ipaddr [0]
[0x00b3:0x00c7:0x00c7] permit ip $ipaddr $ipaddr [0]
[0x00b4:0x00c8:0x00c8] permit ip $ipaddr $ipaddr [0]
[0x00b5:0x00c9:0x00c9] permit ip $ipaddr $ipaddr [0]
[0x00b6:0x00ca:0x00ca] permit ip $ipaddr $ipaddr [0]
[0x00b7:0x00cb:0x00cb] permit ip $ipaddr $ipaddr [0]
[0x00b8:0x00cc:0x00cc] permit ip $ipaddr $ipaddr [0]
[0x00b9:0x00cd:0x00cd] permit ip $ipaddr $ipaddr [0]
[0x00ba:0x00ce:0x00ce] permit ip $ipaddr $ipaddr [0]
[0x00bb:0x00cf:0x00cf] permit ip $ipaddr $ipaddr [0]
[0x00bc:0x00d0:0x00d0] permit ip $ipaddr $ipaddr [0]
[0x00bd:0x00d1:0x00d1] permit ip $ipaddr $ipaddr [0]
[0x00be:0x00d2:0x00d2] permit ip $ipaddr $ipaddr [0]
[0x00bf:0x00d3:0x00d3] permit ip $ipaddr $ipaddr [0]

INSTANCE 0x0
---------------

[0x00c0:0x00d4:0x00d4] permit ip $ipaddr $ipaddr [0]
[0x00c1:0x00d5:0x00d5] permit ip $ipaddr $ipaddr [0]
[0x00c2:0x00d6:0x00d6] permit ip $ipaddr $ipaddr [0]
[0x00c3:0x00d7:0x00d7] permit ip $ipaddr $ipaddr [0]
[0x00c4:0x00d8:0x00d8] permit ip $ipaddr $ipaddr [0]
[0x00c5:0x00d9:0x00d9] permit ip $ipaddr $ipaddr [0]
[0x00c6:0x00da:0x00da] permit ip $ipaddr $ipaddr [0]
[0x00c7:0x00db:0x00db] permit ip $ipaddr $ipaddr [0]
[0x00c8:0x00dc:0x00dc] permit ip $ipaddr $ipaddr [0]
[0x00c9:0x00dd:0x00dd] permit ip $ipaddr $ipaddr [0]
[0x00ca:0x00de:0x00de] permit ip $ipaddr $ipaddr [0]
[0x00cb:0x00df:0x00df] permit ip $ipaddr $ipaddr [0]
[0x00cc:0x00e0:0x00e0] permit ip $ipaddr $ipaddr [0]
[0x00cd:0x00e1:0x00e1] permit ip $ipaddr $ipaddr [0]
[0x00ce:0x00e2:0x00e2] permit ip $ipaddr $ipaddr [289]
[0x00cf:0x00e3:0x00e3] permit ip $ipaddr $ipaddr [0]
[0x00d0:0x00e4:0x00e4] permit ip $ipaddr $ipaddr [0]
[0x00d1:0x00e5:0x00e5] permit ip $ipaddr $ipaddr [0]
[0x00d2:0x00e6:0x00e6] permit ip $ipaddr $ipaddr [0]
[0x00d3:0x00e7:0x00e7] permit ip $ipaddr $ipaddr [0]
[0x00d4:0x00e8:0x00e8] permit ip $ipaddr $ipaddr [110]
[0x00d5:0x00e9:0x00e9] deny ip $ipaddr $ipaddr [0]


L4 protocol cam entries usage: none

No mac protocol cam entries are in use

Peter Paluch · ‎07-27-2023

Hello @YFZH ,

Thank you for all those outputs and clarifying the environment this applies to.

Testing these things in CML is prone to behaving differently. That is because the N9Kv VM does not emulate the behavior of the forwarding hardware fully; rather, it approximates it. There may be differences, sadly.

Either way, this might interest you: I labbed up your simple config on hardware N3048 running 7.0(3)I7(5), I was able to reproduce the issue you have had, and then I realized I'm getting old and rusty... RACLs on Nexus switches apply to transit traffic but not always necessarily to traffic that is terminated or originated by the switch itself. If they should apply to that traffic as well, you must configure this:

ip access-list match-local-traffic

This will cause the ACLs on the interfaces apply equally to transit traffic as well as traffic that is destined to the switch or sourced from it.

If you want to try this out, I strongly advise you to have a safe time period for that - a maintenance window, or at least out of business hours. Just in case it has unforeseen side effects with your particular configuration, we don't want those to show up in prime time.

Best regards,
Peter

Peter Paluch · ‎07-27-2023

Hi @YFZH ,

And yet another update, but this time more serious. I did some internal research, and the

ip access-list match-local-traffic

may have effect on some traffic but not all.

Generally, locally-destined traffic on Nexus 3000 platforms has redirect entries installed to the TCAM to punt that traffic to CPU (if it is destined to one of the local IPs). These redirect TCAM entries are evaluated before the RACL entries, and so the RACL won't get hit.

There is no workaround on this platform possible from the NX-OS itself - this is in fact described in CSCvs40404 I've just found. If you are a

Linux geek

you can possibly enter the bash shell and create iptables rules to block that unwanted locally-destined traffic at the

Linux level

So... in the end... sorry for giving you some false hopes. At least now we know what's going on.

Many thanks to you, and everyone who joined.

Best regards,
Peter

YFZH · ‎07-27-2023

Hi Peter,
Thanks heaps for the lab and research on this!
My target was creating an infrastructure ACL to stop internet traffic towards the IPs on the network devices. Some of our sites has ASR as edge and it went very smoothly. Only when it came to this site using Nexus 3000 as the edge router, things started getting odd. Any suggestions that I can make the similar function as the infrastructure ACL that stops connection from outside to the IP on the switch and permit the traffic towards the hosts attaching to the switch?
I will definitely test the iptable on nexus, but probably won't make it in production as it might be a bit too difficult for operations, config backup for example.

Cheers!
Yifan

Peter Paluch · ‎07-28-2023

Hi Yifan,

Unfortunately, we are facing limitations of the N3048 platform here. This switch simply won't be able to protect itself using RACLs - this is due to the combined architecture of the switching ASIC and NX-OS. I've tested 9.3(9) whch is relatively recent but there's no change in behavior.

Not even the CoPP would work here because the first-gen N3K switches do not support source-based CoPP, so you wouldn't be able to distinguish between management traffic from legit and the other sources.

I'm afraid that the

Linux iptables

remain the only option but that one is admittedly a hack. But it might be possible to store a shell script on the bootflash of the switch that would be executed after a reload or an upgrade.

I have to point out that the N3048 is a data center access layer switch and in the role of an edge router, it is seriously stretched... but you probably know that (by now : ) ).

Best regards,
Peter

YFZH · ‎07-30-2023

Hi Peter,

Really appreciate for your input here. I couldn't agree more, probably it time of considering a different model for the role of edge router.
Thank you!

Best regards,
Yifan

MHM Cisco World · ‎07-30-2023

even if you solve ACL what about NATing ?
Nexus with NAT is not so optimal.

use

ASR1000 or ISR4k

is better for edge router.

YFZH · ‎07-30-2023

Thanks. We don't NAT on edge, so that is not a problem. But totally agree, ASR/ISR is a much better choice.