Solved: Re: Nexus 9K, Having trouble Implementing/Creating/Applying ACL's - Page 2

TheGoob · ‎01-09-2024

Hello

I have a Nexus 9K w/6 vlans

GE 1/1-1/8

vlan1

vlan Interface 192.168.1.1 255.255.255.0

no shut

GE 1/9-1/16

vlan2

vlan Interface 192.168.2.1 255.255.255.0

no shut

GE 1/17-1/24

vlan3

vlan Interface 192.168.3.1 255.255.255.0

no shut

And so on for all 6.

I have ip route 0.0.0.0 0.0.0.0 192.168.1.2 [it's WAN Gateway] and so on to 192.168.3.2 [it's WAN Gateway] but have no issues with each vlan hitting the internet. My issues is I can not get any vlan to allow connectivity to another vlan, more specifically a host on one vlan can't communicate with a host on another. On the Nexus, I can Ping, from 1 vlan to another and back etc. So it is "there" but can not pass data. At this stage, I am fine with opening up ALL ACCESS ALLOW just to get my servers etc up and running.

My only familiarity with ACL's [manually] were my ASA-5508 and FPR1010, but it has been ages doing them CLI. I even tried using same formats as I had on those system but to no avail. I cant not find much reference on the subject, at least that I can understand/relate to.

I only just learned to 'feature interface vlan' so I am sure I am missing things as far as ACL's go. I looked into IP ACL and VLAN ACL, and nothing seems to click.

TheGoob · ‎01-10-2024

Yeah the output may be funky, but at this time just working with vlan4 and vlan2.. My Windows is, in theory, vlan1, but for testing purposes I move it to the vlan 4 [192.168.3.0] Network to try. No, nothing I do can get any vlan to talk to any vlan.. When I put it all back on the FPR and use Nexus as L2, all works fine. It is just something weird on the Nexus I am missing.When I revert back to Nexus being L2 only, and my FPR doing all the ACL/NAT, Windows can SSH perfectly to it.

Christopher_Starcher · ‎01-10-2024

Ok, that sounds like the FPR is where the ACL should be, then. If things work when the 9k is only doing L2, then it sounds like a L3 problem.

If it were my network, I would create another VLAN for the WAN and default route to the FPR and then try again.

Just use one default route for a network between the 9k and the FPR. That can be whatever address range you want to use, just keep the 192.168.0.0/16 networks for your local VLANs on the 9k. Remove the 192.168.x.x interfaces and networks from the FPR.

Once the 9k 'owns' the 192.168.x.x networks, then I suspect this will work. I could be wrong, but a routing scenario is why I was trying to figure out where the SVI's were.

In other words, create a L3 subnet between the 9k and the FPR, using L3 interfaces instead of VLANs and then have one default route to the FPR's address. The rest of the 9k configuration should be ok to leave in place. You should only have local networks for the VLANs on the 9k and one network between the 9k and the FPR, with a default route using the FPR's IP address of that L3 connection.

Once that's all in place, the 9k should own those subnets and I suspect the ACL will then work.

TheGoob · ‎01-10-2024

Well this is trailing back around to my prior/other thread.

My FPR is 1Gbps each Interface. I have 6 STATIC WAN IP's I wanted to assign to each Interface and their own vlan/subnet. I got the Nexus and was using it for L2 purposes only, thinking ok cool everything between vlans will run at 10Gbps [as I paid the hefty price of making every host of importance a 10Gbps NIC] only to realize no no no, if Nexus is L2, unfortunately all 10Gbps will drop down to the FPR 1Gbps throughput, making Nexus 10Gbps and NIC's at 10Gbps irrelevant.

Our conclusion was, leave ALL NAT/ ACL (WAN to LAN) in place, but all lan to lan ACL's become inactive because we are moving the "routing" to the Nexus so my vlans could then utilize the 10Gbps throughput via new [Nexus based] ACL's, but then have static routes back to the FPR for Internet access. Also, because Nexus vlans can not do DHCP, I left the DHCP Server on the FPR. It appears everything works as it should except lan to lan Nexus vlan routing.

So either this thinking is not able to work, or maybe I do need NAT on the Nexus for lan to lan or I just am not sure.

Also, so I cna clarify what your meaning is. On FPR remove all vlans. Create 1 Interface as an Interface [no vlan] of 192.168.8.1 and then on Nexus create an INTERFACE SVI, not a vlan SVI and assign it 192.168.8.2. Now Nexus and FPR communicate via those 2 Interfaces, 192.168.8.0. I create a static route 0.0.0.0 0.0.0.0 192.168.8.1 so all "internet" Traffic knows to go out.

But now how does the FPR still communcate with all my NAT to NAT 1:1, My NAT Port Forwarding etc, would I not need to now create a static route, on FPR to Nexus, 192.168.1.0 192.168.8.1, 192.168.2.0 192.168.8.1 and so on?

Christopher_Starcher · ‎01-10-2024

I guess this scenario is more involved than just getting an ACL to work on a 9k. It sounds like a high level design needs to be done.

NAT would only be needed on the device that is connected to the WAN. If that's your FPR, then your FPR would need to provide NAT for anything passing from internal networks to the public.

I was happy to try and get an ACL working for you. But, putting together a combination of features to satisfy a design is a bit more than I bargained for, to be honest.

Do you have a Cisco partner you can work with on this? If not, you could open a TAC Case with Cisco for configuration assistance and see how far they are willing to help out.

TheGoob · ‎01-10-2024

I feel as if this whole thing got way too complicated and does not need to me. I will approach it as thus, and see if maybe a solution can be isolated and simplified.

I have a home LAN Workshop. I have No Internet, only a NEXUS 9K w/ 6 vlans, and I want each vlan to talk to and access all data on each other vlan [hosts]. Therefore, I need ACL's.

interface Vlan1
no shutdown
ip address 192.168.1.1/24
   GE 1/1 - GE1/8 [vlan1]

interface Vlan2
no shutdown
ip address 192.168.2.1/24
   GE 1/9 - GE 1/16 [vlan2]

interface Vlan3
no shutdown
ip address 192.168.3.1/24
   GE 1/17 - GE 1/24 [vlan3]
 
interface Vlan4
no shutdown
ip address 192.168.4.1/24
   GE 1/25 - GE 1/32 [vlan4]

interface Vlan5
no shutdown
ip address 192.168.5.1/24
   GE 1/33 - GE 1/40 [vlan5]

interface Vlan6
no shutdown
ip address 192.168.6.1/24
   GE 1/41 - GE 1/48 [vlan6]

I will refrain from the multitude of ACL's I have tried, but nothing I do will allow any Host on and vlan to be able to communicate on any Host on any other vlan.

Any possible examples out there or assistance? I did indeed refer to the documentation, and can even post here what I have found [i looked into IP ACL and VACL] and neither have any suggestion at all as to what I am trying to do. I have "some" familiarity with doing ACL's in CLI on FPR1010 and some of my older Catalysts, neither of those formats seems to do me any good.

Everyone else getting me this far has been a blessing, but I seem to simply be stuck on 6 vlans, all on same Nexus, being unable to communicate.

NOTE; When I got this Nexus, because at the time I simply did want just L2 capability, i changed everything to L2... Would this have disabled/removed routing capability as well and needs to be re-enabled for the vlan SVI's?

TheGoob · ‎01-10-2024

Any new ideas would be well received.

TheGoob · ‎01-11-2024

Well it seems I just need to add a 2nd 10Gbps NIC in Host that I require to have 10Gbs and have them have their o2n private network to communicate.

hslai · ‎01-13-2024

Consider doing some packet captures to see how far each connection gets to. See Packet capture in N9k

TheGoob · ‎01-13-2024

Hello, well it seems I found the "problem"...

So, as mentioned I have everything set up, could see the Internet but no data flow from vlan to vlan. On, the Nexus, the vlan Interfaces are all .1 IP, 192.168.1.1, 192.168.2.1 and so on. I need [really, really want] to run a DHCP Server, which from what I have read, Nexus does not doe, and so I run it on the FPR1010.. The vlan Interfaces on the FPR are 192.168.1.2, 192.168.2.2 and so on. I found out that when a HOST obtains it's IP address, it is taking the .2 Address as the Gateway!! Which is why nexus vlans can not talk.If I manually input my gateway on each host to x.1, all vlans talk, but now no Internet!

SO, either Nexus HAS to do dhcp or on the FPR I need to figure out a way to have eahc individual vlan/dhcp server to use 'code 3' and input the Nexus Interface vlan IP's as the Gateways or 3rd, install a 2nd 10Gbps NIC in each host and create my own little vlan for them. I was successful with the code 3 on FPR, but it appears I can only do it for the dhcp servers as a whole, I wish I was able to isolate each dhcp server with their own code 3 each.

TheGoob · ‎01-14-2024

Got it working.

6 wan ip's, 6 vlans, FPR to Nexus, Got Nexus set up at a dhcp server for each vlan. Every vlan cans ee each other, connect to each other and connect to the Internet, and best of all, what started it all in 5 different posts in 5 different formats, i transfer from vlan to vlan 700+/- MBps.

Was not an ACL issue, but got it working.