Achieving 10Gbps LAN Speed (or relatively close)

[TheGoob]

Hi there

So, I have an FPR1010 which is the 1Gbps Interfaces.. Each Interface [using 7 of them] have their own vlan. Each interface is connected to the same Nexus 9K Series Switch, all L2, and their own vlans accordingly.

So, various endpoints connect to their respective Interfaces on Nexus which are associated with their own vlans in relation to the FPR1010 vlans.

Long story short... EVERYTHING connects on the Nexus, which are all 10Gbps Interfaces... But, will vlan1 communicate with vlan2,3,4,5 or 6 at 10Gbps [or relative] or does it drop down to 1Gbps because the "routing" is done on the 1Gbps FPR1010.

I would assume routing logic would dictate the packets never leave the Nexus other than vlan to vlan, but was not sure.

[TheGoob]

Alright, so if I were to decipher that.

FPR;

6 Usable STATIC WAN IP's

'inside' VLAN1, 192.168.1.1

NAT 1 to 1 and Port Forward- as I have them.

Being that each internal subnet has it's own WAN IP, I would do default routes for each 6 of them. So being all subnets on Nexus are connecting through that 1 FPR/Nexus Interface, I would do [on FPR] 192.168.2.0 0.0.0.0 192.168.1.2, which is the Nexus L3 Interface connecting to the FPR? Would it all be the same then like 192.168.3.0 0.0.0.0 192.168.1.2? Clearly I am not understanding that.

I guess my mind is having hard time wrapping around because I do want to keep the 6 subnets, but seems I can't grasp what static routes would be done. I am understand the concept just iffy on implementation. I need to keep reading over your example.

It's like, I do not want this handed to me, but also just at a standstill.

Or wait, did you mean on the Nexus, an L3 for each of the vlans? So on FPR 192.168.2.0 0.0.0.0 192.168.1.2, 192.168.3.0 0.0.0.0 192.168.1.3 and so on?

[Giuseppe Larosa]

Hello @TheGoob ,

>> I would do [on FPR] 192.168.2.0 0.0.0.0 192.168.1.2, which is the Nexus L3 Interface connecting to the FPR? Would it all be the same then like 192.168.3.0 0.0.0.0 192.168.1.2? Clearly I am not understanding that.

Yes using IOS commands you should :

ip route 192.168.2.0 255.255.255.0 192.168.1.2

ip route 192.168.3.0 255.255.255.0 192.168.1.2

note : actually the FP1010 can use a local GUI named FDM and you can configure the static routes on that GUI

up to

ip route 192.168.7.0 255.255.255.0 192.168.1.2

on the Nexus instead:




ip route 0.0.0.0 0.0.0.0 192.168.1.1 

For the NAT you may need  a rule for each internal subnet to have it translated to a different public IP if desired.

Hope to help

Giuseppe

[TheGoob]

Interesting.. So I can have the same route hop for different networks, I did not know this. So yes I can easily do the routing that you mention from FPR to Nexus. On Nexus, I create GE 1/1 as L3 192.168.1.1 which connects to the FPR. I then create, which all vlans on Nexus will use, the route back to the FPR via L3 192.168.1.1 Interface [pp route 0.0.0.0 0.0.0.0 192.168.1.1]

NAT rules on FPR would stay the same [because Outside IP and Inside LAN IP are staying the same, just now being routed via static route]. Are you meaning on the Nexus I would need to create NAT's for the vlans out?

[TheGoob]

Yes, on FPR I have NAT STATIC WAN to LAN NETWORK... Are you suggesting that on the Nexus I would do a NAT of LAN NETWORK [VLAN] to it's STATIC WAN? I guess there would definitely need to be a NAT on the Nexus as well because yeah the Incoming from OUTSIDE uses the NAT on the FPR, but any vlan, though they have their default route out, knows not it's correct WAN IP.

[Joseph W. Doherty]

Why discouraged?

As to confusion, that I can understand, if you don't have lots of experience/knowledge of network engineering.

Nothing at all wrong with each VLAN being its own network.  If fact, that's generally the case.

I'm not familiar with a FPR1010, so cannot say what all the possible options are for doing L3 between it and the Nexus.  Likewise, without more detail how you're using ACLs and NAT, cannot comment on impact of doing L3 on the Nexus.

What I can say, if the Nexus is L3 capable, doing L3 there, transfers between hosts on different VLANs should be as fast as transfers between hosts on the same VLAN, even if they need to be pass through an ACL.  NAT, though, might not be supported on the Nexus.

BTW, with your current topology, not only are inter-VLAN host transfers limited to gig, all VLAN host traffic, to other VLAN hosts, is limited to gig too!

If you can do Etherchannel between the Nexus and FPS1010, in theory, inter-VLAN traffic would have available to it for the aggregate bandwidth of the Etherchannel, but any single flow would still be limited to gig.  (If L3 is done on the Nexus, your bandwidth bottleneck is the fabric bandwidth of the Nexus, would likely can support many, if not all, host ports' bandwidths concurrently.  [Again, for hosts connected to the Nexus, L2, same VLAN, or L3, different VLANs, performance should be the same.])

Again, don't know all your requirements, nor all the features of the FPS1010.  But, if you're looking to improve transfer rates, between hosts on different LAN VLANs, you'll likely need to make some changes.

[TheGoob]

Yeah I suppose discouraged was a bit dramatic, maybe overwhelmed?

As far as my setup, thankfully it is quite generic. I literally have 6 1-to-1 NAT's [WAN to LAN] and then 4-6 NAT's [Port Forward] for Out-to-In access [webs erver, email and simple stuff].

As far as ACL's, I naturally have my inter-vlan access between vlans via Specific Ports etc.. Pretty simple, then of course the Out-To-In ACL's [email, webs erver etc].

So my setup is indeed very basic. It appears that the Nexus currently is simply access ports, all with data flow/limitations of the FPR, which is 1Gb. I will indeed be doing a lot of, hopefully near 10Gb as I can get, among vlans [specific host to specific host] for backup, for just copying and then, I would be installing an app from 1 host on vlan1 to another host ton vlan2, so I want the "install" to cross the network as fast as it can.

It seems the NAT, 1 to 1 also Port forward needs to be done FPR, but would it relieve the drama on the FPR to move the ACL's to the Nexus? Again, I really figure keeping vlan 1-6 on FPR, creating dhcp servers on the vlan interfaces on fpr, on Nexus for each vlan interface make an IP for it's respective network, so vlan 1 interface fpr is 192.168.1.1, vlan 1 interface nexus is 192.168.1.2 and then L2 interfaces would be dhcp from FPR through Nexus routing...And then everything on Nexus would route "locally" not needing to downgrade 10gb to 1gb through FPR. Again, not sure if this is how it works.

I am willing and open to any way that seems most logical. Nothing I have, though I obviously do not want to lose it all, can be erased and remade in a new setup.

[Elliot Dierksen]

I may be a wet blanket here, but just because a device has 8 gigabit interfaces does not mean that is can sustain 8Gb worth of throughput. Here is the spec for a 1010 from the datasheet:

Model	Throughput: Threat Defense Software	IPS Throughput	Interfaces
FPR-1010	890 Mbps	900 Mbps	8 x RJ45

You have to decide on your priority for this use case. Is it securing traffic between VLAN's? If it is, a firewall is a good choice. You do have to pick one that is capable of the throughput level you need. If your priority is throughput between VLAN's, then using the Nexus for layer 3 routing via SVI's would be your best choice.

[TheGoob]

Valid point. As far as FPR1010 goes, I do understand each of the 8 Interfaces are individually 1Gbps (which is still limited by what it connects to, the hard drive speed etc) I by no means meant that the FPR itself would give me any benefit on any port above their individual 1Gbps.

My concern was that every device [of importance] on my various vlans indeed have 10Gbps NIC's and my Nexus Interfaces are all 10Gbps, so my desire was to allow every 10Gbps NIC/Host to be able to transfer data as best to 10Gbps they can [again, with hard drive speed restrictions and so forth and never obtaining true 10gbps] but I come to realize that all routing will still be done via FPR as my [current] Nexus is simply in L2. That is where this all began and it seems that my best bet would be to use the SVI's as suggested, and I am good with this.

I must confess this is for home/fun use but by all means I still want to implement real world configurations even simply for educational purposes, as over the top for me they may be. With that said, I am running multiple email servers, web servers, reverse proxies, dns servers and simple access to my various systems currently [for LAN to LAN] using ACL's based on Ports and then [WAN to LAN] using NAT Port Forward + ACL using Ports. Yeah it is "home" but I still indeed need security but then I do want the most throughput I can achieve.

[paul driver]

Hello

---

@TheGoob wrote:
I come to realize that all routing will still be done via FPR as my [current] Nexus is simply in L2.

---

As stated by others the Nk is L3 compatible so you can have the L3 running on it, if you wish for a solution to be provided so to accomplish inter-vlan 10gb connectivity then suggest you provide as much detail as possible including a topology diagram.

---

[TheGoob]

Well I’ll try to draw out what I HAVE with notes under what I want.. I don’t wanna draw what I want cause if I am wrong on its implementation it’ll just cause chaos. I just don’t know how to draw it out.. I really thought I explained the scenario best I could but I’ll do that in a little bit. Thank you

[Joseph W. Doherty]

BTW, I believe it might be possible to retain much of logical topology and all your physical topology.

It should be possible to have your FPR1010 retain an IP in each VLAN yet allow the Nexus to perform inter-VLAN routing.

Without Etherchannel, what I have mind would better leverage the multiple gig interfaces bandwidth, but from the device capacity specs provided by @Elliot Dierksen that might negate such an advantage.  Still, again, the change might be easier to implement (and possibly easier for you to understand).

Also what I have in mind would not be done, because of the FPR's capacity limitations and/or if Etherchannel could be used.  But, again, if you want to retain much of what you've already done, it might be worth considering.

So what is it?

You define Nexus SVIs to take over the current FPR (gateway) interface IPs.

You provide the FPR interfaces in each VLAN/subnet.

On the Nexus, you define static default routes to each of the FPR's new IPs.  (Optionally, you might configure the SVIs with no redirect.)

If some of your ACLs apply to inter-VLAN traffic, you need to move those to Nexus.

So how does this work?

All off local VLAN traffic goes to same IP, as happens now, but it's now Nexus SVI.  If on another VLAN, Nexus sends it to that subnet.  If non VLAN, Nexus sends traffic to FPR.  As FPR still has interface in each VLAN, its L3 bypasses Nexus for traffic to those VLANs.

[TheGoob]

As I read this it is making sense but still need to go over it a couple times in my head, sort of draw it out, so I can see it as I read it.

[TheGoob]

It should be possible to have your FPR1010 retain an IP in each VLAN yet allow the Nexus to perform inter-VLAN routing.

Does this refer to keeping the vlans configuration as I currently have (along with the DHCP); vlan1 192.168.1.1, vlan2 192.168.2.1 and so on]?

---I suppose I would need to edit my 1-to-1 but instead of x.x.x.177 - to - 192.168.1.0 Dynamic, I would simply create a Static Route, x.x.x.177 255.255.255.0  192.168.1.1 [which would connect to the GE1/1 on the Switch? GAH, That can't be cause I would be making that GE1/1 an SVI using my WAN IP. Or are the Nexus/WAN SVI's going to differ from the links between Nexus and FPR?

You define Nexus SVIs to take over the current FPR (gateway) interface IPs.

So, as an example, GE1/1 [vlan1] I would make an SVI of x.x.x.177 [my 1st WAN IP?]

You provide the FPR interfaces in each VLAN/subnet.

I assume this means keeping the vlan fluent across the devices? So like, 6 Interfaces on FPR would connect to each Nexus vlan? Like I do have currently.

On the Nexus, you define static default routes to each of the FPR's new IPs. (Optionally, you might configure the SVIs with no redirect.)

Yeah I am starting to get ahead of myself.. Cause I have 1.) physical links from FPR #2 vlans with their own networks #3 SVI's. But unsure how they all cohere.

If some of your ACLs apply to inter-VLAN traffic, you need to move those to Nexus.

About the only thing makes most sense so far

So yeah I just need to let this cook in my brain I did want to mention, not every outgoing uses same WAN IP. vlan 1 uses it's own WAN ip, vlan 2 uses it's own WAN IP so on. The different WAN usage isn't just about incoming...I keep all vlans outgoing their own WAN IP too.

---

[Joseph W. Doherty]

Are you doing NAT between VLAN subnets?  Or, only when you go "beyond" the VLAN subnets?

"Does this refer to keeping the vlans configuration as I currently have (along with the DHCP); vlan1 192.168.1.1, vlan2 192.168.2.1 and so on]?"

Possibly.

Basically, in addition to your (6?) FPR (LAN) interfaces you also define a like number of SVIs.  Since (I assume) all your hosts use the FPR interface IPs for gateways, you migrate those IPs to the Nexus SVIs (this so that outbound LAN traffic will start with the Nexus SVIs).  Since you migrated the IPs, the FPR interface need "new" IPs.

What the forging does it define two L3 interfaces on each VLAN, where currently you have just one.  The "new" FPR interface IPs are only use by the Nexus when the destination network is NOT another SVI on the Nexus.

"So, as an example, GE1/1 [vlan1] I would make an SVI of x.x.x.177 [my 1st WAN IP?]"

No, the SVI "takes over" the LAN gateway IP from the FPR.  You would then need to assign another (unused) IP, from the same VLAN subnet to be the "new" FPR interface IP.

For example if VLAN 1 FPR IP is 192.168.1.1 that becomes the SVI 1 IP.  FRP IP might then be 192.168.1.2 (if unused).

"I assume this means keeping the vlan fluent across the devices? So like, 6 Interfaces on FPR would connect to each Nexus vlan? Like I do have currently."

Correct.

"Yeah I am starting to get ahead of myself.. Cause I have 1.) physical links from FPR #2 vlans with their own networks #3 SVI's. But unsure how they all cohere."

Nexus will need IP route statements for networks other that VLAN networks, which might just use default route statements.

E.g. ip route 0.0.0.0 0.0.0.0 192.168.1.2

  (this would support all your VLANs, but you can add additional default route statements to use all the FPR interfaces).

NB: again, if you were designing from scratch, you wouldn't normally do something like the above, but it preserves your existing physical topology and, I believe, requires the least amount of configuration changes.  Also, don't believe doing this, this way, is inherently "bad", again, just unusual.

[TheGoob]

The only NAT that I am doing is on the FPR, and there are only 2 types that I am doing. 1 to 1 Dynamic [WAN to vlan NETWORK] and then WAN to SPECIFIC LAN for Port Forwarding such as Email Servers and WEB Servers. Being that all vlan networks begin on the FPR, that is how all my L2 vlans on the Nexus communicate, as the routing is done on the FPR.

So when you mention that, like the FPR has 6 Interfaces/vlans I do same with SVI.. This is the SVI Interfaces on the Nexus?

So, 6 Interfaces FPR/vlan, 6 SVI Interfaces on Nexus. Are these 6 SVI's each in their own vlan in relation to their network on the FPR? I would assume so. The FPR only mention of the 6 different WAN IP's via NAT, I.E x.x.x.177 Dynamic NAT to 192.168.1.0 [Network]. 192.168.1.0 is vlan1, and GE 1/2 on FPR. On Nexus, GE1/1 is L2-Connected to GE1/2 on FPR and there are 8 Interfaces on the Nexus that are vlan1, which refer back to vlan1 on FPR, which is 192.168.1.0 Network, which is NAT'd to x.x.x.177.

No, the SVI "takes over" the LAN gateway IP from the FPR. You would then need to assign another (unused) IP, from the same VLAN subnet to be the "new" FPR interface IP.

So, then I am to assume that the FPR vlan1 GE1/2 is no longer supplying the DHCP Server, but the FPR 1/2 vlan1 is now obtaining it's IP from the SVI on the Nexus? So we are "hosting" backwards from Nexus SVI to FPR vlan 1? So Nexus SVI vlan 1 would be 192.168.1.1, FPR vlan 1 would be 192.168.1.2, and then the DHCP Server for remaining hosts would be supplied DHCP Server on the Nexus, from the SVI? [This would of course be done 6 times, 6 vlans]. This is how I read the 2nd comment of For example if VLAN 1 FPR IP is 192.168.1.1 that becomes the SVI 1 IP. FRP IP might then be 192.168.1.2 (if unused).  I may have just worded it wrong, but that was how I read that.

E.g. ip route 0.0.0.0 0.0.0.0 192.168.1.2

(this would support all your VLANs, but you can add additional default route statements to use all the FPR interfaces).

I feel I would want to make each vlan/SVI have it's own correct/specific WAN address, in the least the email servers need to have that, but really would like to have it across the board. OUTGOING vlans have their correct/separate WAN IP.

Would those then become 192.168.1.0 255.255.255.0 192.168.1.2, 192.168.2.0 255.255.255.0 192.168.2.2 and so on?

As far as keeping my current structure, the more I think of it, the more I would like to "start from scratch" and do it in a way that in the future, I would have a "known" configuration model because if I had t explain this later, and it not be the norm, I'd not know what to say.  When push comes to shove, this IS for home, so as long as "it works" I am pleased and gracious for. But would if could keep it structured legitimately.

Also, for the FPR and Nexus, I have saved my running configs, so I am not at all against the reset button from scratch, I have a backup OPNSense router/firewall running on a VM I can easily switch too to keep my existing internet up and running while messing around with this.