Achieving 10Gbps LAN Speed (or relatively close)

[TheGoob]

Hi there

So, I have an FPR1010 which is the 1Gbps Interfaces.. Each Interface [using 7 of them] have their own vlan. Each interface is connected to the same Nexus 9K Series Switch, all L2, and their own vlans accordingly.

So, various endpoints connect to their respective Interfaces on Nexus which are associated with their own vlans in relation to the FPR1010 vlans.

Long story short... EVERYTHING connects on the Nexus, which are all 10Gbps Interfaces... But, will vlan1 communicate with vlan2,3,4,5 or 6 at 10Gbps [or relative] or does it drop down to 1Gbps because the "routing" is done on the 1Gbps FPR1010.

I would assume routing logic would dictate the packets never leave the Nexus other than vlan to vlan, but was not sure.

[DanielP211]

Hello!

L3 routing will always pass the FPR so you are limited to 1Gbps. But if you comunicate inside the same broadcast domain (L2) you will have 10Gbps.

BR

[Joseph W. Doherty]

As already noted by @DanielP211 , transit traffic through the FPR1010, for routing between VLANs, would be limited to gig. Traffic on same VLANs shouldn't need to be routed and so not restricted to gig.

You might consider routing on your Nexus, if possible.

[MHM Cisco World]

ANY interface have IP can do fragment and hence it can fragment any traffic have MTU large than it have
if it dont have IP it cannot fragment and traffic drop if MTU is large than MTU config under it 

that ALL 

MHM

[Joseph W. Doherty]

@MHM Cisco World did you post your reply to the correct thread?

[MHM Cisco World]

how the traffic drop down friend ?
it drop down because it pass through L3 interface of FPR and it fragment 
MHM

[Joseph W. Doherty]

Hmm, how do you know this?  I.e. all Nexus traffic is jumbo?

Also, how you know Nexus 10g fragmentation causes a 10x throughput reduction?

I would think most likely cause of 10g slowing to gig would be due to passing through a gig interface.

[Joseph W. Doherty]

Oh, some more MTU considerations.

If Nexus not routing, it would NOT fragment frames.  So, fragmentation would only occur on FPR1010. However if FPR1010 not jumbo capable, how does it receive jumbo frames from the Nexus?

BTW, I do agree, in principle, fragmentation will slow effective throughput rate, at least by bandwidth loss to fragmentation overhead, but don't think that's the primary cause in this case.

[MHM Cisco World]

Not all Sure use jumbo frame 

But if there is decreasing in BW then first point we check mtu.

I think you are correct' maybe I better to include the mtu with other network issue that reduce the BW.

Thanks alot 

MHM

[Joseph W. Doherty]

---

@MHM Cisco World wrote:

But if there is decreasing in BW then first point we check mtu.

Well, if someone describes something like:

host <10g> switch1 <10g> host

and they obtain 10g throughput

but when they have/do

host <10g> switch1 <1g> router1 <1g> switch1 <10g> host

and say they now only obtain gig throughput (which is what I believe the OP describes), personally, I wouldn't consider MTU the first point to consider.

[paul driver]

Hello
If the L3 is on the FPR for all the vlans then inter-vlan communication needs to be forwarded to the FPR via each 1gb interface connected interface  for L2 then no need to route towards the FPR

L3
host  vlan1<10gb>Nexus switch<1gb>FPR (routing decision)
FPR <1gb>Nexus switch<10bg>host vlan 2

L2
host vlan 1<10gb>Nexus switch<10gb>host vlan 1

[TheGoob]

Morning

Alright so I will be truthful, I am sort of discouraged and confused now. 
To simply, on Nexus all Interfaces are L2. I have GE1/1- GE 1-12 vlan 1, GE 1/13 - GE 1/24 vlan 2 and so on. Each vlan (On Nexus GE 1/1 goes to GE 1/2 on FPR 1010, Nexus GE 1/13 goes to GE 1/3 on FPR and so on) so indeed ALL routing is done via FPR1010. 
I would have no issue having the Nexus doing routing, or even PBR, I just sort of liked the fact that each vlan also had its own unique network (192.168.1.0  vlan1, 192.168.2.0 vlan2 and so on  but not at the cost of not being realistic. 
Is there a way to make a L3 Interface for each vlan on the Nexus, which would connect back to its respective port on the FPR1010, and then the remaining Interfaces in each vlan2 be L3 which all routing would be done on Nexus?

Would this change or complicate my ACL’s/NAT’s on the FPR?

[paul driver]

Hello

---

@TheGoob wrote:

I would have no issue having the Nexus doing routing,
Would this change or complicate my ACL’s/NAT’s on the FPR?

---

It would be preferable to have the inter-vlan routing performed on the Nks however however it depends on what the FW is currently doing, it may even be possible to relocate the ACLs also.

As for NAT it will have to stay on the FW but i assume this is for external connection anyway?

[TheGoob]

Hello.

Yes, I have 6 Static WAN IP’s and so for fun, I made a vlan per Static WAN. So all of 192.168.1.0 vlan 1 uses x.x.x.182, all of 192.168.2.0 vlan 2 uses x.x.x.181 and so on. The NAT was for 1-to-1 WAN to LAN (Network). But also of course for Port forwarding to specific LAN IP’s for various access.

Obviously it is more sophisticated than what I suggest but, being the vlans on the FPR are also the DHCP Servers for each vlan, could I assign the vlans on the Nexus their own IP? Like FPR vlan1 is 192.168.1.1 w/DHCP Server. Could I make vlan 1 on Nexus 192.168.1.2 and then all the L2 Interfaces on that vlan would obtain its own 192.168.1.0 IP and then allow inter-vlan routing across the Nexus or does it not work that way? 
Again just a question, please do suggest the reality of it .

[Giuseppe Larosa]

Hello @TheGoob ,

you can move inter VLAN routing to the Nexus enabling the appropriate feature (if needed feature vlan) and then configuring SVIs for each VLAN/subnet on the Nexus itself. Devices will use the VLAN X IP address as their default gateway.

You will need a separate L3 link to the FP1010 in order to route to outer world it is more clean in this way.

The new subnet will be the "inside" of the FP1010.

FP1010 will need static routes for all internal subnets that can be reached via Nexus

the Nexus will need a default static route pointing to the FP1010 IP address in the new subnet.

the NAT configuration of the FP1010 should provide NAT or PAT for all internal subnets that are reached via the Nexus

Hope to help

Giuseppe