Accepted Solutions
Hi,
I think Leo may have an error in his calculation in that he's taken the full-duplex Fabric capacity, but the half-duplex I/O module capacity.
Leo Laohoo wrote:
So for your F248, let's say you fully populate this with 10 Gbps. So this means that you can potentially push up to 480 Gbps. For you to do so you'll need, as a minimum, two FAB-2. With 2 FAB-2, you push up to 440 Gbps.
With two Fabric 2 modules you get 220Gbps of per slot capacity, so your 480Gbps would be slightly higher than 2:1 over-subscribed.
As stated, the Fabric 2 module provides 110Gbps per module, per I/O slot. If you have one Fabric module the maximum per I/O module slot throughput you achieve is 110Gbps, with two it's 220Gbps etc., all way up to 550Gbps with five Fabric 2 modules.
The CiscoLive BRKARC-3470 - Cisco Nexus 7000 Switch Architecture session has details of the Nexus 7000 Fabric architecture which I've taken a couple of slides from to help illustrate.
Each I/O module slot has two traces from its on board fabric ASIC to the chassis Fabric module, and when using Fabric 2 modules, each of these traces will be 55gbps. This is seen in the following slide.
So on the N7K-F248XP-25E I/O module where we have 48 10GE ports to fully utilise the available capacity of the I/O module you need all five Fabric 2 modules installed. This is shown in the next slide from the same presentation.
At the bottom left you can see "Five Fabrics: 480G F2/F2E module has maximum bandwidth".
The kit list you provided has three Fabric 2 modules meaning that any slot in the chassis can get at most 330Gbps fabric capacity. The N7K-F248XP-25E obviously has 48 10GE ports meaning a potential 1.44:1 (480:330) over-subscription.
To add to this, in order to more easily understand the number of Fabric modules required it helps to understand the naming convention of the Nexus I/O modules.
The I/O module name is essentially comprised of a number of fields in the format N7K-ABCCDE-FG, where:
- A = I/O module family e.g., M or F
- B = generation of the module e.g., either 1, 2 or 3
- CC = number of ports on the module e.g., 2, 6, 8, 24, 32 or 48
- D = port speed e.g., G=10/100/1000, X=10G, F=40G or C=100G
- E = connector type e.g., T=RJ45, S=SFP, 2=X2, P=SFP+, Q=QSFP or F=CFP
- F = the Fabric ASIC version of the I/O module e.g., either 1 or 2
- G = the number of Fabric modules required for full line rate I/O
There's also an E or L suffix on certain I/O modules, but we can ignore that in this discussion. Although as Leo has rightfully pointed out in other posts, don't ignore the E when buying F2 modules. Always make sure there's an E at the end
Using the N7K-F248XP-25E as an example, the -25E at the end indicates that the I/O module is Fabric 2 based, and need five fabric modules for full line rate operation.
Hope that clear things up a little.
Regards
Hi,
I think Leo may have an error in his calculation in that he's taken the full-duplex Fabric capacity, but the half-duplex I/O module capacity.
Leo Laohoo wrote:
So for your F248, let's say you fully populate this with 10 Gbps. So this means that you can potentially push up to 480 Gbps. For you to do so you'll need, as a minimum, two FAB-2. With 2 FAB-2, you push up to 440 Gbps.
With two Fabric 2 modules you get 220Gbps of per slot capacity, so your 480Gbps would be slightly higher than 2:1 over-subscribed.
As stated, the Fabric 2 module provides 110Gbps per module, per I/O slot. If you have one Fabric module the maximum per I/O module slot throughput you achieve is 110Gbps, with two it's 220Gbps etc., all way up to 550Gbps with five Fabric 2 modules.
The CiscoLive BRKARC-3470 - Cisco Nexus 7000 Switch Architecture session has details of the Nexus 7000 Fabric architecture which I've taken a couple of slides from to help illustrate.
Each I/O module slot has two traces from its on board fabric ASIC to the chassis Fabric module, and when using Fabric 2 modules, each of these traces will be 55gbps. This is seen in the following slide.
So on the N7K-F248XP-25E I/O module where we have 48 10GE ports to fully utilise the available capacity of the I/O module you need all five Fabric 2 modules installed. This is shown in the next slide from the same presentation.
At the bottom left you can see "Five Fabrics: 480G F2/F2E module has maximum bandwidth".
The kit list you provided has three Fabric 2 modules meaning that any slot in the chassis can get at most 330Gbps fabric capacity. The N7K-F248XP-25E obviously has 48 10GE ports meaning a potential 1.44:1 (480:330) over-subscription.
To add to this, in order to more easily understand the number of Fabric modules required it helps to understand the naming convention of the Nexus I/O modules.
The I/O module name is essentially comprised of a number of fields in the format N7K-ABCCDE-FG, where:
- A = I/O module family e.g., M or F
- B = generation of the module e.g., either 1, 2 or 3
- CC = number of ports on the module e.g., 2, 6, 8, 24, 32 or 48
- D = port speed e.g., G=10/100/1000, X=10G, F=40G or C=100G
- E = connector type e.g., T=RJ45, S=SFP, 2=X2, P=SFP+, Q=QSFP or F=CFP
- F = the Fabric ASIC version of the I/O module e.g., either 1 or 2
- G = the number of Fabric modules required for full line rate I/O
There's also an E or L suffix on certain I/O modules, but we can ignore that in this discussion. Although as Leo has rightfully pointed out in other posts, don't ignore the E when buying F2 modules. Always make sure there's an E at the end
Using the N7K-F248XP-25E as an example, the -25E at the end indicates that the I/O module is Fabric 2 based, and need five fabric modules for full line rate operation.
Hope that clear things up a little.
Regards
