01-26-2013 11:32 AM - edited 03-07-2019 11:20 AM
Dear Experts,
Can you please verify my 6513 Core-switch Linecard throughput check as i am facing slow server performane issue.
1) Are there any performance issues with 6513 module core-switch and i wanted to know i have a sup 720B in this core-switch
which 720Gb/sc throughput does it mean it is switching throughput or Routing?
2) I guessall CEF 6748 module must have 40G spedd below ,Please correct me if i am wrong?
6513_CS1#sh fabric status
slot channel speed module fabric
status status
2 0 20G OK OK
4 0 8G OK OK
5 0 8G OK OK
7 0 20G OK OK
9 0 20G OK OK
9 1 20G OK OK
10 0 20G OK OK
10 1 20G OK OK
11 0 20G OK OK
11 1 20G OK OK
12 0 20G OK OK
12 1 20G OK OK
13 0 20G OK OK
6513_CS1#sh module
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
2 24 CEF720 24 port 1000mb SFP WS-X6724-SFP SAL09264AH4
3 48 48 port 10/100/1000mb EtherModule WS-X6148-GE-45AF SAL09169UD7
4 6 Firewall Module WS-SVC-FWM-1 SAD091600HL ......(Currently NOT IN USE)
5 24 SFM-capable 24-port 100FX Multi mode WS-X6524-100FX-MM SAL09264L2V
7 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAD092509K5
8 0 Supervisor-Other Unknown Unknown
9 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL09295N1N
10 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL0926444T
11 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL09285L8N
12 8 CEF720 8 port 10GE with DFC WS-X6708-10GE SAL1551YXER
13 24 CEF720 24 port 1000mb SFP WS-X6724-SFP SAL09264AGN
6513_CS1#sh fabric utilization all
slot channel speed Ingress % Egress %
2 0 20G 0 0
4 0 8G 0 0
5 0 8G 0 0
7 0 20G 0 0
9 0 20G 0 0
9 1 20G 0 0
10 0 20G 0 0
10 1 20G 0 0
11 0 20G 0 0
11 1 20G 0 0
12 0 20G 0 0
12 1 20G 0 0
13 0 20G 0 0
6513_CS1#sh fabric switching-mode
Global switching mode is Truncated
dCEF mode is not enforced for system to operate
Fabric module is not required for system to operate
Modules are allowed to operate in bus mode
Truncated mode is allowed, due to presence of DFC, CEF720 module
Module Slot Switching Mode
2 Crossbar
3 Bus
4 Crossbar
5 Crossbar
7 Bus
9 Crossbar
10 Crossbar
11 Crossbar
12 dCEF
13 Crossbar
Appreciate your inputs!
Regards,
Akber Mirza.
Solved! Go to Solution.
01-27-2013 04:37 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
1) Are there any performance issues with 6513 module core-switch and i wanted to know i have a sup 720B in this core-switchwhich 720Gb/sc throughput does it mean it is switching throughput or Routing?
The 720 Gbps is the fabric (duplex) bandwidth capacity.
Yes there are many possible performance issues, especially as you're configured.
Sup720 only provides (in a 6513) 40 Gbps to slots 9-13, other slots are only 20 Gbps.
Non-CEF256 cards are either 32 Gbps (duplex) shared bus (6148) or 8 Gbps to the fabric (6548).
Classic bus cards (6148) can not directly communicate with CEF720 cards (67xx). Data has to be relayed through the sup.
I recall (?) 65xx cards with both fabric and bus connections have a static option whether to transmit using their fabric or bus connection. Default might be to use bus connection, in your chassis fabric connection likely better as most of your cards have fabric connections.
Any classic bus card in the chassis limits sup's packets-per-second performance. Sup720 is rated 15 Mpps in non-Compact mode or 30 Mpps in Compact mode (your running in truncate mode). Also 30 Mpps is documented for all packet sizes, so performance difference might be much, much more than 2x.
Older line card often have much more limited performance per port, i.e. ports groups share buffers or ASICs.
The Sup720 performance PPS performance is shared by the whole chassis unless you also have dCEF (DFC) cards. (Your 6708 has a DFC, standard.)
Minimum size Ethernet packets requires 1.455 Mpps per 1 Gbps, so 15 Mpps is might max out with as little at 5 Gbps (duplex).
2) I guessall CEF 6748 module must have 40G spedd below ,Please correct me if i am wrong?
They have that much bandwidth to the fabric (as also noted by Reza). To take full advantage of it, you need to account for much else (see above).
PS:
Suggestions:
Resolve issue with 2nd sup - doesn't appear to be in correct standby mode. (Won't change performance but if active sup fails . . .)
Replace the classic bus card (6148) with a fabric card. (I.e. get chassis to run in compact mode.)
When working with high density gig, avoid using all but 67xx line cards. (Note: Your 6524 is a non-issue, as it's only 24 FE ports with 8 Gbps fabric.)
Move 6724 card, if possible, to 20 Gbps slot. (I.e. reserve your 40 Gbps slots for cards that require 40 Gbps.)
PPS:
For even more performance, as it appears all your CEF720 cards but the 6708 are CFCs, you can selectively install DFCs or you can upgrade your Sup. Both the VS-S720 and Sup2T provide higher centralized PPS. (Sup720s, I believe, are EOL.)
01-28-2013 05:47 PM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
#1 In truncate mode, sup can provides up to 15 Mpps. It's possible you might exceed the forwarding performance of the sup with enough busy gig ports.
#2 Truncate mode offers 15 Mpps, compact mode 30 Mpps; i.e. 2x performance.
#3 I don't know, although I've seen is on other sup cards.
#4 Would expect L3 forwarding as fast as L2 forwarding. Was this test done with exactly the same two ports and two hosts?
01-26-2013 12:38 PM
Hi,
Yes, the maximum backplane speed for the 67xx series cards with sup-720 is 40 Gig per slot. (2 channels each 20Gig)
see link below:
HTH
01-27-2013 04:37 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
1) Are there any performance issues with 6513 module core-switch and i wanted to know i have a sup 720B in this core-switchwhich 720Gb/sc throughput does it mean it is switching throughput or Routing?
The 720 Gbps is the fabric (duplex) bandwidth capacity.
Yes there are many possible performance issues, especially as you're configured.
Sup720 only provides (in a 6513) 40 Gbps to slots 9-13, other slots are only 20 Gbps.
Non-CEF256 cards are either 32 Gbps (duplex) shared bus (6148) or 8 Gbps to the fabric (6548).
Classic bus cards (6148) can not directly communicate with CEF720 cards (67xx). Data has to be relayed through the sup.
I recall (?) 65xx cards with both fabric and bus connections have a static option whether to transmit using their fabric or bus connection. Default might be to use bus connection, in your chassis fabric connection likely better as most of your cards have fabric connections.
Any classic bus card in the chassis limits sup's packets-per-second performance. Sup720 is rated 15 Mpps in non-Compact mode or 30 Mpps in Compact mode (your running in truncate mode). Also 30 Mpps is documented for all packet sizes, so performance difference might be much, much more than 2x.
Older line card often have much more limited performance per port, i.e. ports groups share buffers or ASICs.
The Sup720 performance PPS performance is shared by the whole chassis unless you also have dCEF (DFC) cards. (Your 6708 has a DFC, standard.)
Minimum size Ethernet packets requires 1.455 Mpps per 1 Gbps, so 15 Mpps is might max out with as little at 5 Gbps (duplex).
2) I guessall CEF 6748 module must have 40G spedd below ,Please correct me if i am wrong?
They have that much bandwidth to the fabric (as also noted by Reza). To take full advantage of it, you need to account for much else (see above).
PS:
Suggestions:
Resolve issue with 2nd sup - doesn't appear to be in correct standby mode. (Won't change performance but if active sup fails . . .)
Replace the classic bus card (6148) with a fabric card. (I.e. get chassis to run in compact mode.)
When working with high density gig, avoid using all but 67xx line cards. (Note: Your 6524 is a non-issue, as it's only 24 FE ports with 8 Gbps fabric.)
Move 6724 card, if possible, to 20 Gbps slot. (I.e. reserve your 40 Gbps slots for cards that require 40 Gbps.)
PPS:
For even more performance, as it appears all your CEF720 cards but the 6708 are CFCs, you can selectively install DFCs or you can upgrade your Sup. Both the VS-S720 and Sup2T provide higher centralized PPS. (Sup720s, I believe, are EOL.)
01-27-2013 11:12 AM
Hi,
Appreciate your detailed explaination!
Can you please brief me what will be the benifit i get if run the switch in Compact mode ? ( currently inTruncated mode).
Also are there any performance issues with sup720B or 6513 module as i am facing slowness in servers connected to
module 9 ( 40Gbps) ?
6513_CS1#sh fabric status
slot channel speed module fabric
status status
2 0 20G OK OK
4 0 8G OK OK
5 0 8G OK OK
7 0 20G OK OK
9 0 20G OK OK
9 1 20G OK OK
10 0 20G OK OK
10 1 20G OK OK
11 0 20G OK OK
11 1 20G OK OK
12 0 20G OK OK
12 1 20G OK OK
13 0 20G OK OK
Rgards,
Akber Mirza.
01-27-2013 04:05 PM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
Can you please brief me what will be the benifit i get if run the switch in Compact mode ? ( currently inTruncated mode).
2x (or more) sup forwarding performance.
Also are there any performance issues with sup720B or 6513 module as i am facing slowness in servers connected tomodule 9 ( 40Gbps) ?
It possible running in truncate mode and with only one dCEF enabled card, you're hitting sup's maximum forwarding capacity but there might be other causes. You would need to define "slowness", why it seems restricted to just module 9 and what's the topology between end-hosts.
01-28-2013 10:26 AM
HEllo,
I would like to inform you that there are no issues with module 9 of 6513, I just took it as an example as i just
tested the throughput on hosts connected to that module.
Can you please explain your below comment as i coudnt understand
""
It possible running in truncate mode and with only one dCEF enabled card, you're hitting sup's maximum forwarding capacity but there might be other causes. """.
2) Can you further brief me what is the difference interms performance with switch running in Compact or Truncated mode in detail please?
Appreciate your comments.
6513_CS1#sh fabric switching-mode
Global switching mode is Truncated
dCEF mode is not enforced for system to operate
Fabric module is not required for system to operate
Modules are allowed to operate in bus mode
Truncated mode is allowed, due to presence of DFC, CEF720 module
Module Slot Switching Mode
2 Crossbar
3 Bus
4 Crossbar
5 Crossbar
7 Bus
9 Crossbar
10 Crossbar
11 Crossbar
12 dCEF
13 Crossbar
3) Can you tell why module 7 is showing BUS instead of dCEF ?
4) I am getting a throughput between intra vlan host speed 1G using jperf as 0.75 Gb/sec
where as intravlanhost speed 1G as 0.40Gb/sec for the samehosts.
Is this a normal behaviour ?
Regards,
Akber Mirza.
01-28-2013 05:47 PM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
#1 In truncate mode, sup can provides up to 15 Mpps. It's possible you might exceed the forwarding performance of the sup with enough busy gig ports.
#2 Truncate mode offers 15 Mpps, compact mode 30 Mpps; i.e. 2x performance.
#3 I don't know, although I've seen is on other sup cards.
#4 Would expect L3 forwarding as fast as L2 forwarding. Was this test done with exactly the same two ports and two hosts?
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: