Catalyst 4500 series SUP7E perfomance comparision question

[Alexey Stytsenko]

Hi folks!

I have few questions about SUP7E perfomance. There are they:

Catalyst 4500 SUP7e switching capacity is 848 Gbps

Catalyst 6500 Sup720 backplane is 720 Gbps

So this is mean that SUP7E has capability to switch more traffic than SUP720 ?

Second question - what size pakets must be to gain bandwidth of 848 gbps when SUP7E IPv4 throughput is 250 mpps? 1500 byte packets give us only about 375 Gbps.

And last question -

Catalyst 4500 SUP7e IPv4 throughput is 250 mpps

Catalyst 6500 Sup720 without using distributed forwarding (NO DFC3!!!) is 30 mpps

SUP7E has so poferful architecture that dominate over Sup720 architecture 8 times???

What purpose of using such distributed architectures(as Sup720) if centralized can offer such perfomance?

[Leo Laohoo]

Hi Alexey,

I'm no expert with the Sup7E, however, I would like to request if you can compare the performance of the Sup7E with the  Sup2T (link included).

The reason why is because the Sup720 is a 5 year old technology.  The Sup7E for the 4500 is less than a year old.  The Sup2T, however, is also new (officially released last week).

[Alexey Stytsenko]

SUP2T use PFC4 card which offers 60 mpps of centralized forwarding.

So last question as well can be applied to Sup2T architecture

But first two questions is still topical cos it seems to me that i misunderstood smth in them.

[Joseph W. Doherty · Accepted solution]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Fabric bandwidths are generally defined for both in and out.  So, for example a 2 gig port switch, to support wire-speed or line-rate, would need a 4 Gbps fabric.  The fabric bandwidth requirement is independent of frame size.

Forwarding rates are defined for traffic flowing unidirectionally through the device, so the same 2 gig port switch, to guarantee Ethernet line rate would need to provide about 2.976 Mpps.  (NB: this for minimal size frames, the pps requirement would decrease for larger frames.  For example, 1518 byte frame for the same 2 gig would only require .162 Mpps)  (NB: for one gig, Ethernet min frames need 1.488 Mpps, 1518 byte frames need 81 Kpps.)

"So this is mean that SUP7E has capability to switch more traffic than SUP720 ?"

Yes, it does.  More fabric bandwidth, and much, much, much more forwarding performance for the supervisor itself. 

"Second question - what size pakets must be to gain bandwidth of 848 gbps  when SUP7E IPv4 throughput is 250 mpps? 1500 byte packets give us only  about 375 Gbps."

Packet size doesn't matter. To fully utilize 848 Gbps, you would need a Ethernet forwarding rate of about 285 Mpps to guarantee line-rate or wire-speed for all frame sizes.

"Catalyst 6500 Sup720 without using distributed forwarding (NO DFC3!!!) is 30 mpps"

That's for fabric line cards - half that for non-fabric line cards.

"SUP7E has so poferful architecture that dominate over Sup720 architecture 8 times???"

Correct for just for supervisor forwarding performance.

"What purpose of using such distributed architectures(as Sup720) if centralized can offer such perfomance?"

That's how the 6500 architecture offers additional forwarding performance.

dCEF720 line cards, with their DFCs, offer 48 Mpps (per card).

PS:

As the other posters have noted, the SUP7E is very recent, and its performance outclasses a SUP720 but the even more recent SUP2T with its newer corresponding line cards offers about twice the performance; 1 Tbps fabric and up to 720 Mpps in a 6513 chassis.  (NB: the Mpps performance doesn't guarantee wire-speed/line-rate performance for all Ethernet packet sizes, but real-world packets generally aren't all minimum sized.)

[Alexey Stytsenko]

Great answer m8! I highly appreciate it, btw u wrote -

Packet size doesn't matter. To fully utilize 848 Gbps, you would need a Ethernet forwarding rate of about 285 Mpps to guarantee line-rate or wire-speed for all frame sizes.

Before i suppose that switch bandwidth must be equal throughput(in millions of packets) multiplied by a maximum packet size, but it seems u use different approach in u calculations - can u explain please?

[Joseph W. Doherty]

Disclaimer

The  Author of this posting offers the information contained within this  posting without consideration and with the reader's understanding that  there's no implied or expressed suitability or fitness for any purpose.  Information provided is for informational purposes only and should not  be construed as rendering professional advice of any kind. Usage of this  posting's information is solely at reader's own risk.

Liability Disclaimer

In  no event shall Author be liable for any damages whatsoever (including,  without limitation, damages for loss of use, data or profit) arising out  of the use or inability to use the posting's information even if Author  has been advised of the possibility of such damage.

Posting

The reason I noted packet size doesn't matter, because fabric bandwidth is similar to a link supporting a specific bandwidth.  For example, 100 Mbps doesn't care whether frames are 64 bytes, 1518 bytes or 9000 bytes, bandwidth measures the bit transmission rate of the medium.

Where fabric bandwidth does become important is when it might become a bottleneck.  So again in the case of my 2 gig port switch, if it only had a fabric bandwidth of 2 Gbps, traffic could queue to cross the fabric.

An example of fabric bandwidth being a possible bottleneck, on the 6509 with the sup720, there's 40 Gbps (duplex) bandwidth to the fabric, per slot.  For a 8x10gig card, the fabric bandwidth doesn't support the bandwidth of the line card.  I.e. it's possible traffic will queue to the fabric (also why the 6708 has so much more hardware buffer space vs. the 6704).

Beside raw bandwidth, routers and switches need to examine packets and frames to make forwarding decisions.  This takes some time and so hardware is also rated for it's forwarding performance (packet per second, PPS).

Minimum size (64 byte) Ethernet packets, if transmitted back-to-back without any additional inter-packet delay, at gig bandwidth, will transfer at 1,488 million packets per second.  1518 byte Ethernet packets, also transmitted back-to-back without any additional interpacket delay, at gig bandwidth, will transmit at 81 thousand packets per second.  A device making forwarding decisions would need to meet there PPS rates if it it to continue to forward the frames at bandwidth's capacity.

Since real-world networks generally don't see every port in a switch running at 100% all the time, fabric bandwidths and PPS rates may not, and often not, guarantee full performance.  (Wire-rate and line-speed network devices, often have more to do with bragging rights rather than actual need.  However, knowing the internal bandwidth and forwarding performance of a network device does allow properly "sizing" its capabilities relative to actual need beyond just summing up all the port bandwidths.)

PS:

If this doesn't provide the information your trying to understand, let us know.

[Alexey Stytsenko]

>>The reason I noted packet size doesn't matter, because fabric  bandwidth is similar to a link supporting a specific bandwidth.  For  example, 100 Mbps doesn't care whether frames are 64 bytes, 1518 bytes  or 9000 bytes, bandwidth measures the bit transmission rate of the  medium.

But when we consider the link - we can utilize it on 100% but with fabric bandwidth we have restriction - throughput, and so we can never utilize 720 Gbps bandwidth on sup720 switch fabric - maximum what we can get, as i mentioned above, is (1518 * 250 000 000 )/10^9 = 379,5 Gbps.

So i can make conclusion that switch fabric bandwidth or swich capacity is not a real measure of switch perfomance, but simple cumulative value for all slots bandwidth?

[Joseph W. Doherty]

Disclaimer

The    Author of this posting offers the information contained within this    posting without consideration and with the reader's understanding that    there's no implied or expressed suitability or fitness for any  purpose.   Information provided is for informational purposes only and  should not   be construed as rendering professional advice of any kind.  Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In    no event shall Author be liable for any damages whatsoever  (including,   without limitation, damages for loss of use, data or  profit) arising  out  of the use or inability to use the posting's  information even if  Author  has been advised of the possibility of such  damage.

Posting

Yes, fabric can be a restriction.  But, no, you can utilize 720 Gbps on a sup720.  For larger packet/frame sizes, and with DFC line cards, the platform can fully load the fabric.

Remember when you calculate pps vs. fabric bandwidth, you need to double the pps value or half the fabric value.  Additionally, you need to know the pps rate for specific packet/frame sizes.  Today, most pps rates are for Ethernet minimum packet/frame size.  Usually actual pps rates for other packet/frame size are non-linear.

For example, you've taken a frame size of 1518 times 250 Mpps, but is this pps documented for 1518?

Also for your calculation you also need to remember the 1518 is in bytes, so you need to multiply by another 8 for bit per second.  I.e. your 379.5 is GBps or 3,036 Gbps and would need a fabric of 6,076 Gbps.

[Alexey Stytsenko]

>>Remember when you calculate pps vs. fabric bandwidth, you need to  double the pps value or half the fabric value.  Additionally, you need  to know the pps rate for specific packet/frame sizes.  Today, most pps  rates are for Ethernet minimum packet/frame size.  Usually actual pps  rates for other packet/frame size are non-linear.

Why? Mpps is value given for a whole system not only one direction of traffic, and 720 Gbps (sup 720) is bidirectional bandwidth as well.

>>Also for your calculation you also need to remember the 1518 is in  bytes, so you need to multiply by another 8 for bit per second.  I.e.  your 379.5 is GBps or 3,036 Gbps and would need a fabric of 6,076 Gbps.

Oops u right! I absolutely forgot about Bytes -> Bits translation, but now result is horrible value of 6 Tbps. So this is was mistake to calculate fabric max bandwith based on throughput values and packets size. And throughput in 64 bytes packets - is maximum throughput system can grant, then for 1518 bytes packets thoughput will significantly differs in decreasing side?

[Joseph W. Doherty · Accepted solution]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Why? Mpps is value given for a whole system not only one direction of  traffic, and 720 Gbps (sup 720) is bidirectional bandwidth as well.

Yes, pps is given for the whole system, but you only need to account for one direction of traffic; fabric specifications tend to account for duplex bandwidth.  It's really the same bandwidth, but the pps and fabric account for them differently; similar to a 100 Mbps (duplex) link might be noted as providing 200 Mbps of bandwidth.

For example:

Assuming we have a 4x gig (duplex) port switch, how much bandwidth do we have 4 gig or 8 gig?  Since each port is duplex, we must have 8 gig, yet one port's input is another's output, so there's only 4 gig being transferred.  The fabric bandwidth, if it can support all the ports at their full rate, would be noted as 8 gig.  The forwarding pps, would only need to account for 4 gig.  For Ethernet 64 bytes this would be 4 x 1.488 Mpps, for Ethernet 1,518 bytes this would be 4 x 81 Kpps.

And throughput in 64 bytes packets - is maximum throughput system can  grant, then for 1518 bytes packets thoughput will significantly differs  in decreasing side?

No, just the opposite.  Bigger packets required pps rates, for same transfer rate, decrease.  (Again, for gig Ethernet, 1.488 Mpps @ 64 vs. 81 Kpps @ Kpps.)

PS:

A 6513's slots 1-8 provide 20 (duplex), slots 9-13 provide 40 (duplex).  Fabric bandwidth would be 8*40=320 Gbps, 5*80=400 Gbps; 320 + 400 = 720 Gbps

PPS:

4500 series provides 6, 24 (SUP6) or 48 (SUP7E) Gbps per slot (with corresponding line cards).

6500 series provides 20 or 40 (SUP720), or 80 (SUP2T) Gbps per slot (with corresponding line cards).

[Alexey Stytsenko]

>>And throughput in 64 bytes packets - is maximum throughput system  can  grant, then for 1518 bytes packets >>thoughput will significantly  differs  in decreasing side?

>>No, just the opposite.  Bigger packets required pps rates, for same transfer rate, decrease.  (Again, for gig >>Ethernet, 1.488 Mpps @ 64 vs. 81 Kpps @ Kpps.)

U misunderstand me here, i mean that if switch fabric provide 250 mpps with 64 byte packets, consequently it doesnt mean that perfomance (in mpps) of switch fabric will be same with 1518 byte packets? Or it will?

And previously u wrote that

>>Packet size doesn't matter. To fully utilize 848 Gbps, you would need a Ethernet forwarding rate of about 285 Mpps to guarantee line-rate or wire-speed for all frame sizes.

how u calculate this?

P.S. Great tnx m8 for u answers - i got a lot of interesting things from they!

[Joseph W. Doherty]

Disclaimer

The  Author of this posting offers the information contained within this  posting without consideration and with the reader's understanding that  there's no implied or expressed suitability or fitness for any purpose.  Information provided is for informational purposes only and should not  be construed as rendering professional advice of any kind. Usage of this  posting's information is solely at reader's own risk.

Liability Disclaimer

In  no event shall Author be liable for any damages whatsoever (including,  without limitation, damages for loss of use, data or profit) arising out  of the use or inability to use the posting's information even if Author  has been advised of the possibility of such damage.

Posting

U misunderstand me here, i mean that if switch fabric provide 250 mpps  with 64 byte packets, consequently it doesnt mean that perfomance (in  mpps) of switch fabric will be same with 1518 byte packets? Or it will?

No, larger packets don't guarantee better performance, but it almost always does.  At least it shouldn't be worst.  The only way to know for sure is if the vendor documents the performance for a particular size (of if you have the equipment to test yourself).

>>Packet size doesn't matter. To fully utilize 848 Gbps, you  would need a Ethernet forwarding rate of about 285 Mpps to guarantee  line-rate or wire-speed for all frame sizes.

how u calculate this?

Take fabric bandwidth divide by two, then divide by line-rate/wire-speed Ethernet pps per gbs, e.g. 848 / 2 = 424; 424 / 1.488 = 284.9