cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4167
Views
5
Helpful
16
Replies

6500: Crossbar mode or Bus mode on Sup720 uplinks ???

ovt
Level 4
Level 4

Hi!

Which mode (bus or crossbar) is used when Sup720 uplinks send traffic to other modules? The following output shows that BUS MODE is used, but I don't understand why -- Sup720 has 20Gbps channel to the internal crossbar. Also, "global switching mode" was "flow-through" until "fabric switching-mode allow truncated threshold 1" is configured (the default threshold is 2). Can anybody explain why ???

cat6506E#sh fabric switching-mode

Global switching mode is Truncated

dCEF mode is not enforced for system to operate

Fabric module is not required for system to operate

Modules are allowed to operate in bus mode

Truncated mode is allowed, due to presence of CEF720 module

Module Slot Switching Mode

1 Bus

2 Crossbar

5 Bus

-------------------------------------

cat6506E#sh module

Mod Ports Card Type Model Serial No.

--- ----- -------------------------------------- ------------------ -----------

1 48 48-port 10/100/1000 RJ45 EtherModule WS-X6148A-GE-TX SAL1117MG0E

2 48 CEF720 48 port 1000mb SFP WS-X6748-SFP SAL1131W4S3

5 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAL1052CGFR

16 Replies 16

Well...

Unfortunately the discussion is going in the wrong direction...

It's still absolutely unclear what does "bus" (or "crossbar" or "dCEF") mean _in_front_of_ the Sup module. I.e.:

5 Bus

Your guess is that "it uses the bus to communicate with classic modules and crossbar to communicate with fabric-enabled modules". Yes, this makes sense. The problem, however, is that the "Global switching mode" was flow-through until I set the threshold to 1. This probably means that the Sup720 is _not_ considered fabric-enabled in this particular chassis with the modules installed. This, in turn, means that the Sup uplinks are operating in the bus mode.

So, my guess is, yes the Sup is using both bus and crossbar to pass data _between_ classic and fabric-enabled modules. And this is very well described in the Networkers 2007 presentations. _But_ the Sup uplinks are operating in the _bus_ mode. I.e. the entire packet, received thru the uplink port, is sent

to the DBus. I don't think we see "5 - bus" just because we have classic modules in the chassis and because the Sup needs to pass data between classc and fabric-enabled modules. Of course, I might be wrong.

rseiler
Level 3
Level 3

Why on earth do you have a 6148A-GE-TX (junk) blade in this switch at all? Can't you hear the switch running slower?

What is connected to that blade and what speeds are those devices using (10/100/1000)?

The bigger problem than the fabric switching-mode is the architecture of the 6148-GE-TX blade (Note that the 6548 has the same architecture on the port side, albeit CEF256):

Not sure you are aware, but that blade is configured as 6 groups of 8 ports. In other words, a 6 Gigabit blade, not 48 Gb (or 20 or 40). This is a wiring closet blade for *very* lightly used PCs connected at a Gig. NOT for a server farm, distribution, or core.

There are several problems with this blade:

1. There are not per-port buffers, only per-group buffers across 8 ports.

This can create buffering issues that *will* effect all 8 ports. The bad thing is, this can happen even if all the ports are running at 100 Mb!!!

2. Drop counters are not exposed to the cli because the 'hidden' pinnacle ASIC has no idea which port the buffer problem effects. This behavior can be modified by using the 'hol-blocking enable' hidden command on the ports, see next item. This effectively carves up the per-group buffer into smaller chunks and managing them per port in software.

3. Head of line blocking (HOL) is a *huge* issue with these blades. Basically a bunch of traffic can be *destined* for a port-group and starve the output buffers for all other ports in the group. This will essentially stop traffic to all other ports in that group until the HOL blocking issue dissipates, which could be never. This is particularly a problem with a heavily utilized server and/or using a port in the port-group as a span destination.

I have clients that have these blades who have been told by Cisco in TAC cases that they purchased the wrong (cheaper) blades and are using them for the wrong purpose and will need to use only 6 ports on the blade (every 8th port) to resolve their performance issue! Cisco documentation clearly states that these blades (6148-GE-TX, 6548-GE-TX) are *only* wiring closet blades.

Just my 2 cents since I see a lot of these conversations on these forums about the fabric switching-mode when the incorrectly used blades are a far bigger issue. Who cares how fast traffic gets switched between the blades when the 6148-GE-TX blade itself is 8 to 1 oversubscribed before the traffic even makes it to/from the backplane!!!

Review Cisco Networking for a $25 gift card