Re: 3750 Multicast QOS

Jesse Hottle · ‎01-30-2019

Hello,

I am having a tough go at prioritizing multicast traffic on our office 3750 switches. We have a single 3750 connected to our main fiber ring switch which is marking all multicast traffic on our primary IPTV vlan (500) with a cos value of 5. Once the multicast packets hit our 3750 the switch is marking these packets with a dscp value of 0. I have tried configuring the cos-dscp map to mark cos values to 46 as well as enable cos as well as configured each switch interface to trust the cos value.

Can someone help me out here? The problems I am seeing is the multicast streams are breaking up and tiling when connected directly to the 3750 on access interfaces.

Also our main uplink interface to our ring fiber switch is a trunk interface if that makes a difference. We also have "non" multicast traffic going across the main trunk interface as well.

Any help would be appreciated.

Thank you,

vb10 · ‎01-30-2019

Hello Jesse,

Do I understand correctly, that multicast traffic arrives on 3750 uplink (which is trunk) from upstream fiber switch, and is being forwarded to receivers, connected directly to access ports on this 3750 switch? Or there are additional hops between 3750 and receivers?

Does 3750 switch perform L3 routing for this multicast traffic, or only L2 switching?

How do you check CoS/DSCP values in packets? Do you perform SPAN and verify with wireshark?

Could you please share some statistics?

sh run | inc mls qos

show mls qos

show run int <uplink>

show run int <downlink>

show int <uplink>

show int <downlink>

show mls qos interface <uplink> queuing

show mls qos interface <downlink> queuing

show proc cpu sort (only total utilization and several top process are needed )

Thanks.

Jesse Hottle · ‎01-30-2019

Hello,

Thank you for the response. To answer your question, the multicast traffic is coming in over a trunk interface on the 3750. Some receivers are directly connected to this same 3750 on access ports but there are at least two other 3750's connected to this same switch over additional trunk interfaces with receivers also connect to these other switches too. Every connected switch is also a 3750.

All these switches are just doing L2 switching with IGMP snooping enabled.

I have used Wireshark to verify the CoS vaules of the packets coming from our fiber uplink switch (this is not a Cisco switch but a Dasan ring switch). We have no issues with receivers connected directly to the ring switch with our IPTV multicast feeds. I have a policy in place on our ring switch which is marking packets on our IPTV vlan to CoS 5 (this puts these packet in the high priority on our ring switch).

I currently have QOS disabled on the switch as of this morning.

#sh run | inc mls qos
mls qos map cos-dscp 0 8 16 24 32 46 48 56
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos
mls qos trust cos

interface GigabitEthernet1/0/3 - - RING UPLINK
description Trunk to ge1/1 on Dasan M2200
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 3,4,50-52,55,60,61,96,100,150,157,165,180,275
switchport trunk allowed vlan add 500,510,525,600,602,650
switchport mode trunk
mls qos trust cos
no ip igmp snooping tcn flood
end

interface FastEthernet1/0/46 - - DOWNLINK to a direct receiver
description STB
switchport access vlan 500
switchport mode access
switchport protected
switchport port-security
mls qos trust cos
spanning-tree portfast
spanning-tree bpdufilter enable
no ip igmp snooping tcn flood
end

#sh int g1/0/3 - - RING UPLINK
GigabitEthernet1/0/3 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 0014.a8a6.541b (bia 0014.a8a6.541b)
Description: Trunk to ge1/1 on Dasan M2200
MTU 9000 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 3/255, rxload 36/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 1000Mb/s, link type is auto, media type is 10/100/1000BaseTX SFP
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 00:00:02
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 142781000 bits/sec, 13986 packets/sec
5 minute output rate 12888000 bits/sec, 5563 packets/sec
46208 packets input, 52565884 bytes, 0 no buffer
Received 9225 broadcasts (9092 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 9092 multicast, 0 pause input
0 input packets with dribble condition detected
15481 packets output, 4645554 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out

sh int f1/0/46 - DOWNLINK to a direct receiver
FastEthernet1/0/46 is up, line protocol is up (connected)
Hardware is Fast Ethernet, address is 0014.a8a6.5432 (bia 0014.a8a6.5432)
Description: STB
MTU 1900 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 16/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:08, output hang never
Last clearing of "show interface" counters 00:00:30
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 6602000 bits/sec, 617 packets/sec
4 packets input, 256 bytes, 0 no buffer
Received 3 broadcasts (3 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 3 multicast, 0 pause input
0 input packets with dribble condition detected
18567 packets output, 24816078 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out

sh mls qos interface g1/0/3 queueing
GigabitEthernet1/0/3
QoS is disabled. When QoS is enabled, following settings will be applied
Egress Priority Queue : disabled
Shaped queue weights (absolute) : 25 0 0 0
Shared queue weights : 25 25 25 25
The port bandwidth limit : 100 (Operational Bandwidth:100.0)
The port is mapped to qset : 1

#sh mls qos interface f1/0/46 queueing
FastEthernet1/0/46
QoS is disabled. When QoS is enabled, following settings will be applied
Egress Priority Queue : disabled
Shaped queue weights (absolute) : 25 0 0 0
Shared queue weights : 25 25 25 25
The port bandwidth limit : 100 (Operational Bandwidth:100.0)
The port is mapped to qset : 1

sh proc cpu sort
CPU utilization for five seconds: 15%/0%; one minute: 7%; five minutes: 7%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
4 595670663 28201015 21122 9.58% 1.23% 0.87% 0 Check heaps
104 152996597 67683499 2260 0.31% 0.04% 0.00% 0 hpm counter proc
206 18188322 115984940 156 0.15% 0.08% 0.01% 0 IP Input
3 24687 28161 876 0.00% 0.00% 0.00% 0 SpanTree Helper
2 5954 13636635 0 0.00% 0.00% 0.00% 0 Load Meter
6 0 2 0 0.00% 0.00% 0.00% 0 Timers
5 669 14398 46 0.00% 0.00% 0.00% 0 Pool Manager
8 0 1 0 0.00% 0.00% 0.00% 0 Crash writer
9 86210467 327986363 262 0.00% 0.02% 0.03% 0 ARP Input
7 0 1 0 0.00% 0.00% 0.00% 0 HRPC asic-stats
11 0 1 0 0.00% 0.00% 0.00% 0 AAA_SERVER_DEADT
12 0 2 0 0.00% 0.00% 0.00% 0 AAA high-capacit
10 0 1 0 0.00% 0.00% 0.00% 0 CEF MIB API
14 24 18 1333 0.00% 0.00% 0.00% 0 Entity MIB API
15 0 1 0 0.00% 0.00% 0.00% 0 IFS Agent Manage
16 4823 1137641 4 0.00% 0.00% 0.00% 0 IPC Dynamic Cach
17 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager
18 23954 67683662 0 0.00% 0.00% 0.00% 0 IPC Periodic Tim
19 0 1 0 0.00% 0.00% 0.00% 0 IPC Managed Time
13 0 1 0 0.00% 0.00% 0.00% 0 Policy Manager
21 9292 4550532 2 0.00% 0.00% 0.00% 0 IPC Seat Manager

vb10 · ‎01-30-2019

Thanks Jesse,

I see, that "show interface" counters were cleared recently, did you noticed any drops before clearing them? Probably, you could monitor drops counter for some time.

Do you still observe the problem with multicast when QoS is disabled?

Based on the outputs, honestly, I don't see obvious issues with 3750, at least for now.

CoS/DSCP values are important mostly, when switch performs egress queuing on egress interface(s) towards receiver.

But now QoS is disabled, so single queue is used and traffic is enqueued on FIFO basis.

Based on ingress rate (almost 0) and egress rate on FastEthernet1/0/46 (around 6Mbps), I assume, that the only traffic which is forwarded via this interface is multicast stream. So, there is no other traffic, which would "compete" with multicast on receiver interface, meaning that CoS/DSCP value is not so important in this particular scenario. Rate is also quite low to exceed resources.

Interrupts in CPU utilization are "0", meaning that there are no issues with multicast programming, and all traffic is forwarded in hardware as expected and it should not cause any delays.

The most "utilized" part is uplink - interface between 3750 and ring switch. Input rate is 142Mbps. Doesn't seem too much, but it's average for 5 min. In case of short microbursts (fractions of seconds) we will not see it in this statistics.

I would suggest you to check, if there are traffic bursts on link between 3750 and ring switch, which might cause issues, using following options:

1. Check statistics on ring switch for interface towards 3750 (something similar to "show interface" or show mls qos queuing) to see, if there any drops, and if they are growing.

2. Perform SPAN from 3750 uplink interface and build graph in wireshark with short interval - 0.01 sec, or 0.001 sec. It will show bursts, if they are on the link

3. Do you have only one uplink to ring switch? If there is a redundant link, probably you can move only multicast traffic to other link and verify performance.