Citrix Users Feel Stuck with a 3850 Stack

[saviolabao2]

Hello,

We are facing a very weird issue here, let me clarify it:

1. We have some Citrix VDI users here, they use HP thin clients to connect to the server end virtual machines. The structure is like:  Thin Clinet ----- Access Switch ------ Core Switch Stack ----- Server, NETAPP is also connected directly to the core switch, 1G copper connection everywhere.

2. Everything went well before we refreshed the two core switches, we were using a 3750X stack (two members) as core switches and we refreshed them with two 3850 (Also run as a stack).

3. After the refresh, Citrix users started to complain that they will be stuck when they do some actions such as double clicking something, typing in something, etc. For example, when they try to open the outlook (by double clicking the outlook icon on the desktop), there will be no response, even the icon will not be highlighted (as selected by single click), for several seconds, then the outlook is opened directly (so the action is actually done). It feels like the image just hang there for several seconds.

4. We changed nothing but only the core switches then users started to complain. They are two 3850 (WS-C3850-48T-S and WS-C3750X-48T-S before) running with version IOS XE 3.7.5.

5. We found that there were some output drops on those interfaces connected to servers/NETAPP, we tried to increate the port buffer and the drops haven't happened after that. We also tried to downgrade the version to 3.6.6 but not useful.

6. We tried with a laptop to login the Citrix desktop, still have the issue is not from the thin client, so the . We also tried from a remote site (so that the traffic only goes through router ---- Core SW ---- Server), still have the issue, so the issue is not from the access switches.

6. We did not change any configuration of the core switches before/after the refresh, attached with the configuration.

Have you met any similar issue before or do you have any idea?

Thanks

Sijian Bao

[Deepak Kumar]

Hi, 

There will be many reasons for dops. Maybe any media issue, Policies error, buffer issue, etc. 

Please go through the below documents and check again:

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-3850-series-switches/200594-Catalyst-3850-Troubleshooting-Output-dr.html

Regards,

Deepak Kumar

[saviolabao2]

Hello Deepak,

Yes we've read that when we saw the output drops under the interfaces. Then we followed it to increase the buffer (qos queue-softmax-multiplier command), we do not see output drops any more after that. However the issue is still there. We also monitored the statistics of the drops before, the number did not increase when some users reported that they met the issue just now.

Thanks

Sijian Bao

[Georg Pauwen]

Hello,

did you have any Citrix/Ica related QoS configured on the 3750s ? 

Either way, if you don't see any congestion on the interfaces, QoS probably won't help anyway.

Can you post the output of 'show buffers' from the 3850s ?

[saviolabao2]

Hello,

We did not have any specific QoS configured for VDI (just some auto qos/mls qos trust cos) before and now. The traffic of Citrix is not much (around 100-200k per user and lest then 100M for each interface connected to the server).

Here's the output of 'show buffer':
CNGZPSVRRM3850CORE#show buffers
PDS public buffers

Public buffer pools:
PDS Buffer Pool 0, 2048 bytes (total 2048, permanent 2048, transient 0
peak 1313 @ 6917941):
938 in free list (2048 max allowed)
54029477 hits, 0 misses, 54028367 free hits, 0 free misses
PDS Buffer Pool 1, 12288 bytes (total 341, permanent 341, transient 0):
341 in free list (341 max allowed)
0 hits, 0 misses, 0 free hits, 0 free misses
SM Buf Group TX, 516 bytes (total 32, permanent 32, transient 0
peak 5 @ 20471):
32 in free list (32 max allowed)
84752 hits, 0 misses, 84752 free hits, 0 free misses


Header pools:
Packet Header Memory, 176 bytes (total 4681, permanent 4681, transient 0
peak 221 @ 6917941):
4663 in free list (4681 max allowed)
58190254 hits, 0 misses, 58190236 free hits, 0 free misses
Buffer Header Memory, 88 bytes (total 16384, permanent 16384, transient 0
peak 1313 @ 6917941):
15273 in free list (16384 max allowed)
54114231 hits, 0 misses, 54113120 free hits, 0 free misses

IOSd private buffers:

Buffer elements:
2016 in free list
103821182 hits, 0 misses, 1242 created

Public buffer pools:
Small buffers, 104 bytes (total 600, permanent 600):
600 in free list (100 min, 1250 max allowed)
50157030 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Middle buffers, 600 bytes (total 450, permanent 450):
448 in free list (50 min, 1000 max allowed)
229968 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Big buffers, 1536 bytes (total 450, permanent 450, peak 451 @ 1d23h):
450 in free list (25 min, 1500 max allowed)
470826 hits, 0 misses, 1 trims, 1 created
0 failures (0 no memory)
VeryBig buffers, 4520 bytes (total 10, permanent 10, peak 20 @ 1d23h):
10 in free list (0 min, 300 max allowed)
68 hits, 5 misses, 11 trims, 11 created
0 failures (0 no memory)
Large buffers, 5024 bytes (total 10, permanent 10, peak 11 @ 1d23h):
10 in free list (0 min, 30 max allowed)
2 hits, 0 misses, 1 trims, 1 created
0 failures (0 no memory)
VeryLarge buffers, 8232 bytes (total 50, permanent 50):
50 in free list (0 min, 150 max allowed)
28 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Huge buffers, 18024 bytes (total 5, permanent 5, peak 75 @ 1d23h):
4 in free list (0 min, 13 max allowed)
76144 hits, 35 misses, 71 trims, 71 created
0 failures (0 no memory)

Interface buffer pools:
ng3k_rx32 buffers, 1 bytes (total 1, permanent 1):
1 in free list (0 min, 1 max allowed)
0 hits, 0 misses
ng3k_rx21 buffers, 5 bytes (total 5, permanent 5):
5 in free list (0 min, 5 max allowed)
0 hits, 0 misses
CF Small buffers, 104 bytes (total 101, permanent 100, peak 101 @ 1d23h):
101 in free list (100 min, 200 max allowed)
7 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
BIPC small buffers, 128 bytes (total 251, permanent 250, peak 251 @ 1d23h):
251 in free list (250 min, 250 max allowed)
66 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
Generic ED Pool buffers, 512 bytes (total 101, permanent 100, peak 101 @ 1d23h):
101 in free list (100 min, 100 max allowed)
0 hits, 0 misses
BIPC middle buffers, 600 bytes (total 301, permanent 300, peak 301 @ 1d23h):
301 in free list (300 min, 300 max allowed)
49268 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
CF Middle buffers, 600 bytes (total 101, permanent 100, peak 101 @ 1d23h):
101 in free list (100 min, 200 max allowed)
2293 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
Syslog ED Pool buffers, 600 bytes (total 133, permanent 132, peak 133 @ 1d23h):
101 in free list (132 min, 132 max allowed)
479 hits, 0 misses
CF Big buffers, 1536 bytes (total 26, permanent 25, peak 26 @ 1d23h):
26 in free list (25 min, 50 max allowed)
57 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
ng3k_rx6 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx7 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx8 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx9 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx10 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx11 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx12 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx13 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx14 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx15 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx17 buffers, 1600 bytes (total 768, permanent 768):
768 in free list (0 min, 768 max allowed)
477030 hits, 0 misses
ng3k_rx18 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
241275 hits, 0 misses
ng3k_rx19 buffers, 1600 bytes (total 1024, permanent 1024):
1024 in free list (0 min, 1024 max allowed)
556 hits, 0 misses
ng3k_rx20 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx22 buffers, 1600 bytes (total 16, permanent 16):
16 in free list (0 min, 16 max allowed)
6536 hits, 1 misses
ng3k_rx23 buffers, 1600 bytes (total 16, permanent 16):
16 in free list (0 min, 16 max allowed)
2288 hits, 0 misses
ng3k_rx24 buffers, 1600 bytes (total 16, permanent 16):
16 in free list (0 min, 16 max allowed)
0 hits, 0 misses
ng3k_rx25 buffers, 1600 bytes (total 16, permanent 16):
16 in free list (0 min, 16 max allowed)
0 hits, 0 misses
ng3k_rx26 buffers, 1600 bytes (total 16, permanent 16):
16 in free list (0 min, 16 max allowed)
0 hits, 0 misses
ng3k_rx27 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
351041 hits, 0 misses
ng3k_rx28 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
963759 hits, 0 misses
ng3k_rx29 buffers, 1600 bytes (total 2048, permanent 2048):
2047 in free list (0 min, 2048 max allowed)
393618 hits, 0 misses
ng3k punt fallback pool buffers, 1600 bytes (total 1500, permanent 1500):
1500 in free list (0 min, 0 max allowed)
0 hits, 0 misses
ng3k_rx33 buffers, 1600 bytes (total 128, permanent 128):
128 in free list (0 min, 128 max allowed)
9553 hits, 0 misses
ng3k_rx34 buffers, 1600 bytes (total 128, permanent 128):
128 in free list (0 min, 128 max allowed)
213966 hits, 0 misses
ng3k_rx35 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx36 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx37 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx38 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
16687 hits, 0 misses
ng3k_rx40 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
9 hits, 0 misses
ng3k_rx39 buffers, 2048 bytes (total 1024, permanent 1024):
1024 in free list (0 min, 1024 max allowed)
0 hits, 0 misses
BIPC buffers, 4096 bytes (total 2, permanent 2):
2 in free list (1 min, 8 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
CF VeryBig buffers, 4520 bytes (total 3, permanent 2, peak 3 @ 1d23h):
3 in free list (2 min, 4 max allowed)
1 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
CF Large buffers, 5024 bytes (total 2, permanent 1, peak 2 @ 1d23h):
2 in free list (1 min, 2 max allowed)
0 hits, 0 misses, 17 trims, 18 created
0 failures (0 no memory)
ng3k punt jumbo pool buffers, 9300 bytes (total 128, permanent 128):
128 in free list (0 min, 0 max allowed)
0 hits, 0 misses
BIPC Medium buffers, 16384 bytes (total 6, permanent 5, peak 6 @ 1d23h):
6 in free list (5 min, 5 max allowed)
0 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
BIPC Large buffers, 65535 bytes (total 3, permanent 2, peak 3 @ 1d23h):
3 in free list (2 min, 2 max allowed)
0 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
IPC small buffers, 128 bytes (total 1200, permanent 1200):
1178 in free list (400 min, 1200 max allowed)
137345 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache
IPC middle buffers, 600 bytes (total 1200, permanent 1200):
1200 in free list (400 min, 1200 max allowed)
3343 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache
IPC Emergency buffers, 4096 bytes (total 301, permanent 300, peak 301 @ 1d23h):
301 in free list (300 min, 300 max allowed)
0 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache
IPC buffers, 4096 bytes (total 1200, permanent 1200):
1197 in free list (400 min, 1200 max allowed)
26586 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache
IPC Medium buffers, 16384 bytes (total 40, permanent 40):
40 in free list (20 min, 40 max allowed)
0 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache
Private Huge IPC buffers, 18024 bytes (total 1, permanent 0, peak 1 @ 1d23h):
1 in free list (0 min, 4 max allowed)
0 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
Private Huge buffers, 65280 bytes (total 1, permanent 0, peak 1 @ 1d23h):
1 in free list (0 min, 4 max allowed)
0 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
IPC Large buffers, 65535 bytes (total 104, permanent 104, peak 105 @ 1d23h):
104 in free list (20 min, 104 max allowed)
0 hits, 0 misses, 1 trims, 1 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache

Header pools:
Header buffers, 0 bytes (total 267, permanent 256, peak 267 @ 1d23h):
11 in free list (10 min, 512 max allowed)
253 hits, 3 misses, 22631 trims, 22642 created
0 failures (0 no memory)
256 max cache size, 256 in cache
0 hits in cache, 0 misses in cache

Particle Clones:
1024 clones, 0 hits, 0 misses

Public particle pools:
F/S buffers, 256 bytes (total 384, permanent 384):
128 in free list (128 min, 1024 max allowed)
256 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
256 max cache size, 256 in cache
0 hits in cache, 0 misses in cache
Normal buffers, 512 bytes (total 512, permanent 512):
384 in free list (128 min, 1024 max allowed)
128 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
128 max cache size, 128 in cache
0 hits in cache, 0 misses in cache

Thanks,
Sijian Bao

[Deepak Kumar]

Hi, 

I hope above documents you have checked carefully. Can you share following commands output in a text file:

1. sh int XX.X 

2. sh controllers XX.X  utilization

4. show processes CPU | i ^PID|Input

and 5. Sho Buffer

Regards,

Deepak Kumar

[saviolabao2]

Hello Deepak,

Here's the output:

1. sh int XX.X
CNGZPSVRRM3850CORE#show int gi2/0/22
GigabitEthernet2/0/22 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is ec1d.8b19.3c96 (bia ec1d.8b19.3c96)
Description: To_GZPESX002_DT02
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: Class-based queueing
Output queue: 0/40 (size/max)
5 minute input rate 3480000 bits/sec, 446 packets/sec
5 minute output rate 2436000 bits/sec, 456 packets/sec
67014421 packets input, 76750554155 bytes, 0 no buffer
Received 176443 broadcasts (6006 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 6006 multicast, 0 pause input
0 input packets with dribble condition detected
118904926 packets output, 137135277551 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

2. sh controllers XX.X utilization
CNGZPSVRRM3850CORE#show controllers gi2/0/22 utilization
Receive Bandwidth Percentage Utilization : 0
Transmit Bandwidth Percentage Utilization : 0

4. show processes CPU | i ^PID|Input
CNGZPSVRRM3850CORE#show processes CPU | i ^PID|Input
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

5. Show Buffer
CNGZPSVRRM3850CORE#show buffers
PDS public buffers

Public buffer pools:
PDS Buffer Pool 0, 2048 bytes (total 2048, permanent 2048, transient 0
peak 1313 @ 6917941):
938 in free list (2048 max allowed)
54029477 hits, 0 misses, 54028367 free hits, 0 free misses
PDS Buffer Pool 1, 12288 bytes (total 341, permanent 341, transient 0):
341 in free list (341 max allowed)
0 hits, 0 misses, 0 free hits, 0 free misses
SM Buf Group TX, 516 bytes (total 32, permanent 32, transient 0
peak 5 @ 20471):
32 in free list (32 max allowed)
84752 hits, 0 misses, 84752 free hits, 0 free misses


Header pools:
Packet Header Memory, 176 bytes (total 4681, permanent 4681, transient 0
peak 221 @ 6917941):
4663 in free list (4681 max allowed)
58190254 hits, 0 misses, 58190236 free hits, 0 free misses
Buffer Header Memory, 88 bytes (total 16384, permanent 16384, transient 0
peak 1313 @ 6917941):
15273 in free list (16384 max allowed)
54114231 hits, 0 misses, 54113120 free hits, 0 free misses

IOSd private buffers:

Buffer elements:
2016 in free list
103821182 hits, 0 misses, 1242 created

Public buffer pools:
Small buffers, 104 bytes (total 600, permanent 600):
600 in free list (100 min, 1250 max allowed)
50157030 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Middle buffers, 600 bytes (total 450, permanent 450):
448 in free list (50 min, 1000 max allowed)
229968 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Big buffers, 1536 bytes (total 450, permanent 450, peak 451 @ 1d23h):
450 in free list (25 min, 1500 max allowed)
470826 hits, 0 misses, 1 trims, 1 created
0 failures (0 no memory)
VeryBig buffers, 4520 bytes (total 10, permanent 10, peak 20 @ 1d23h):
10 in free list (0 min, 300 max allowed)
68 hits, 5 misses, 11 trims, 11 created
0 failures (0 no memory)
Large buffers, 5024 bytes (total 10, permanent 10, peak 11 @ 1d23h):
10 in free list (0 min, 30 max allowed)
2 hits, 0 misses, 1 trims, 1 created
0 failures (0 no memory)
VeryLarge buffers, 8232 bytes (total 50, permanent 50):
50 in free list (0 min, 150 max allowed)
28 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
Huge buffers, 18024 bytes (total 5, permanent 5, peak 75 @ 1d23h):
4 in free list (0 min, 13 max allowed)
76144 hits, 35 misses, 71 trims, 71 created
0 failures (0 no memory)

Interface buffer pools:
ng3k_rx32 buffers, 1 bytes (total 1, permanent 1):
1 in free list (0 min, 1 max allowed)
0 hits, 0 misses
ng3k_rx21 buffers, 5 bytes (total 5, permanent 5):
5 in free list (0 min, 5 max allowed)
0 hits, 0 misses
CF Small buffers, 104 bytes (total 101, permanent 100, peak 101 @ 1d23h):
101 in free list (100 min, 200 max allowed)
7 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
BIPC small buffers, 128 bytes (total 251, permanent 250, peak 251 @ 1d23h):
251 in free list (250 min, 250 max allowed)
66 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
Generic ED Pool buffers, 512 bytes (total 101, permanent 100, peak 101 @ 1d23h):
101 in free list (100 min, 100 max allowed)
0 hits, 0 misses
BIPC middle buffers, 600 bytes (total 301, permanent 300, peak 301 @ 1d23h):
301 in free list (300 min, 300 max allowed)
49268 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
CF Middle buffers, 600 bytes (total 101, permanent 100, peak 101 @ 1d23h):
101 in free list (100 min, 200 max allowed)
2293 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
Syslog ED Pool buffers, 600 bytes (total 133, permanent 132, peak 133 @ 1d23h):
101 in free list (132 min, 132 max allowed)
479 hits, 0 misses
CF Big buffers, 1536 bytes (total 26, permanent 25, peak 26 @ 1d23h):
26 in free list (25 min, 50 max allowed)
57 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
ng3k_rx6 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx7 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx8 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx9 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx10 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx11 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx12 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx13 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx14 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx15 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx17 buffers, 1600 bytes (total 768, permanent 768):
768 in free list (0 min, 768 max allowed)
477030 hits, 0 misses
ng3k_rx18 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
241275 hits, 0 misses
ng3k_rx19 buffers, 1600 bytes (total 1024, permanent 1024):
1024 in free list (0 min, 1024 max allowed)
556 hits, 0 misses
ng3k_rx20 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx22 buffers, 1600 bytes (total 16, permanent 16):
16 in free list (0 min, 16 max allowed)
6536 hits, 1 misses
ng3k_rx23 buffers, 1600 bytes (total 16, permanent 16):
16 in free list (0 min, 16 max allowed)
2288 hits, 0 misses
ng3k_rx24 buffers, 1600 bytes (total 16, permanent 16):
16 in free list (0 min, 16 max allowed)
0 hits, 0 misses
ng3k_rx25 buffers, 1600 bytes (total 16, permanent 16):
16 in free list (0 min, 16 max allowed)
0 hits, 0 misses
ng3k_rx26 buffers, 1600 bytes (total 16, permanent 16):
16 in free list (0 min, 16 max allowed)
0 hits, 0 misses
ng3k_rx27 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
351041 hits, 0 misses
ng3k_rx28 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
963759 hits, 0 misses
ng3k_rx29 buffers, 1600 bytes (total 2048, permanent 2048):
2047 in free list (0 min, 2048 max allowed)
393618 hits, 0 misses
ng3k punt fallback pool buffers, 1600 bytes (total 1500, permanent 1500):
1500 in free list (0 min, 0 max allowed)
0 hits, 0 misses
ng3k_rx33 buffers, 1600 bytes (total 128, permanent 128):
128 in free list (0 min, 128 max allowed)
9553 hits, 0 misses
ng3k_rx34 buffers, 1600 bytes (total 128, permanent 128):
128 in free list (0 min, 128 max allowed)
213966 hits, 0 misses
ng3k_rx35 buffers, 1600 bytes (total 96, permanent 96):
96 in free list (0 min, 96 max allowed)
0 hits, 0 misses
ng3k_rx36 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx37 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
0 hits, 0 misses
ng3k_rx38 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
16687 hits, 0 misses
ng3k_rx40 buffers, 1600 bytes (total 512, permanent 512):
512 in free list (0 min, 512 max allowed)
9 hits, 0 misses
ng3k_rx39 buffers, 2048 bytes (total 1024, permanent 1024):
1024 in free list (0 min, 1024 max allowed)
0 hits, 0 misses
BIPC buffers, 4096 bytes (total 2, permanent 2):
2 in free list (1 min, 8 max allowed)
0 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
CF VeryBig buffers, 4520 bytes (total 3, permanent 2, peak 3 @ 1d23h):
3 in free list (2 min, 4 max allowed)
1 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
CF Large buffers, 5024 bytes (total 2, permanent 1, peak 2 @ 1d23h):
2 in free list (1 min, 2 max allowed)
0 hits, 0 misses, 17 trims, 18 created
0 failures (0 no memory)
ng3k punt jumbo pool buffers, 9300 bytes (total 128, permanent 128):
128 in free list (0 min, 0 max allowed)
0 hits, 0 misses
BIPC Medium buffers, 16384 bytes (total 6, permanent 5, peak 6 @ 1d23h):
6 in free list (5 min, 5 max allowed)
0 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
BIPC Large buffers, 65535 bytes (total 3, permanent 2, peak 3 @ 1d23h):
3 in free list (2 min, 2 max allowed)
0 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
IPC small buffers, 128 bytes (total 1200, permanent 1200):
1178 in free list (400 min, 1200 max allowed)
137345 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache
IPC middle buffers, 600 bytes (total 1200, permanent 1200):
1200 in free list (400 min, 1200 max allowed)
3343 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache
IPC Emergency buffers, 4096 bytes (total 301, permanent 300, peak 301 @ 1d23h):
301 in free list (300 min, 300 max allowed)
0 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache
IPC buffers, 4096 bytes (total 1200, permanent 1200):
1197 in free list (400 min, 1200 max allowed)
26586 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache
IPC Medium buffers, 16384 bytes (total 40, permanent 40):
40 in free list (20 min, 40 max allowed)
0 hits, 0 fallbacks, 0 trims, 0 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache
Private Huge IPC buffers, 18024 bytes (total 1, permanent 0, peak 1 @ 1d23h):
1 in free list (0 min, 4 max allowed)
0 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
Private Huge buffers, 65280 bytes (total 1, permanent 0, peak 1 @ 1d23h):
1 in free list (0 min, 4 max allowed)
0 hits, 0 misses, 2830 trims, 2831 created
0 failures (0 no memory)
IPC Large buffers, 65535 bytes (total 104, permanent 104, peak 105 @ 1d23h):
104 in free list (20 min, 104 max allowed)
0 hits, 0 misses, 1 trims, 1 created
0 failures (0 no memory)
0 max cache size, 0 in cache
0 hits in cache, 0 misses in cache

Header pools:
Header buffers, 0 bytes (total 267, permanent 256, peak 267 @ 1d23h):
11 in free list (10 min, 512 max allowed)
253 hits, 3 misses, 22631 trims, 22642 created
0 failures (0 no memory)
256 max cache size, 256 in cache
0 hits in cache, 0 misses in cache

Particle Clones:
1024 clones, 0 hits, 0 misses

Public particle pools:
F/S buffers, 256 bytes (total 384, permanent 384):
128 in free list (128 min, 1024 max allowed)
256 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
256 max cache size, 256 in cache
0 hits in cache, 0 misses in cache
Normal buffers, 512 bytes (total 512, permanent 512):
384 in free list (128 min, 1024 max allowed)
128 hits, 0 misses, 0 trims, 0 created
0 failures (0 no memory)
128 max cache size, 128 in cache
0 hits in cache, 0 misses in cache


Thanks
Sijian Bao

[Georg Pauwen]

Hello,

try and increase the huge buffers, you have misses and trims. Depending on what software you are running, the syntax should be something like this:

3850(config)#buffers huge permanent 75
3850(config)#buffers huge initial 50
3850(config)#buffers huge max-free 100

[saviolabao2]

Hello Georg,

I've tried to increase that yesterday and I could not see any misses or trims until now, unfortunately the issue is still there.

Regards,
Sijian Bao

[Georg Pauwen]

Hello,

which IOS version are you running on the 3850s (show version) ? You might want to apply a QoS policy that prioritizes Citrix/ICA. I think the DSCP to COS and vice versa default mappings are different on the 3850 than on the 3750, so it might just be that Citrix traffic gets pushed through a lower priority queue.

Can you post the output of the three commands below:

show platform qos policies port sw [sw#]
show platform qos queue config interface_type [interface_name]
show platform qos queue stats interface_type [interface_name]

When you configure a class map on your 3850, what are your 'match' options ?

3850#conf t

3850(config)#class-map match-any CITRIX_ICA

3850(config-map)#match ?