10-10-2016 07:06 AM - edited 03-17-2019 08:19 AM
Hello all,
I have an issue with a 4506e chassis that was put in to replace a failing 3750 stack. The chassis is running a SUP 7 (WS-X45-SUP7-E) with five 48 port line cards (WS-X4648-RJ45V+E). The software version on the SUP is 03.06.05.E (cat4500e-universal.SPA.03.06.05.E.152-2.E5.bin).
The VoIP is based on CallManager with a mixture of 7940 and 7960 Cisco IP phones - these are the 100mb models. We are seeing a lot of jitter and packet loss on internal and external calls made with the phones, even between phones on the same line card. This is present whether AutoQoS is applied or removed. Not that it should be necessary as we are not going through any low bandwidth trunks. It is also present after I have set the interface speed and duplex settings on the switchport and phone.
Does anyone have any insight into why this may be happening and any possible remedial steps I could take?
Many thanks,
Steve
10-14-2016 02:31 AM
Interesting indeed, but that just raises more questions.
Did the phones get new loads since you changed the option 150?
If you ping from just a PC at 100mb, no phone, do you still get 100% but with occasional long delay packets?
Clear the counters on the switch ports and run the pings through the phones and verify you see the packet drops on the switch port.
Run test calls with the phones periodically to make sure there is consistency in those results.
10-15-2016 12:13 PM
The phones should have been operating with the cached loads from their last communication with the TFTP server prior to me changing the switch. Nothing about the load has changed so it shouldn't have changed anything.
Yes, pings are still giving the same results as before. Looking at the stats all the traffic is either going in queue 1 or queue 8.
Test calls between handsets on the same and different blades are fine - minimal jitter and no loss.
10-12-2016 09:29 AM
I was going to ask about the possibility of duplex mismatches but read the other thread and see that you have that covered.
What I do notice is that when doing the ping test at gig speed you get minimal delay and no packet loss. However at 100mb without the phone, you have no packet loss, but there is a very high maximum delay factor. With the phone you have the same high maximum delay along with the packet loss. So at 100mb, these sporadic delays are amplified with the phones and are why you are seeing the jitter and packet loss .
The only things I can suggest are:
1- Try the "speed auto 10 100" on the phone interfaces with the phones set to auto and see if this makes a difference. (A very long shot, but easy to try).
2- Not sure if it is an option but if you have a spare SUP7, try replacing it. The scheduling and queuing is controlled by the supervisor.
3- Try an older rev blade like an WS-X4248-RJ45V with the older phones. It's a 10/100 blade and should be compatible with the SUP 7E (https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-4500-series-switches/data_sheet_c78-612364.html)
4- Open a CCO case to see if they can determine if there is a bug or even a back plane issue. You've done enough here to describe the issue in greater detail.
Not sure what else to suggest but hope this helps.
10-13-2016 01:31 AM
1. I had tried the speed auto 100 with the phone set to auto and also with the phone set to 100 FULL - made no difference.
2. As it happens I do have a spare SUP7. I called our supplier for an RMA and the new unit arrived the day before yesterday. It is running an older version of IOS which I plan to test with.
3. I'll investigate this if the other SUP7 is unsuccessful.
4. I'll proceed with that is the 1-3 doesn't work out.
Thanks for the responses - much appreciated.
10-11-2016 09:05 AM
Aside from a bug or hardware issue here are some suggestions:
1- Clear counters and verify the physical layer is clean on all phone and uplink interfaces. (Input, output errors, runts, etc.)
2- Verify that this is not only a VoIP issue and that the similar packet loss isn't being experienced during data transfers or normal usage.
3- Get sniffer traces of calls and verify that the QoS settings are being preserved and not being altered or changed to best effort.
4- Verify (with a sniffer trace) that the phones on the same blade are communicating directly to each other and not having to go through an intermediate resource.
5- Try the "sh int gx/x counter detail" on one of the interfaces that has a phone to determine if traffic is in the proper queues and which queues are dropping the packets.
6- If you want, provide the switch's config for review.
10-12-2016 12:17 AM
Thanks for the reply.
1. I am getting output drops but I expect to see that given that the bandwidth is being reduced to 100mb. Aside from that the interfaces are clean.
2. The packet loss is being experienced on the voice and data vlans. The PCs (connected to the switchport on the phones) are dropping packets while pinging the switch.
3. I'll look at doing this but don't have a great deal of experience tracing and such.
4. As above.
5.
6. Here is the config. I've taken out the majority of similarly configured interfaces for ease of reading:
4506e#sh run
Building configuration...
Current configuration : 48165 bytes
!
! Last configuration change at 14:43:12 UTC Tue Oct 11 2016
!
version 15.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
!
hostname 4506e
!
boot-start-marker
boot-end-marker
!
!
vrf definition mgmtVrf
!
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family
!
enable secret 5 $1$4xW2$r8sninHTDPvENjxnBBXNA1
!
username admin privilege 15 secret 5 $1$.cN8$KNVH8upXn3h6u1JJaJ9li0
no aaa new-model
!
!
!
!
!
!
ip domain-name .co.uk
ip dhcp excluded-address 10.102.1.1 10.102.1.9
ip dhcp excluded-address 10.102.1.193 10.102.1.254
ip dhcp excluded-address 192.168.2.1 192.168.2.64
ip dhcp excluded-address 192.168.2.219 192.168.2.254
!
ip dhcp pool DATA
import all
network 192.168.2.0 255.255.255.0
default-router 192.168.2.2
domain-name .co.uk
dns-server 192.168.2.33 192.168.2.34
netbios-name-server 192.168.1.15
!
ip dhcp pool VOICE
import all
network 10.102.1.0 255.255.255.0
default-router 10.102.1.2
domain-name .co.uk
option 150 ip 10.102.1.2
dns-server 192.168.2.33
lease 8
!
!
!
power redundancy-mode redundant
!
spanning-tree mode rapid-pvst
no spanning-tree optimize bpdu transmission
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
class-map match-all AutoQos-4.0-Scavenger-Classify
match access-group name AutoQos-4.0-ACL-Scavenger
class-map match-all AutoQos-4.0-Signaling-Classify
match access-group name AutoQos-4.0-ACL-Signaling
class-map match-any AutoQos-4.0-Priority-Queue
match cos 5
match dscp ef
match dscp cs5
match dscp cs4
class-map match-all AutoQos-4.0-VoIP-Data-Cos
match cos 5
class-map match-any AutoQos-4.0-Multimedia-Stream-Queue
match dscp af31
match dscp af32
match dscp af33
class-map match-all AutoQos-4.0-Network-Mgmt
match dscp cs2
class-map match-all AutoQos-4.0-VoIP-Signal-Cos
match cos 3
class-map match-any AutoQos-4.0-Multimedia-Conf-Queue
match cos 4
match dscp af41
match dscp af42
match dscp af43
match access-group name AutoQos-4.0-ACL-Multimedia-Conf
class-map match-any AutoQos-4.0-Transaction-Data
match dscp af21
match dscp af22
match dscp af23
class-map match-all AutoQos-4.0-Network-Ctrl
match dscp cs7
class-map match-all AutoQos-4.0-Scavenger
match dscp cs1
class-map match-all AutoQos-4.0-Default-Classify
match access-group name AutoQos-4.0-ACL-Default
class-map match-any AutoQos-4.0-Signaling
match dscp cs3
match cos 3
class-map match-any AutoQos-4.0-Bulk-Data-Queue
match cos 1
match dscp af11
match dscp af12
match dscp af13
match access-group name AutoQos-4.0-ACL-Bulk-Data
class-map match-all AutoQos-4.0-Transaction-Classify
match access-group name AutoQos-4.0-ACL-Transactional-Data
class-map match-all AutoQos-4.0-Broadcast-Vid
match dscp cs5
class-map match-any AutoQos-4.0-Bulk-Data
match dscp af11
match dscp af12
match dscp af13
class-map match-any AutoQos-4.0-Scavenger-Queue
match dscp cs1
match cos 1
match access-group name AutoQos-4.0-ACL-Scavenger
class-map match-any AutoQos-4.0-VoIP
match dscp ef
match cos 5
class-map match-any AutoQos-4.0-Multimedia-Conf
match dscp af41
match dscp af42
match dscp af43
class-map match-any AutoQos-4.0-Control-Mgmt-Queue
match cos 3
match dscp cs7
match dscp cs6
match dscp cs3
match dscp cs2
match access-group name AutoQos-4.0-ACL-Signaling
class-map match-all AutoQos-4.0-Bulk-Data-Classify
match access-group name AutoQos-4.0-ACL-Bulk-Data
class-map match-any AutoQos-4.0-Trans-Data-Queue
match cos 2
match dscp af21
match dscp af22
match dscp af23
match access-group name AutoQos-4.0-ACL-Transactional-Data
class-map match-any AutoQos-4.0-Multimedia-Stream
match dscp af31
match dscp af32
match dscp af33
class-map match-any AutoQos-4.0-VoIP-Data
match dscp ef
match cos 5
class-map match-all AutoQos-4.0-Internetwork-Ctrl
match dscp cs6
class-map match-all AutoQos-4.0-Realtime-Interact
match dscp cs4
class-map match-all AutoQos-4.0-Multimedia-Conf-Classify
match access-group name AutoQos-4.0-ACL-Multimedia-Conf
class-map match-any AutoQos-4.0-VoIP-Signal
match dscp cs3
match cos 3
!
policy-map AutoQos-4.0-Output-Policy
class AutoQos-4.0-Scavenger-Queue
bandwidth remaining percent 1
class AutoQos-4.0-Priority-Queue
priority
police cir percent 30 bc 33 ms
class AutoQos-4.0-Control-Mgmt-Queue
bandwidth remaining percent 10
class AutoQos-4.0-Multimedia-Conf-Queue
bandwidth remaining percent 10
class AutoQos-4.0-Multimedia-Stream-Queue
bandwidth remaining percent 10
class AutoQos-4.0-Trans-Data-Queue
bandwidth remaining percent 10
dbl
class AutoQos-4.0-Bulk-Data-Queue
bandwidth remaining percent 4
dbl
class class-default
bandwidth remaining percent 25
dbl
policy-map AutoQos-4.0-Input-Policy
class AutoQos-4.0-VoIP
class AutoQos-4.0-Broadcast-Vid
class AutoQos-4.0-Realtime-Interact
class AutoQos-4.0-Network-Ctrl
class AutoQos-4.0-Internetwork-Ctrl
class AutoQos-4.0-Signaling
class AutoQos-4.0-Network-Mgmt
class AutoQos-4.0-Multimedia-Conf
class AutoQos-4.0-Multimedia-Stream
class AutoQos-4.0-Transaction-Data
class AutoQos-4.0-Bulk-Data
class AutoQos-4.0-Scavenger
policy-map AutoQos-4.0-Cisco-Phone-Input-Policy
class AutoQos-4.0-VoIP-Data-Cos
set dscp ef
police cir 128000 bc 8000
exceed-action set-dscp-transmit cs1
exceed-action set-cos-transmit 1
class AutoQos-4.0-VoIP-Signal-Cos
set dscp cs3
police cir 32000 bc 8000
exceed-action set-dscp-transmit cs1
exceed-action set-cos-transmit 1
class class-default
set dscp default
set cos 0
!
!
!
!
!
!
!
interface Port-channel1
switchport
switchport trunk allowed vlan 20,102
switchport mode trunk
!
interface FastEthernet1
vrf forwarding mgmtVrf
no ip address
speed auto
duplex auto
!
interface TenGigabitEthernet1/1
!
interface TenGigabitEthernet1/2
!
interface TenGigabitEthernet1/3
!
interface TenGigabitEthernet1/4
!
interface GigabitEthernet2/1
description Trunk link to server room
switchport trunk allowed vlan 20,102
switchport mode trunk
qos trust device cisco-phone
channel-group 1 mode active
!
interface GigabitEthernet2/2
switchport access vlan 20
switchport voice vlan 102
auto qos voip cisco-phone
qos trust device cisco-phone
spanning-tree portfast
!
interface GigabitEthernet2/3
switchport access vlan 20
switchport voice vlan 102
auto qos voip cisco-phone
qos trust device cisco-phone
spanning-tree portfast
!
interface GigabitEthernet2/4
switchport access vlan 20
switchport voice vlan 102
auto qos voip cisco-phone
qos trust device cisco-phone
spanning-tree portfast
!
interface GigabitEthernet2/5
description CallManager Subscriber
switchport access vlan 102
switchport voice vlan 102
speed 100
duplex full
spanning-tree portfast
!
interface GigabitEthernet2/6
description CallManager Publisher
switchport access vlan 102
switchport voice vlan 102
speed 100
duplex full
spanning-tree portfast
!
interface GigabitEthernet2/7
description Unity
switchport access vlan 102
speed 100
duplex full
spanning-tree portfast
!
interface GigabitEthernet2/8
description CUBE Router
switchport access vlan 102
switchport voice vlan 102
auto qos trust
spanning-tree portfast
service-policy input AutoQos-4.0-Input-Policy
!
interface GigabitEthernet2/9
description IPT Resource
switchport access vlan 20
speed 100
duplex full
spanning-tree portfast
!
interface GigabitEthernet2/10
description ISDN Router
switchport mode trunk
!
interface GigabitEthernet2/11
switchport access vlan 20
switchport voice vlan 102
auto qos voip cisco-phone
qos trust device cisco-phone
spanning-tree portfast
!
interface GigabitEthernet2/12
switchport access vlan 20
switchport voice vlan 102
auto qos voip cisco-phone
qos trust device cisco-phone
spanning-tree portfast
!
interface GigabitEthernet2/13
switchport access vlan 20
switchport voice vlan 102
auto qos voip cisco-phone
qos trust device cisco-phone
spanning-tree portfast
!
interface GigabitEthernet2/14
switchport access vlan 20
switchport voice vlan 102
auto qos voip cisco-phone
qos trust device cisco-phone
spanning-tree portfast
!
interface GigabitEthernet2/15
switchport access vlan 20
switchport voice vlan 102
auto qos voip cisco-phone
qos trust device cisco-phone
spanning-tree portfast
!
interface GigabitEthernet6/47
switchport access vlan 20
switchport voice vlan 102
auto qos voip cisco-phone
qos trust device cisco-phone
spanning-tree portfast
!
interface GigabitEthernet6/48
switchport access vlan 20
switchport voice vlan 102
auto qos voip cisco-phone
qos trust device cisco-phone
spanning-tree portfast
!
interface Vlan1
no ip address
shutdown
!
interface Vlan20
description ACCESS VLAN
ip address 192.168.2.2 255.255.255.0
!
interface Vlan102
description VOICE VLAN
ip address 10.102.1.2 255.255.255.0
!
ip default-gateway 192.168.2.1
ip forward-protocol nd
no ip http server
ip route 0.0.0.0 0.0.0.0 192.168.2.7
ip route 93.95.124.81 255.255.255.255 192.168.2.7
!
ip access-list extended AutoQos-4.0-ACL-Bulk-Data
permit tcp any any eq ftp
permit tcp any any eq ftp-data
permit tcp any any eq 22
permit tcp any any eq smtp
permit tcp any any eq 465
permit tcp any any eq 143
permit tcp any any eq 993
permit tcp any any eq pop3
permit tcp any any eq 995
permit tcp any any eq 1914
ip access-list extended AutoQos-4.0-ACL-Default
permit ip any any
ip access-list extended AutoQos-4.0-ACL-Multimedia-Conf
permit udp any any range 16384 32767
ip access-list extended AutoQos-4.0-ACL-Scavenger
permit tcp any any eq 1214
permit udp any any eq 1214
permit tcp any any range 2300 2400
permit udp any any range 2300 2400
permit tcp any any eq 3689
permit udp any any eq 3689
permit tcp any any range 6881 6999
permit tcp any any eq 11999
permit tcp any any range 28800 29100
ip access-list extended AutoQos-4.0-ACL-Signaling
permit tcp any any range 2000 2002
permit tcp any any range 5060 5061
permit udp any any range 5060 5061
ip access-list extended AutoQos-4.0-ACL-Transactional-Data
permit tcp any any eq 443
permit tcp any any eq 1521
permit udp any any eq 1521
permit tcp any any eq 1526
permit udp any any eq 1526
permit tcp any any eq 1575
permit udp any any eq 1575
permit tcp any any eq 1630
permit udp any any eq 1630
!
!
!
!
line con 0
stopbits 1
line vty 0 4
password
login
line vty 5 15
password
login
!
end
10-12-2016 04:42 AM
If you are dropping packets when simply pinging the switch from PCs that would indicate a larger problem and quite probably just not a phone issue.
Check the CPU utilization (sh proc cpu & sh proc cpu history). Also check for high traffic utilization on any port, especially trunk or server ports. That would be a good start.
I don't understand why the Voice DHCP scope has the option 150 pointing to the voice VLAN interface on the switch. It usually points to a TFTP server. Not sure what you are doing there.
From the counter output, it looks like the majority of packets being dropped are the best effort (data) packets. I agree if port utilization is minimal, this shouldn't be happening, but like I said there appears to be a bigger issue here.
Also, some minor points:
On G2/1, the trunk link to the server room. The qos trust device cisco phone will result in all incoming packets to be marked best effort. I'd remove it if you don't want that to happen.
On G2/5,6 & 8, you just need the access VLAN and not the voice VLAN.
The phone ports have "auto qos voip cisco phone" but I don't see any of the auto generated service policies applied. Not sure why that is. Try removing the command and re-applying it to one interface to see if that changes anything on the interface.
10-12-2016 05:11 AM
Good catch on the option 150. That is supposed to point at our CallManager - i'll change that.
The changes you suggest to the ports make sense so i'll go ahead and make those.
When I issued the auto qos voip cisco-phone command the policies were applied. I had removed them to establish whether the policies were having an effect on the jitter/packet loss. As it turns out it didn't make much difference whether the policies were applied or not.
CPU output as follows:
4506e#sh proc cpu
Core 0: CPU utilization for five seconds: 9%; one minute: 9%; five minutes: 9%
Core 1: CPU utilization for five seconds: 9%; one minute: 10%; five minutes: 10%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
1 853 728 1172 0.00 0.00 0.00 0 init
2 9 246 38 0.00 0.00 0.00 0 kthrd
3 263 38545 6 0.00 0.00 0.00 0 migration/0
4 0 3 3 0.00 0.00 0.00 0 sirq-high/0
5 1218968 235727009 5 0.00 0.00 0.00 0 sirq-timer/0
6 0 9 3 0.00 0.00 0.00 0 sirq-net-tx/0
7 69 5127 13 0.00 0.00 0.00 0 sirq-net-rx/0
8 0 7 0 0.00 0.00 0.00 0 sirq-block/0
9 0 8 0 0.00 0.00 0.00 0 sirq-block-iopo
10 8 2780 3 0.00 0.00 0.00 0 sirq-tasklet/0
11 215288 64449630 3 0.00 0.00 0.00 0 sirq-sched/0
12 0 11 0 0.00 0.00 0.00 0 sirq-hrtimer/0
13 774673 235727016 3 0.00 0.00 0.00 0 sirq-rcu/0
14 521 59616 8 0.00 0.00 0.00 0 desched/0
15 56 8467 6 0.00 0.00 0.00 0 migration/1
16 0 15 0 0.00 0.00 0.00 0 sirq-high/1
17 593524 235726839 2 0.00 0.00 0.00 0 sirq-timer/1
18 0 21 1 0.00 0.00 0.00 0 sirq-net-tx/1
19 163 12520 13 0.00 0.00 0.00 0 sirq-net-rx/1
20 0 19 0 0.00 0.00 0.00 0 sirq-block/1
21 0 20 0 0.00 0.00 0.00 0 sirq-block-iopo
22 3 905 3 0.00 0.00 0.00 0 sirq-tasklet/1
23 192560 71151539 2 0.00 0.00 0.00 0 sirq-sched/1
24 0 28 2 0.00 0.00 0.00 0 sirq-hrtimer/1
25 617997 235726845 2 0.00 0.00 0.00 0 sirq-rcu/1
26 64 8763 7 0.00 0.00 0.00 0 desched/1
27 436930 2895494 150 0.00 0.00 0.00 0 events/0
28 36000 1021794 35 0.00 0.00 0.00 0 events/1
29 0 28 0 0.00 0.00 0.00 0 cpuset
30 18 1467 12 0.00 0.00 0.00 0 khelper
33 0 30 0 0.00 0.00 0.00 0 netns
34 0 31 0 0.00 0.00 0.00 0 async/mgr
154 1346 157184 8 0.00 0.00 0.00 0 sync_supers
156 66 3541 18 0.00 0.00 0.00 0 bdi-default
158 2197 3730 589 0.00 0.00 0.00 0 kblockd/0
159 328 309 1064 0.00 0.00 0.00 0 kblockd/1
169 0 36 0 0.00 0.00 0.00 0 ksuspend_usbd
174 0 38 0 0.00 0.00 0.00 0 khubd
177 0 38 0 0.00 0.00 0.00 0 kseriod
179 3 43 71 0.00 0.00 0.00 0 kmmcd
205 0 59 3 0.00 0.00 0.00 0 rpciod/0
206 0 57 2 0.00 0.00 0.00 0 rpciod/1
241 0 44 0 0.00 0.00 0.00 0 kswapd0
242 0 45 0 0.00 0.00 0.00 0 aio/0
243 0 46 0 0.00 0.00 0.00 0 aio/1
244 0 47 0 0.00 0.00 0.00 0 nfsiod
246 0 48 0 0.00 0.00 0.00 0 xfs_mru_cache
247 0 49 0 0.00 0.00 0.00 0 xfslogd/0
248 0 50 0 0.00 0.00 0.00 0 xfslogd/1
249 0 51 0 0.00 0.00 0.00 0 xfsdatad/0
250 0 52 0 0.00 0.00 0.00 0 xfsdatad/1
251 0 53 0 0.00 0.00 0.00 0 xfsconvertd/0
252 0 54 0 0.00 0.00 0.00 0 xfsconvertd/1
253 0 55 0 0.00 0.00 0.00 0 crypto/0
254 0 56 0 0.00 0.00 0.00 0 crypto/1
436 0 61 0 0.00 0.00 0.00 0 mtdblockd
488 0 62 0 0.00 0.00 0.00 0 usbhid_resumer
558 93 824 113 0.00 0.00 0.00 0 udevd
706 163 5255 31 0.00 0.00 0.00 0 loop1
722 0 72 1 0.00 0.00 0.00 0 loop2
738 332 6898 48 0.00 0.00 0.00 0 loop3
1587 5 758 6 0.00 0.00 0.00 0 udevd
1597 2 755 3 0.00 0.00 0.00 0 udevd
1663 705 700 1007 0.00 0.00 0.00 0 dbus-daemon
2061 66 603 109 0.00 0.00 0.00 0 klogd
2073 2 601 4 0.00 0.00 0.00 0 portmap
2338 1 771 2 0.00 0.00 0.00 0 platform.mount
2343 1 771 1 0.00 0.00 0.00 0 inotifywait
2372 0 580 1 0.00 0.00 0.00 0 sshd
2384 2 575 3 0.00 0.00 0.00 0 xinetd
2388 1 576 3 0.00 0.00 0.00 0 xinetd
2417 0 70 0 0.00 0.00 0.00 0 lockd
2418 1 333 4 0.00 0.00 0.00 0 nfsd
2419 1 334 3 0.00 0.00 0.00 0 nfsd
2420 1 335 3 0.00 0.00 0.00 0 nfsd
2421 1 336 3 0.00 0.00 0.00 0 nfsd
2422 1 337 3 0.00 0.00 0.00 0 nfsd
2423 1 338 3 0.00 0.00 0.00 0 nfsd
2424 1 339 4 0.00 0.00 0.00 0 nfsd
2425 2 339 6 0.00 0.00 0.00 0 nfsd
2431 0 597 0 0.00 0.00 0.00 0 rpc.mountd
2795 1 698 2 0.00 0.00 0.00 0 sh
2798 1 698 2 0.00 0.00 0.00 0 inotifywait
3505 0 79 0 0.00 0.00 0.00 0 edac-poller
3527 0 80 0 0.00 0.00 0.00 0 vsi work/0
3528 0 81 0 0.00 0.00 0.00 0 vsi work/1
3899 9644 944363 10 0.00 0.00 0.00 0 watchdog
4274 144523 1886961 76 0.00 0.01 0.01 0 system_mgr
4485 14956 1046883 14 0.00 0.00 0.00 1088 slproc
4520 32783 1550712 21 0.00 0.00 0.01 1088 hwcontrol
4526 14746 1040706 14 0.00 0.00 0.00 1088 ns_oir_proxy
4528 14934 1043185 14 0.00 0.00 0.00 0 profiled
4529 59908 90875 659 0.00 0.01 0.05 0 oom_poll.sh
4530 16041 1046693 15 0.00 0.00 0.00 0 obfld
4531 14607 1043184 14 0.00 0.02 0.00 0 consoled
4532 14986 1043182 14 0.00 0.00 0.00 0 system_mgr_cli
4533 244509 6080184 40 0.00 0.00 0.01 0 plogd
4534 14393 1043228 13 0.00 0.00 0.00 0 psdprov
4535 239448 22247769 10 0.04 0.01 0.04 0 pdsd
4536 15270 1043253 14 0.00 0.01 0.02 0 osinfo-provider
4537 14718 1043211 14 0.00 0.00 0.00 0 oscore_p
4538 26667 1155404 23 0.00 0.03 0.04 0 netd
4539 322616 1890199 170 0.04 0.04 0.04 0 mem_mgmt
4540 42223 1983546 21 0.00 0.00 0.02 0 mgmte_tap
4541 22816 1072338 21 0.00 0.01 0.00 0 licensed
4542 14918 1060315 14 0.00 0.00 0.00 0 installer
4543 629296 24454520 376 0.12 0.12 0.12 0 ffm
4544 18494 1075988 17 0.00 0.08 0.28 0 eicored
4545 15785 1076406 14 0.00 0.00 0.00 0 dtmgr
4546 572065 1984535 288 0.04 0.03 0.04 0 cpumemd
4547 43955 2001064 21 0.00 0.00 0.02 0 liin_tap
4549 16118 1049655 15 0.00 0.06 0.06 0 cli_agent
4551 15412 5351150 2 0.00 0.00 0.00 0 snmp_subagent
4552 30940 1985917 15 0.00 0.01 0.02 0 erm_p
4553 25983 1985908 13 0.00 0.00 0.00 0 ngdumper_provid
4554 25757 1985917 12 0.00 0.03 0.04 0 sysinfo_p
4576 14918 1039280 14 0.00 0.00 0.00 0 fs_cmd
4587 16524 1145074 14 0.00 0.00 0.00 0 licenseagentd
4588 15220 1045335 14 0.00 0.00 0.00 0 iifd
4589 2600001 888453590 210 9.23 9.78 9.78 0 iosd
4590 15305 1063394 14 0.00 0.00 0.01 0 ha_mgr
6048 15 877 17 0.00 0.00 0.00 0 epc_sm_liaison
6072 0 878 0 0.00 0.00 0.00 34816 script
6078 4 877 4 0.00 0.00 0.00 34816 sh
6084 1 883 2 0.00 0.00 0.00 34816 tail
6269 98 810 121 0.00 0.00 0.00 34816 nova_sntp.sh
6399 212 901 236 0.00 0.00 0.00 34816 rollback_timer.
6494 0 808 0 0.00 0.00 0.00 0 nova_sntp.sh
6498 50 808 63 0.00 0.00 0.00 0 xlogger
6499 3 814 4 0.00 0.00 0.00 0 sntp
6554 1 889 1 0.00 0.00 0.00 0 rollback_timer.
6557 51 889 57 0.00 0.00 0.00 0 xlogger
6561 2 893 2 0.00 0.00 0.00 0 inotifywait
15085 0 258 0 0.00 0.00 0.00 0 flush-3:0
15132 1 90875 0 0.00 0.00 0.00 0 sleep
4506e#sh proc cpu history
History information for system:
11111
999990000088888888887777777777777779999977777777776666677777
100
90
80
70
60
50
40
30
20
10 ************************************************************
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per second (last 60 seconds)
22112 1112 1 1 1 1 11 1 1 1 1 1 1 11111 1 1111 1 1 1111
713029922929081819759539495995809979188703396921257089180008
100
90
80
70
60
50
40
30 *
20 ** * ** * * * * * ** * * *
10 ############################################################
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
22231111122111 21 12223222411 11 1111111111413432311 11111 11111 11111
159490375294319559313231810799933861440140253802068679044449070088100488
100
90
80
70
60
50
40 * * * *
30 *** * * * * * * *****
20 ***** **** ** ********** * ********** * * *
10 ########################################################################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
History information for core 0:
1 11111
399999000009999999999777778888877777999997777777777666667777
100
90
80
70
60
50
40
30
20
10 ************************************************************
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per second (last 60 seconds)
111111 111111 1 1 1 1111 1 1 11 1 111111111111 1 1111
973071922880190890928310494985800939997501314001248999081012
100
90
80
70
60
50
40
30
20 ** * ** * **
10 ############################################################
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
12223111112111121111313221311 111 1 11 11111212322311 11111 1 11111111
616821033901101240296700693699123949329030138654522549922340949960101350
100
90
80
70
60
50
40 *
30 *** * * * * * *** *
20 ***** ** * ********** ******** * *
10 ########################################################################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
History information for core 1:
1 11111 11111
099999000008888877777777777777777777000008888877777777778888
100
90
80
70
60
50
40
30
20
10 ************************************************************
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per second (last 60 seconds)
32112 1112211 111 1111111111111112 1 22111 1 1111 1 11 1112
698188022261282048880760407106012029381005389973378390282004
100
90
80
70
60
50
40 *
30 ** * *
20 *** * ** * ** * * * ** * * * * *
10 ############################################################
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
233421121231111211123242325221111 11111111115235334121111111121121111121
704142619688530971445704654100054993671251398461524920267562202230312631
100
90
80
70
60 *
50 * * *
40 * * * * * * * *** *
30 **** ** * *** *** * *****
20 ***** ******* ** ********** * * ** * ********** **** * * **
10 ########################################################################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
10-12-2016 06:47 AM
Just curious, how much packet loss is there when you ping the switch from a PC or simply go PC to PC on the same switch? Does it happen whether or not you are going through a phone? Is there a chance it can be isolated to a particular blade?
10-12-2016 07:56 AM
The packet loss varies. When pinging from a workstation to the switch it will drop anywhere from 5 - 10%. The packet loss/jitter on the phones looks like this:
I have posted the results of some directed connected and phone connected ping tests I ran in the other thread below (i'd post the results again but I don't want to make this even longer). I tested all blades to try and isolate, they are all performing similarly.
10-15-2016 01:14 AM
The chassis is running a SUP 7 (WS-X45-SUP7-E) with five 48 port line cards (WS-X4648-RJ45V+E).
I've got a 4510R+E with Sup7E and six of these X4648 but I'm running 3.7.2E.
My chassis has phones and wireless access points in them. I have no issue about call quality.
10-15-2016 12:24 PM
Thanks for the reply.
That is interesting. Can you share a little more about your setup. Are you using AutoQoS or do you have a custom policy (or no policy?). What model of phone do you use? Are you SIP or ISDN for your external calls?
10-15-2016 03:40 PM
What model of phone do you use? Are you SIP or ISDN for your external calls?
All Cisco shop. Cisco phones 7945/65/75 talking to a full CUCM cluster. Aside from the corded phones, some sites have 7925G wireless phones. Work OK.
Are you using AutoQoS or do you have a custom policy (or no policy?).
We've laid down our own fibre optic links (plural form) all over the city. Some people like to put QoS on the switches but when I see them I rip them out.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide