Power in line problem on my WS-C3560X-48P

[rkamonou]

Hi, everyone. Can anyone help with this problem regarding PoE issue on my Cisco C3560X switch?

This morning I was trying to configure a new IP phone and I realised that any time I plugged the phone into a switch port which was already working with another phone, the port PoE Operation mode goes off systematically. I tried on 4 ports with already working IP phones and the same problem occurred on those ports. I stop unplugging other working ports and I tried resetting power inline options on the switch but nothing worked. Any idea or solution to this problem?

Below is the output of the power inline status:   

asw-sroom(config-if)#do sh power inli
Available:467.0(w) Used:172.9(w) Remaining:294.1(w)

Interface Admin   Oper Power    Device   Class Max (Watts)
--------- ------ ---------- ------- ------------------- ----- ----
Gi0/1 auto on 6.4 IP Phone 8945 2 30.0
Gi0/2 auto off 0.0 n/a n/a 30.0
Gi0/3 auto off 0.0 n/a n/a 30.0
Gi0/4 auto off 0.0 n/a n/a 30.0
Gi0/5 auto off 0.0 n/a n/a 30.0
Gi0/6 auto on 6.4 Ieee PD 2 30.0
Gi0/7 auto on 15.4 AIR-CAP3502I-A-K9 3 30.0
Gi0/8 auto on 6.4 IP Phone 8845 2 30.0
Gi0/9 auto off 0.0 n/a n/a 30.0
Gi0/10 auto on 6.4 Ieee PD 2 30.0
Gi0/11 auto on 6.4 Ieee PD 2 30.0
Gi0/12 auto off 0.0 n/a n/a 30.0
Gi0/13 auto on 6.4 Ieee PD 2 30.0
Gi0/14 auto off 0.0 n/a n/a 30.0
Gi0/15 auto on 19.1 IP Phone 9971 3 30.0
Gi0/16 auto on 6.4 IP Phone 8945 2 30.0
Gi0/17 auto on 6.4 IP Phone 8945 2 30.0
Gi0/18 auto on 6.4 IP Phone 8945 2 30.0
Gi0/19 auto on 6.4 Ieee PD 2 30.0
Gi0/20 auto off 0.0 n/a n/a 30.0
Gi0/21 auto on 15.4 CIVS-IPC-6400 0 30.0
Gi0/22 auto off 0.0 n/a n/a 30.0
Gi0/23 auto off 0.0 n/a n/a 30.0
Gi0/24 auto off 0.0 n/a n/a 30.0
Gi0/25 auto off 0.0 n/a n/a 30.0
Gi0/26 auto off 0.0 n/a n/a 30.0
Gi0/27 auto off 0.0 n/a n/a 30.0
Gi0/28 auto off 0.0 n/a n/a 30.0
Gi0/29 auto off 0.0 n/a n/a 30.0
Gi0/30 auto off 0.0 n/a n/a 30.0
Gi0/31 auto off 0.0 n/a n/a 30.0
Gi0/32 auto on 6.4 IP Phone 8945 2 30.0
Gi0/33 auto off 0.0 n/a n/a 30.0
Gi0/34 auto on 6.4 IP Phone 8945 2 30.0
Gi0/35 auto off 0.0 n/a n/a 30.0
Gi0/36 auto on 15.4 CIVS-IPC-6400 0 30.0
Gi0/37 auto on 15.4 CIVS-IPC-6400 0 30.0
Gi0/38 auto off 0.0 n/a n/a 30.0
Gi0/39 auto off 0.0 n/a n/a 30.0
Gi0/40 auto on 15.4 CIVS-IPC-2835 4 30.0
Gi0/41 auto off 0.0 n/a n/a 30.0
Gi0/42 auto off 0.0 n/a n/a 30.0
Gi0/43 auto off 0.0 n/a n/a 30.0
Gi0/44 auto off 0.0 n/a n/a 30.0
Gi0/45 auto off 0.0 n/a n/a 30.0
Gi0/46 auto off 0.0 n/a n/a 30.0
Gi0/47 auto off 0.0 n/a n/a 30.0
Gi0/48 auto off 0.0 n/a n/a 30.0

Thanks

[Mark Malone]

HI
What type of phone are you connecting to the ports that your seeing this ?
any alerts appearing in logs when you connect it , is the port staying hard down or going err-disabled ?

[rkamonou]

Hi, Mark Malone thank you for your reply.

In total, I have tried with three different CISCO phones 8845, 8945 and 9971. The port remains up and data is working fine on those ports (PC connect to those ports receive valid IP), only PoE Opr mode remains OFF.

Phones which are currently connected to the switch are working fine but immediately I remove the ethernet cable and plugged it back, PoE stops working.

As of now, I have configured another WS-C2960X-24PS as a backup and I have connected those phones which ports PoE went Off, this helps me reduce complaint from users and gives me time to troubleshoot.    

The troubleshooting I have done so far consisted of first disabling and enabling PoE on the faulty ports, change power consumption values, Administratively shutting down and enabling the port.

below is a small log from. I also found that there are CRC errors increasing on one trunk port so I disable PoE on both switches ports, I have attached a snapshot of that. 

Below is are lines taking from the log (no specific error fund apart from port going up and down when trying to troubleshoot ) : 

Log Buffer (4096 bytes):
changed state to up
Dec 11 16:24:10 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to down
Dec 11 16:24:12 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to up
Dec 11 16:24:54 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to down
Dec 11 16:24:55 Kin: %LINK-3-UPDOWN: Interface GigabitEthernet0/6, changed state to down
Dec 11 16:24:57 Kin: %LINK-3-UPDOWN: Interface GigabitEthernet0/6, changed state to up
Dec 11 16:24:58 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to up
Dec 11 16:25:51 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/20, changed state to down
Dec 11 16:25:52 Kin: %LINK-3-UPDOWN: Interface GigabitEthernet0/20, changed state to down
Dec 11 16:29:20 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to down
Dec 11 16:29:25 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to up
Dec 11 16:29:29 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to down
Dec 11 16:29:31 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to up
Dec 11 16:29:32 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to down
Dec 11 16:29:34 Kin: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/6, changed state to up

[Leo Laohoo]

Post the complete output to the command "sh interface <BLAH> controll".

[paul driver]

Hello

Do you have any port security applied?
Can you also post a configuration of one of your access-ports -please.

res
Paul

[rkamonou]

Thank you, Paul, for your feedback. Surely I will rate all helpful posts.

**

Port security is not enabled on that switch. only Voice and Data VLAN are configured. This switch is one among 45 in my network with the same configuration.

Also, I have disabled PoE on my trunk ports.   

Below are few lines of my access and trunk port.

Thanks. 

interface GigabitEthernet0/30
switchport access vlan 208
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet0/47
description UPLINK-to-D-building-5thFlr-SW
switchport trunk encapsulation dot1q
switchport mode trunk
power inline never

[paul driver]

Hello

I dont see any voice vlan applied  I assume you've taken this off for TS purposes

Can you post what Leo suggested -
sh int xxx controllers

This is happening just to this switch correct?

res
Paul

[rkamonou]

This is the output. 

asw-kin-hq-old-sroom#sh interfaces gig0/20 controller
GigabitEthernet0/20 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 6400.f101.1314 (bia 6400.f101.1314)
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:03:11, output 00:00:01, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1000 bits/sec, 1 packets/sec
5 minute output rate 9000 bits/sec, 14 packets/sec
1783450 packets input, 159153529 bytes, 0 no buffer
Received 22055 broadcasts (5202 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 5202 multicast, 0 pause input
0 input packets with dribble condition detected
14765912 packets output, 10446382923 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out

Transmit GigabitEthernet0/20 Receive
1856448715 Bytes 159153801 Bytes
6139748 Unicast frames 1761395 Unicast frames
484408 Multicast frames 5203 Multicast frames
8141762 Broadcast frames 16854 Broadcast frames
0 Too old frames 150371074 Unicast bytes
0 Deferred frames 914891 Multicast bytes
0 MTU exceeded frames 3095260 Broadcast bytes
0 1 collision frames 0 Alignment errors
0 2 collision frames 0 FCS errors
0 3 collision frames 0 Oversize frames
0 4 collision frames 0 Undersize frames
0 5 collision frames 0 Collision fragments
0 6 collision frames
0 7 collision frames 362417 Minimum size frames
0 8 collision frames 1260467 65 to 127 byte frames
0 9 collision frames 133287 128 to 255 byte frames
0 10 collision frames 14651 256 to 511 byte frames
0 11 collision frames 4256 512 to 1023 byte frames
0 12 collision frames 8351 1024 to 1518 byte frames
0 13 collision frames 0 Overrun frames
0 14 collision frames 0 Pause frames
0 15 collision frames
0 Excessive collisions 0 Symbol error frames
0 Late collisions 0 Invalid frames, too large
0 VLAN discard frames 23 Valid frames, too large
0 Excess defer frames 0 Invalid frames, too small
1637639 64 byte frames 0 Valid frames, too small
743698 127 byte frames
6381096 255 byte frames 0 Too old frames
71004 511 byte frames 0 Valid oversize frames
26093 1023 byte frames 0 System FCS error frames
3188950 1518 byte frames 0 RxPortFifoFull drop frame
2717438 Too large frames
0 Good (1 coll) frames
0 Good (>1 coll) frames

[Leo Laohoo]

Output looks fine. The issue could be a configuration issue of the interface.
What kind of phones are these?

[rkamonou]

The types of CISCO phones I'm using are Cisco 8845, 8945 and 9971.This segment of the LAN has been operating without any problems for one year now. I have even avail five switch ports in the IT workshop for the troubleshoot and configuration of new IP Phones. Can this kind of problem originate from another switch on a trunk link?

[Leo Laohoo]

What is the IOS the switch is running on?

[rkamonou · Accepted solution]

Thanks for your support.

The version of the IOS  12.2(53)SE2  and the current image installed is c3560e-universalk9-mz.122-53.SE2.bin.

[Leo Laohoo]

Whao!
That's one buggy software there, mate.
Could you try something better, like the latest 12.2(55)SE?

[rkamonou]

Thanks, Leo, I will do as advised.

As of now, I'm proceeding on leave, and I have shared info I got from this forum with my colleagues. So far, since this switch is linked to critical offices, I've configured a new switch, and I have advised my colleagues to switch any user having the problem to the new switch. If not done until my return from leave, I'll do it myself as you advised and I think this may solve the problem.

I planned to download c3560e-universalk9-mz.122-55.SE3.bin, is it ok or do you recommend a different one this is the link to the list of IOS*** https://software.cisco.com/download/release.html?mdfid=280831079&softwareid=280805680&release=12.2.55-SE5 *****

Also, I'm still wondering, what could be the possible cause of this problem? Because similar switches with same IOS version are still in production and working fine. Anyway, if the upgrade of the IOS solves the problem, I'll make it a project to upgrade all similar switches.

Thanks a lot for your support. 

[Leo Laohoo]

Could you try something better, like the latest 12.2(55)SE?