Every friday morning: Transceiver problem?

[davhalcisco]

Every friday morning at about 2:40 central, this message pops up in the router's log, and then the ethernet0 interface goes down, up, down again, and up again in a total of 4 seconds. Any ideas why it does this n a regular, predicatble basis?No one is going in and unplugging the ethernet interface.... IOS is 12.0(27), and it is a 1601.

Oct 29 02:41:46 CDT: %QUICC_ETHER-1-LOSTCARR: Unit 0, lost carrier. Transceiver problem?

Oct 29 02:41:47 CDT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0, changed state to down

Oct 29 02:41:58 CDT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0, changed state to up

Oct 29 02:42:20 CDT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0, changed state to down

Oct 29 02:42:21 CDT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0, changed state to up

[Prashanth Krishnappa]

What connects to this ethernet0 interface? Check to see if there is any scheduled maintenance at that time every week on it.

[davhalcisco]

It is simply a 2980g switch. That switch does not see that port going up and down, only the router sees it. There is not maintenenace on that switch at that time. There is only about 16 other computers in that switch also. It isnt very busy.

[thisisshanky]

How many times has this pattern repeated ???

[davhalcisco]

at least 3 months I know. I cant find an exact starting date.

[Kevin Dorrell]

Do you have a network management system? Might it be worth monitoring SNMP during that time? If you do have SNMP active, try setting an access-class on the SNMP management of both the switch and the router (especially the switch in this case) so that at least you can be sure who is managing it.

Do you have logging activated on the switch? Does that give any clues? You will have to look in the logging buffer, because at the moment it goes down, it probably cannot log to syslog.

Kevin Dorrell

Luxembourg

[davhalcisco]

Now it happens at 1:40 after daylight savings change. weird.

I will try to do some SNMP monitoring this time. We use CiscoWorks to help manage this device. The only thing I get in the logs is what is posted in the original post. I turned on debugging on that interface to try and catch some more data. Also, a show int for that interface tells me I have had 75 lost carriers since the last clear counter.... the switch doesnt see that happening.

ROUTER:

=======

Ethernet0 is up, line protocol is up

Hardware is QUICC Ethernet, address is

MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec, rely 255/255, load 1/255

Encapsulation ARPA, loopback not set, keepalive set (10 sec)

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:21, output 00:00:01, output hang never

Last clearing of "show interface" counters 4w5d

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue 0/40 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

44109683 packets input, 343236113 bytes, 32 no buffer

Received 381637 broadcasts, 0 runts, 0 giants, 32 throttles

3 input errors, 0 CRC, 3 frame, 0 overrun, 0 ignored, 0 abort

0 input packets with dribble condition detected

42865306 packets output, 3947051696 bytes, 0 underruns

75 output errors, 273284 collisions, 1 interface resets

0 babbles, 0 late collision, 191638 deferred

75 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

Switch:

=====

FastEthernet0/10 is up, line protocol is up

MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive not set

Auto-duplex (Half), Auto Speed (10), 100BaseTX/FX

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:17, output 00:00:00, output hang never

Last clearing of "show interface" counters never

Queueing strategy: fifo

Output queue 0/40, 0 drops; input queue 0/75, 0 drops

5 minute input rate 4000 bits/sec, 4 packets/sec

5 minute output rate 5000 bits/sec, 4 packets/sec

3112869 packets input, 1131096987 bytes

Received 9814 broadcasts, 0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 2250 ignored

0 watchdog, 5216 multicast

0 input packets with dribble condition detected

3512125 packets output, 372329410 bytes, 0 underruns

0 output errors, 11344 collisions, 1 interface resets

0 babbles, 0 late collision, 42300 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

[Kevin Dorrell]

There might be a clue here, or it might be a symptom. I notice that the router has "received" 32 throttles, one for each day since the counters were last reset. Co-incidence maybe? I asked myself what throttles were about, and I found this explanation:

Throttles are a good indication of an overloaded router. They show the number of times the receiver on the port has been disabled, possibly due to buffer or processor overload. Together with high CPU utilization on an interrupt level, throttles indicate that the router is overloaded with traffic.

This was from http://www.cisco.com/en/US/products/hw/routers/ps359/products_tech_note09186a00801c2af0.shtml

It was news to me that the router could disable the port when it gets overloaded. So the question is really, is there a peak load at that hour of the morning: a backup or something like that?

75 output errors and 75 lost carriers? That is an average of 2 per day, plus 11 extras. Does the syslog always show two carrier breaks during the incident, or are there sometimes three?

That's all I have for the moment, but I'll get back to you if I think of anything else.

Kevin Dorrell

Luxembourg