Solved: Re: Solving output drops and input errors

jtillman11 · ‎10-17-2018

Below is output from the LAN interface on our Cisco 1941 router. This interface connects directly to a port on an HP Procurve switch. Note the output errors, CRC error, and input errors in bold. I have read several posts and articles describing what these are but I am at a loss at how to correct them. I can post additional information as needed.

GigabitEthernet0/0 is up, line protocol is up
Hardware is CN Gigabit Ethernet, address is a493.4c42.d4e0 (bia a493.4c42.d4e0)
Description: G0/0 LAN Interface
Internet address is 10.0.3.1/24
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 2/255, rxload 4/255
Encapsulation 802.1Q Virtual LAN, Vlan ID 1., loopback not set
Keepalive set (10 sec)
Full Duplex, 100Mbps, media type is RJ45
output flow-control is unsupported, input flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/2 (size/max/drops/flushes); Total output drops: 2698225
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1866000 bits/sec, 276 packets/sec
5 minute output rate 785000 bits/sec, 224 packets/sec
3810276624 packets input, 1578361743 bytes, 2697505 no buffer
Received 57105739 broadcasts (35039136 IP multicasts)
0 runts, 0 giants, 0 throttles
2 input errors, 1 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 44759351 multicast, 0 pause input
2365867680 packets output, 921099490 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
9 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

Joseph W. Doherty · ‎10-19-2018

"According to the whitepaper, the maximum throughput of the 1941 is 569Mbps."

Actually the whitepaper has the 1941's maximum throughput as 2,932 Mpps (see table one). Your 569 Mbps value is from table four, correct?

This gen of ISRs performance depends very much on traffic mix and configuration. It's very, very variable.

"I don't see a limit of 25Mbps in that whitepaper, am I overlooking something?"

No, it's not a limit, just a Cisco recommendation, see figure one (end of document).

Those Cisco recommendations are very conservative, to help insure a certain level of performance might always be obtained, regardless of traffic or configuration. But, indeed, when not doing VPN, QoS or PAT (etc.), yes likely you can obtain more than 25 Mbps (duplex). That said, I would consider a 1941 a bit light for full 100 Mbps (if that's what you're working with). If your CPU isn't peaking at 100%, you should be good for now.

View solution in original post

Daniele Giordano · ‎10-17-2018

Hi, the show interface shows "Last clearing of "show interface" counters never".
This means that no one has cleared the counters.

What is the uptime? If the uptime is high, few errors like 2 input errors, 1 CRC are completely normal.

The strange conuter is the Total output drops: 2698225.

Output drops means that the interface is congested.

The hardware is 'CN Gigabit Ethernet' but the interface has negotiated 'Full Duplex, 100Mbps, media type is RJ45'.

So, if sometimes the used bandwidth is greater than 100 Mbit/s the packets are dropped.

Check the counters named:

5 minute input rate 1866000 bits/sec, 276 packets/sec
5 minute output rate 785000 bits/sec, 224 packets/sec

Reduce the 'load interval' and check for congested interval.

You can monitor the used bandwidth using SNMP tools like nagios, zabbix or other just to have the history of the utilization.

More info about drops are at this link:

https://www.cisco.com/c/en/us/support/docs/routers/10000-series-routers/6343-queue-drops.html

Regards.

jtillman11 · ‎10-17-2018

Where/how would I reduce the 'load interval'? The system uptime is 8 weeks. I use SolarWinds Network Performance Monitor for SNMP monitoring. Below are the results for this interface. The blue bar is the average bps received and the green is average bps transmitted. The utilization meter shows only 8% receive utilization and 1% transmit utilization.

jtillman11 · ‎10-17-2018

I changed the load-interval for that interface to 30sec. I then changed my SolarWinds polling schedule to poll every 1 minute (lowest I could set it).

Daniele Giordano · ‎10-17-2018

Ok, well done.

CRC are very low for 8 weeks, so CRC are not a problem.

My suggestion is to clear the counters using 'clear counter interface ...' and check if the interface is congested during next days.
Checks for spike also.... sometimes spikes are not captured using snmp.

Regards.

jtillman11 · ‎10-17-2018

Will do. What would be the best way to find spikes and determine the cause of them?

Leo Laohoo · ‎10-17-2018

"2" CRC is nothing. I'm not really concerned about it.
The Total Output Drops, however, is a concern. Is QoS enabled on this port?

dbeattie · ‎10-18-2018

As others have said, the 2 CRCs are unlikely to be an issue. I note that your queue depth is default and I can't see if you have a QoS policy applied. Since the output queue has only 2 flushes and no drops, and there are no other indications of output errors, can you confirm if you have a QoS policy, Access List or IOS firewall applied to the interface?

Thanks,

Dave

jtillman11 · ‎10-18-2018

There is no QoS policy applied to the interface. I do have PBR applied to the interface to route traffic coming from a video surveillance server over a different WAN link, but that is it.

interface GigabitEthernet0/0
description G0/0 LAN Interface
ip address 10.0.3.1 255.255.255.0
ip pim dense-mode
ip policy route-map DW_MWAVE_TRAFFIC
duplex auto
speed auto
end

route-map DW_MWAVE_TRAFFIC, permit, sequence 10
Match clauses:
ip address (access-lists): DW_2
Set clauses:
ip next-hop 192.168.100.3
Policy routing matches: 1260529 packets, 295909881 bytes
route-map DW_MWAVE_TRAFFIC, permit, sequence 20
Match clauses:
ip address (access-lists): DW_1
Set clauses:
ip next-hop 192.168.100.1
Policy routing matches: 2612228269 packets, 3854077734 bytes

Extended IP access list DW_1
10 permit ip host 10.0.3.50 10.0.0.0 0.0.0.255 (2612234398 matches)
Extended IP access list DW_2
10 permit ip host 10.0.3.50 10.0.6.0 0.0.0.255 (1260558 matches)

paul driver · ‎10-18-2018

Hello

@jtillman11 wrote:

Below is output from the LAN interface on our Cisco 1941 router. This interface connects directly to a port on an HP Procurve switch. Note the output errors, CRC error, and input errors in bold. I have read several posts and articles describing what these are but I am at a loss at how to correct them. I can post additional information as needed.

GigabitEthernet0/0 is up, line protocol is up
Hardware is CN Gigabit Ethernet, address is a493.4c42.d4e0 (bia a493.4c42.d4e0)
Description: G0/0 LAN Interface
Internet address is 10.0.3.1/24
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 2/255, rxload 4/255
Encapsulation 802.1Q Virtual LAN, Vlan ID 1., loopback not set

Due you have any sub-interfaces relating to this interface?

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

jtillman11 · ‎10-18-2018

I do! Here is their information.

GigabitEthernet0/0.2 is up, line protocol is up
Hardware is CN Gigabit Ethernet, address is a493.4c42.d4e0 (bia a493.4c42.d4e0)
Internet address is 10.0.103.254/24
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 3/255, rxload 5/255
Encapsulation 802.1Q Virtual LAN, Vlan ID 2.
ARP type: ARPA, ARP Timeout 04:00:00
Keepalive set (10 sec)
Last clearing of "show interface" counters never

interface GigabitEthernet0/0.2
encapsulation dot1Q 2
ip address 10.0.103.254 255.255.255.0
ip pim dense-mode
end

GigabitEthernet0/0.3 is up, line protocol is up
Hardware is CN Gigabit Ethernet, address is a493.4c42.d4e0 (bia a493.4c42.d4e0)
Internet address is 10.0.43.254/24
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 3/255, rxload 5/255
Encapsulation 802.1Q Virtual LAN, Vlan ID 17.
ARP type: ARPA, ARP Timeout 04:00:00
Keepalive set (10 sec)
Last clearing of "show interface" counters never

interface GigabitEthernet0/0.3
encapsulation dot1Q 17
ip address 10.0.43.254 255.255.255.0
ip pim dense-mode
end

dbeattie · ‎10-18-2018

OK - it is unusual to show interface drops without having a cause, so I am starting to think of bug-type scenarios. My first thought would be issues concerning PIM dense-mode on multiple sub-interfaces on the same interface. Do your multi-cast services seem to be working properly?

Joseph W. Doherty · ‎10-18-2018

Input queue: 0/75/0/2 (size/max/drops/flushes); Total output drops: 2698225
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1866000 bits/sec, 276 packets/sec
5 minute output rate 785000 bits/sec, 224 packets/sec
3810276624 packets input, 1578361743 bytes, 2697505 no buffer

You may need to do some buffer tuning, or enable buffers tune automatic (if supported).

BTW, Cisco recommends a 1941 for up to 25 Mbps (duplex).

jtillman11 · ‎10-18-2018

Thanks! I've never messed with buffers so any documentation you could point me to?

Regarding the throughput, could you show me the documentation stating that? It's hard to find a solid answer on what the actual throughput is on a 1941 but the majority of things I've read say 140-150Mbps.

Joseph W. Doherty · ‎10-18-2018

"Thanks! I've never messed with buffers so any documentation you could point me to?"

https://www.cisco.com/c/en/us/support/docs/routers/10000-series-routers/15091-buffertuning.html

However, again, if your platform supports, recommend you use the buffers auto tune feature.

"Regarding the throughput, could you show me the documentation stating that?"

See attachment.