𝐒𝐮𝐦𝐦𝐚𝐫𝐲:
SNMP Engine hangs and enters a loop until eventually reporting %SNMP-3-INPUT_QFULL_ERR when the SNMP queue reaches 1000/1000.
%SNMP-3-INPUT_QFULL_ERR related bugs CSCvj26291, CSCvm33538 and CSCvn31630 appear to have been reintroduced in 15.7(3)M9.
This information is to assist anyone encountering a %SNMP-3-INPUT_QFULL_ERR Cisco SNMP Engine bug, which affects but is not limited to CISCO1941/K9 routers with an xDSL EHWIC.
After in-depth testing and passing this information to Cisco TAC, they declined a request to update any related known bugs with the findings, as the software is 'End of Software Maintenance'.
𝐃𝐞𝐬𝐜𝐫𝐢𝐩𝐭𝐢𝐨𝐧:
On a CISCO1941/K9 router with an xDSL EHWIC, EHWIC-VA-DSL-A, running IOS Version 15.6(3)M8 -
SNMP queries to multiple Abstract Syntax Notation One (ASN.1) 'transmission' Object Identifiers (OIDs), {iso(1) identified-organization(3) dod(6) internet(1) mgmt(2) mib-2(1) transmission(10)} / 1.3.6.1.2.1.10, causes the SNMP Engine to stop responding to further SNMP requests until the SNMP Engine Queue is cleared.
In simpler terms, SNMP queries to multiple OIDs, within the ADSL-LINE-MIB.mib and VDSL2-LINE-MIB.mib MIBs, causes the SNMP Engine to stop responding to further SNMP requests until the SNMP Engine Queue is cleared and in some cases restarted.
𝐒𝐲𝐦𝐩𝐭𝐨𝐦𝘀:
SNMP queries to multiple OIDs in the ADSL-LINE-MIB.mib and VDSL2-LINE-MIB.mib MIBs, cause the SNMP Engine to stop responding to further SNMP requests until the SNMP Engine Queue is cleared.
Device logs may show the following errors, if the SNMP Engine Queue becomes full, e.g. following 1000 or more affected SNMP queries:
%SNMP-3-INPUT_QFULL_ERR: Packet dropped due to input queue full
𝗖𝗮𝘃𝗲𝗮𝘁𝘀:
The issue only appears to affect routers with an xDSL Interface which is in the 'down/initializing' Status.
The issue does not affect routers with an xDSL Interface which is in the ‘administratively down’ Status.
The issue does not affect routers with an xDSL Interface which is in the ‘up’ Status.
Cisco Bug CSCvn31630, states the issue is resolved in 15.7(3)M4, however, when testing, the issue occurs in 15.7(3)M9.
Cisco Bug CSCvj26291, states the issue is resolved in 15.7(3)M3, however, when testing, the issue occurs in 15.7(3)M9.
Cisco Bug CSCvm33538, states the issue is resolved in 15.7(3)M4, however, when testing, the issue occurs in 15.7(3)M9.
𝐀𝐝𝐝𝐢𝐭𝐢𝐨𝐧𝐚𝐥 𝐍𝐨𝐭𝐞𝐬:
Upgrading CISCO1941/K9 router, running IOS Version 15.6(3)M8, with a EHWIC-VA-DSL-A, to the current Cisco recommended version IOS version 15.7(3)M9 is not feasible, as the issue occurs on devices running 15.7(3)M9.
On a device running 15.7(3)M9, a restart of the SNMP Engine is required as an additional step, after applying the workaround; before the issue is be resolved.
An SNMP Engine restart can be performed by issuing the configuration command 'no snmp-server', followed by any previous 'snmp-server' configuration line to restore the configuration without any loss.
Cisco bugs indicate an upgrade to software version 15.8(3)M1 or higher may resolve the issue.
𝗧𝗲𝘀𝘁𝗶𝗻𝗴:
Tested against a CISCO1921/K9 router, with a EHWIC-VA-DSL-A, running IOS Version 15.7(3)M4, where the ATM interface Status is 'down' and the Protocol is 'down', and the issue is present.
Tested against a CISCO1941/K9 router, with a EHWIC-VA-DSL-A, running IOS Version 15.7(3)M9, where the ATM interface Status is 'down' and the Protocol is 'down', and the issue is present.
Tested against a CISCO1941/K9 router, with a EHWIC-VA-DSL-A, running IOS Version 15.8(3)M7, where the ATM interface Status is 'down' and the Protocol is 'down', and the issue appears to be resolved.
Tested against a C897VA-K9 router, running IOS Version 15.8(3)M7, where the ATM interface Status is 'initializing' and the Protocol is 'down', and this issue appears to be resolved.
𝐑𝐞𝐜𝐨𝐦𝐦𝐞𝐧𝐝𝐚𝐭𝐢𝐨𝐧𝘀:
Upgrade affected CISCO1941/K9 and any affected routers, with an EHWIC-VA-DSL-A, to the current Cisco Latest release, IOS version 15.8(3)M7
Where IOS version 15.8(3)M7 is not available for older router models, upgrade affected router hardware to a newer model or implement the workaround bearing in mind the CAVEAT NOTE.
𝐖𝐨𝐫𝐤𝐚𝐫𝐨𝐮𝐧𝐝:
Apply an SNMP view and Exclude the ADSL-LINE-MIB and the VDSL2-LINE-MIB from the SNMP view:
Filter SNMP queries to affected OIDs, using SNMP views.
CAVEAT NOTE: The workaround may affect the ability of Network Management Systems to collect xDSL capacity data, so should only be applied if a software upgrade is not possible, or if the Data from the MIBs is not required.
Filtering can be applied to the Parent Node Name OIDs, as per the following example:
snmp-server view <filteroid> adslLineMib excluded
snmp-server view <filteroid> vdsl2MIB excluded
𝐀𝐧 𝐞𝐱𝐚𝐦𝐩𝐥𝐞 of the configuration, using an SNMPv2 view called filteroid, would be as follows:
snmp-server view filteroid iso included
snmp-server view filteroid vdsl2MIB excluded
snmp-server view filteroid adslLineMib excluded
snmp-server community <communityname> view filteroid RO 10
𝐀𝐥𝐭𝐞𝐫𝐧𝐚𝐭𝐢𝐯𝐞𝐥𝐲, Exclude specifically affected OIDs, for the ADSL-LINE-MIB and the VDSL2-LINE-MIB, from the SNMP view:
Filtering can be applied to the Children Node Name OIDs, as per the following example:
snmp-server view <filteroid> adslLineTable excluded
snmp-server view <filteroid> adslAtucPhysTable excluded
snmp-server view <filteroid> adslAturPhysTable excluded
snmp-server view <filteroid> adslAtucChanTable excluded
snmp-server view <filteroid> adslAturChanTable excluded
snmp-server view <filteroid> adslAtucPerfDataTable excluded
snmp-server view <filteroid> adslAturPerfDataTable excluded
snmp-server view <filteroid> adslAtucChanPerfDataTable excluded
snmp-server view <filteroid> adslAturChanPerfDataTable excluded
snmp-server view <filteroid> adslAtucChanIntervalTable excluded
snmp-server view <filteroid> adslAturChanIntervalTable excluded
snmp-server view <filteroid> xdsl2Objects excluded
