02-21-2025 11:53 AM - edited 02-21-2025 11:54 AM
Hello,
We operate a ISE cluster where two of them do show some issue with snmp traps / polling:
Cisco Identity Services Engine
---------------------------------------------
Version : 3.2.0.542
Build Date : Wed Oct 19 16:27:24 2022
Install Date : Wed Jul 26 16:49:04 2023
Cisco Identity Services Engine Patch
---------------------------------------------
Version : 2
Config Example:
snmp-server enable
snmp-server trap dskThresholdLimit 25
snmp-server engineid <engine ID>
snmp-server host <NMS IP> version 2c <Community String>
snmp-server host <NMS IP> version 2c <Community String>
test with snmp walk
$ snmpwalk -v 2c -c <Community String> <NMStationIP1> 1.3.6.1.2.1.1
Based on Wireshark capture:
Seq SRC DST Info
1 NMS IP ISE Node get-next-request 1.3.6.1.2.1.1
2 ISE Node NMS snmpv2-trap 1.3.6.1.2.1.1.3.0 1.3.6.1.6.3.1.1.4.1.0 1.3.6.1.6.3.1.1.4.3.0
3 NMS IP ISE Node Destination unreachable (Port unreachable)
Syslog:
2025-02-20T01:22:46.623491+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpcfg.c[71] [system]: Config Success
2025-02-20T01:22:46.627579+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpd_conf.c[713] [system]: Configured SysContact Unknown
2025-02-20T01:22:46.627775+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpd_conf.c[729] [system]: Configured Syslocation Unknown
2025-02-20T01:22:46.627814+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpd_conf.c[899] [system]: DiskThresholdPercentage set to 25
2025-02-20T01:22:46.627847+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpd_conf.c[819] [system]: Configured trapcommunity set to default
2025-02-20T01:22:46.627879+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpd_conf.c[496] [system]: Version: v2c
2025-02-20T01:22:46.627911+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpd_conf.c[514] [system]: TrapCommunity: <CommunityString>
2025-02-20T01:22:46.627939+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpd_conf.c[550] [system]: Inserted first host/ip <NMStationIP1>
2025-02-20T01:22:46.627970+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpd_conf.c[835] [system]: Adding Trap clients
2025-02-20T01:22:46.628002+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpd_conf.c[853] [system]: snmp server started
2025-02-20T01:22:46.628033+00:00 ISE-Node-u5 ADE-SERVICE[1790]: [6078]:[info] snmp: cars_snmpcfg.c[97] [system]: Config Success
:
/opt/CSCOcpm/appsrv/apache-tomcat-8.5.13/lib/snmp4j-2.3.1.jar:
/opt/CSCOcpm/appsrv/apache-tomcat-8.5.13/lib/snmptrap-2.4.0-357.jar:
/opt/CSCOcpm/appsrv/apache-tomcat-8.5.13/lib/snmpquery-2.4.0-357.jar:
/opt/CSCOcpm/appsrv/apache-tomcat-8.5.13/lib/snmpinfrastructure-2.4.0-357.jar:
Attribute:ProbeName value:snmpquery
Attribute:probeclass value:com.cisco.profiler.probes.snmpquery.SNMPQuery
Attribute:ProbeName value:snmptrap
Attribute:probeclass value:com.cisco.profiler.probes.snmptrap.SNMPTrapListener
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.16/lib/snmptrap-2.7.0-356.jar:
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.16/lib/snmpquery-2.7.0-356.jar:
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.16/lib/snmpinfrastructure-2.7.0-356.jar:
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.16/lib/snmp4j-2.6.2.jar:
Attribute:ProbeName value:snmpquery
Attribute:probeclass value:com.cisco.profiler.probes.snmpquery.SNMPQuery
Attribute:ProbeName value:snmptrap
Attribute:probeclass value:com.cisco.profiler.probes.snmptrap.SNMPTrapListener
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.54/lib/snmpquery-3.2.0-542.jar:
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.54/lib/snmptrap-3.2.0-542.jar:
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.54/lib/snmpinfrastructure-3.2.0-542.jar:
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.54/lib/snmp4j-2.6.2.jar:
Attribute:ProbeName value:snmpquery
Attribute:probeclass value:com.cisco.profiler.probes.snmpquery.SNMPQuery
Attribute:ProbeName value:snmptrap
Attribute:probeclass value:com.cisco.profiler.probes.snmptrap.SNMPTrapListener
drop table UPSWizrdCnfgrtn_snmpIpRngs
create table UPSWizrdCnfgrtn_snmpIpRngs
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.54/lib/snmpinfrastructure-3.2.0-542.jar: FAILED
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.54/lib/snmpquery-3.2.0-542.jar: FAILED
/opt/CSCOcpm/appsrv/apache-tomcat-9.0.54/lib/snmptrap-3.2.0-542.jar: FAILED
verified: /opt/CSCOcpm/appsrv/apache-tomcat-9.0.54/lib/snmpinfrastructure-3.2.0-542.jar from: patch 2
verified: /opt/CSCOcpm/appsrv/apache-tomcat-9.0.54/lib/snmpquery-3.2.0-542.jar from: patch 2
verified: /opt/CSCOcpm/appsrv/apache-tomcat-9.0.54/lib/snmptrap-3.2.0-542.jar from: patch 2
Solved! Go to Solution.
02-26-2025 01:50 PM - edited 02-26-2025 01:51 PM
Ok, I may have found the answer.
I agree, this ISE is loaded with bugs, at least it keeps the Cisco engineers busy/employed:
02-23-2025 02:14 PM
SNMP in ISE has been quite buggy historically. In SE 3.3p4 it's finally stable. I had issues with older versions - you might want to patch your system. The only thing you can do as a user (apart from patching and upgrading) is to toggle the SNMP server off, and on again. Or I also had success in the past with disabling SNMP, rebooting the node, and then enabling it again. If it's still broken, then engage TAC.
Also check that there are no firewall rules that might be blocking UDP/161 and UDP/162
02-26-2025 01:42 PM
Hi Arne,
Thank you again. With the help of snmpwalk I cycled through all three existing ISE clusters [all three running different release] and it turned out that all nodes in one cluster do not receive snmp responses; firewall issue can be ruled out. Therefore, I am wondering, if there is a known issue with release Version: 3.2.0.542.
Result:
Version: 3.2.0.542, patch 2, ADE-OS Version: 3.2.0.401 ==> no snmp response
Version: 2.7.0.356, patch 2,3,6, ADE-OS Version: 2.7.0.356 ==> working
Version: 3.1.0.518, patch 3, ADE-OS Version: 3.1.0.518 ==> working
Thank you again.
I appreciate any help/feedback
02-26-2025 01:50 PM - edited 02-26-2025 01:51 PM
Ok, I may have found the answer.
I agree, this ISE is loaded with bugs, at least it keeps the Cisco engineers busy/employed:
02-26-2025 02:11 PM
Working one:
Version : 2.7.0.356
# show process | include snmp
root 27611 pts/4 27607 Wed Feb 26 22:09:50 2025 grep -A 0 -B 0 snmp 00:00:00
--
root 29317 ? 1 Thu Jan 5 19:53:19 2023 /usr/sbin/snmpd -LS0-6d -Lf 09:26:20
Non-working one:
Version : 3.2.0.542
n#show process | include snmp
root 3662629 ? 1 Thu Feb 20 01:34:33 2025 /usr/sbin/snmpd -LS0-6d -Lf 00:06:3
02-26-2025 02:14 PM
The fix is either to upgrade to a less bug-full version or remove and add snmp config; the latter may have to be repeated a number of times. Reloading the node might be another option, but without Cisco coverage there is always a risk to run into another surprise. Thank all for your comments and help.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide