SNMP EngineID same on multiple routers ?

[scain2]

Hi, just discovered that 3 out of over 200 of our routers are having issues with our management station as they have the same snmp enginerID !?!

Deployed snmp v3 to over 200 routers and all functioning fine, however have found 3 routers, 2 running 15.1.4 M2, 1 running 15.1.4 M1 have same engine id when using sh snmp engineID. Have not set engineID on any router ever. These were new routers deployed as part of WAN upgrade and only ones running 15.1. I don't want to manually set engineID unless last resort. Cannot find anything in bug toolkit for 15.1.4 relating to engineID

Have discovered that the snmp engineID used MAC add of 1st interface, on each of these its and embedded service engine interface0/0, this is shutdown on each router but does have a MAC add of 0000.0000.0000, and appears to giving the same engineID on each router

[Joe Clarke]

What is the engine ID assigned, and what is the specific image name you are running?  You may have very well found a new bug.

[scain2]

The engine id on all 3 routers is :

Local SNMP engineID: 800000090300000000000000

Remote Engine ID          IP-addr    Port

Images are

c3900-universalk9-mz.SPA.151-4.M2.bin

c2951-universalk9-mz.SPA.151-4.M1.bin

I've found embedded-service-engine interface if always 1st interface on router with following mac settings

Embedded-Service-Engine0/0 is administratively down, line protocol is down

  Hardware is Embedded Service Engine, address is 0000.0000.0000 (bia 0000.0000.0000)

So snmp engineID is always

800000090300000000000000

[jobalt116]

I have the same exact issue, but different IOS and many more than 3 routers...50+:

c2900-universalk9-mz.SPA.152-3.T.bin

[Joe Clarke]

What is the engine ID?  Did you copy and paste the config from device to device?  I couldn't reproduce on my 15.2(3)T devices.

[scain2]

Hi, I did report to Cisco and they were going to look into possible bug fix, not seen reply yet.

The workaround I used was to manually set the snmp engine id using a MAC add from one of the interfaces.

Issue: Same snmp engineID local appears on multiple routers.

show snmp engineID gives:

Local SNMP engineID: 800000090300000000000000

Remote Engine ID          IP-addr    Port

This causes issues with some monitoring systems as sees multiple devices with same snmp engineID and not pick up monmitoring and issues with snmp v3 users passwords.

I have found this to be down to, embedded-service-engine interface, is always 1st interface on router with following mac settings

Embedded-Service-Engine0/0 is administratively down, line protocol is down

  Hardware is Embedded Service Engine, address is 0000.0000.0000 (bia 0000.0000.0000)

Engine ID is made up of Cisco standard id 80000009030 + MAC of 1st interface 0000.0000.0000

So snmp engineID is always

800000090300000000000000

So I have just used the command snmp-server engineID local 80000009030070CA9B42E980

Where 070CA9B42E980 is a MAC add of an interface on the router.

Worked fine for me, hope this helps out.

[Joe Clarke]

This is bug CSCts87275.  It is fixed in 15.3(1)T and higher for the ISRG2.

[Joe Clarke]

FYI, the workaround is manually configure a unique SNMP engineID on all affected platforms.

[jobalt116]

The engine ID I have is 800000090300000000000000 as well. We did copy the config from a template router to the rest to deploy them, but the engineID was not set in the config. I wouldn't think that this would carry over. It should, or at least I thought it should, generate this from the mac address.

I attempted to statically set the engineID, but unfortunately I still cannot establish an snmpv3 session on multiple routers. I did discover that even though I ran the command no snmp-server user etc., but I still see the user with engineID of 800000090300000000000000 when I issue a show snmp user... I am assuming this is the problem, and I am assuming that a reboot would fix this but that is not an option right now.

On another note, all of these routers are 2911s. I have ONE 2921 running 15.3(1)T and even though the embedded Service Engine has a mac of 0000.0000.0000, the engine ID was pulled from the NEXT interface (Gi0/0) for the engine ID...strange.

[scain2]

Hi, I used a template copied to 180+ Cisco 1801 routers and had no problem with them (although they did not have embedded service engines) All I set was a snmp-server view,snmp-server group and snmp-server user.

I used the same template on the 3945 with embedded service engine, and gave same snmp engine id on each one using MAC add of 0000.0000.0000.

I deleted all snmp settings then set snmp engine-id local 80000009030MACadd of an interface, then same template as before snmp view,snmp group and snmp user and all ok.

I haven't had to reboot to get snmp to work before, but have had to delete all snmp settings and re-add.

Another issue I have had with snmpv3 is that the user setting does not show when using show snmp-ser user (and it is not put into the show running-config anyway), but if there is an issue it creates another line above the snmp-server group line, with something like snmp-server USER FFFF.FFFF.FFFFF, and have found I have had to delete this line and then recreate the snmp-server user command again and all is fine

hope this helps feel free to contact me via this and let me know how you get on.

[jobalt116]

Well, the mostly good news is that changing the engine ID worked for all of my other routers. There is just this one that is fighting with me. I was able to remove the "stuck" user, but I cannot create a successful snmp connection to the device. I even removed all snmp config, saved, rebooted, added v3 config, and still have the same issue.

sh snmp:

...

SNMP Engine:

   queue 0/1000 (current/max), 0 dropped

    0 Unknown Security Models

    0 SNMP Invalid Messages

    0 SNMP Unknown PDU handlers

    0 Unsupported Security Level

    0 Unknown User Names

    12 Unknown EngineIDs

    0 Not In Time Windows

    0 Wrong MD5 or SHA Digests

    0 Decryption Errors

debug snmp detail

debug snmp packets:

Apr 24 18:17:35.609: SNMP: Packet received via UDP from {ipaddress} on {interface}

SrParseV3SnmpMessage: No matching Engine ID.

SrParseV3SnmpMessage: Failed.

SrDoSnmp: authentication failure, Unknown Engine ID

Apr 24 18:17:35.613: SNMP: Report, reqid 24746175, errstat 0, erridx 0

internet.6.3.15.1.1.4.0 = 12

Apr 24 18:17:35.613: SNMP: Packet sent via UDP to {ipaddress}

process_mgmt_req_int: UDP packet being de-queued

[Joe Clarke]

In what order did you do this?  The engineID must be configured before any user.  If you configured the engineID after configuring the user, then you will need to repeat the process.

[jobalt116]

Must've been the wrong order, because re-doing it was the fix....thanks!!

[scain2]

Glad you got it fixed, one to remember for another time.