SG500 - Fatal Errors & Reboots

[DJX995]

These fatal errors and subsequent reboots are observed on my SG500-52P when I was reconfiguring the LAG ports on my server.

I had the ports added to a LAG on the switch but the ports on the server were still just regular access ports.

I was doing some other things and left it like this for a little while but the switch was very angry...

Also, please note that, after these reboots on it's own, the switch actually stopped passing traffic and had to be manually power cycled by hand to restore connectivity.

2147477095	2013-Oct-10 13:28:09	Emergency	%2SWIPMBRG-F-IPMBRGADDEGRESSPORTS: Failed to add port 1001 to videx 4096 ,Err message Operation failed  ***** FATAL ERROR *****  Reporting Task: BRMN. Softw are Version: 1.3.5.58 (date  10-Oct-2013 time  17:17:15) 0x16b9ec 0x167b58 0x762560 0x4e8fe8 0x4e98f8 0x4e9b18 0xa06870 0x7b1a34 0x5c39ec 0x5c442c 0x 5ccc74 0x5b0384 0x5baedc 0x69fe18 0x1223f0  ***** END OF FATAL ERROR *****
2147478733	2013-Oct-10 13:29:12	Emergency	%2SWIPMBRG-F-IPMBRGADDEGRESSPORTS: Failed to add port 1001 to videx 4096 ,Err message Operation failed  ***** FATAL ERROR *****  Reporting Task: BRMN. Softw are Version: 1.3.5.58 (date  10-Oct-2013 time  17:17:15) 0x16b9ec 0x167b58 0x762560 0x4e8fe8 0x4e98f8 0x4e9b18 0xa06870 0x7b1a34 0x5c39ec 0x5c442c 0x 5ccc74 0x5b0384 0x5baedc 0x69fe18 0x1223f0  ***** END OF FATAL ERROR *****
2147480371	2013-Oct-10 14:46:35	Emergency	%2SWIPMBRG-F-IPMBRGADDEGRESSPORTS: Failed to add port 1001 to videx 4096 ,Err message Operation failed  ***** FATAL ERROR *****  Reporting Task: BRMN. Softw are Version: 1.3.5.58 (date  10-Oct-2013 time  17:17:15) 0x16b9ec 0x167b58 0x762560 0x4e8fe8 0x4e98f8 0x4e9b18 0xa06870 0x7b1a34 0x5c39ec 0x5c442c 0x 5ccc74 0x5b0384 0x5baedc 0x69fe18 0x1223f0  ***** END OF FATAL ERROR *****
2147482009	2013-Oct-10 13:33:49	Emergency	%2SWIPMBRG-F-IPMBRGADDEGRESSPORTS: Failed to add port 1001 to videx 4096 ,Err message Operation failed  ***** FATAL ERROR *****  Reporting Task: BRMN. Softw are Version: 1.3.5.58 (date  10-Oct-2013 time  17:17:15) 0x16b9ec 0x167b58 0x762560 0x4e8fe8 0x4e98f8 0x4e9b18 0xa06870 0x7b1a34 0x5c39ec 0x5c442c 0x 5ccc74 0x5b0384 0x5baedc 0x69fe18 0x1223f0  ***** END OF FATAL ERROR *****
2147483647	2013-Oct-18 14:02:41	Emergency	%2SWIPMBRG-F-IPMBRGADDEGRESSPORTS: Failed to add port 1001 to videx 4096 ,Err message Operation failed  ***** FATAL ERROR *****  Reporting Task: BRMN. Softw are Version: 1.3.5.58 (date  10-Oct-2013 time  17:17:15) 0x16b9ec 0x167b58 0x762560 0x4e8fe8 0x4e98f8 0x4e9b18 0xa06870 0x7b1a34 0x5c39ec 0x5c442c 0x 5ccc74 0x5b0384 0x5baedc 0x69fe18 0x1223f0  ***** END OF FATAL ERROR *****

[Tom Watts]

Hi Fratiani,

Can you provide the follow0

• Config file
• Show tech
• Simple topology diagram including configuration snippit of whatever is LAG connection to the switch
• How often does the error occur and does it seem to be same time always, sporadic times or inconsistent, number of hours or days elapsed?
• Any network activity at the time of the problem or an idle network?

If you're not comfortable to post this information please email me at tmw0402@hotmail.com

-Tom
Please mark answered for helpful posts

[DJX995]

Got this sometime last night:

2147475097

2014-Feb-24 04:21:32

Emergency

%2SWIPMBRG-F-IPMBRGADDEGRESSPORTS: Failed to add port 1001 to videx 4096 ,Err message Operation failed  ***** FATAL ERROR *****  Reporting Task: BRMN. Softw are Version: 1.3.7.18 (date  12-Jan-2014 time  18:04:29) 0x16b9f8 0x167b64 0x762c24 0x4e9270 0x4e9b80 0x4e9da0 0xa09294 0x7b2100 0x5c4028 0x5c4a68 0x 5cd2b0 0x5b09c0 0x5bb518 0x6a0454 0x1223fc  ***** END OF FATAL ERROR *****

Do you still want me to give you the above info?

[Tom Watts]

Hi Fratiani, can you provide steps to recreate or the configurations you are using ? In addition, a network topology including layer 2/3 nodes, IP addresses, VLAN schema will be very helpful.

-Tom
Please mark answered for helpful posts
http://blogs.cisco.com/smallbusiness/

[Nagaraja Thanthry]

Hello FratianiD,

Please contact Cisco Support and open a Service Request so we can investigate the root cause for the error messages you are getting. You can use the following contact information to reach Cisco support.

http://www.cisco.com/c/en/us/support/web/tsd-cisco-small-business-support-center-contacts.html

Thanks,

Nagaraja

[DJX995]

Nagaraja,

I can't contact support as my unit was purchased out of warranty.

Tom,

I'll email you the specifics if thats okay?

[Tom Watts]

Hi Fratiani, yes please send an email.

-Tom
Please mark answered for helpful posts
http://blogs.cisco.com/smallbusiness/

[Nagaraja Thanthry]

Hello FratianiD,

Please go ahead and contact Cisco Support and have them refer to this forum.

Thanks,

Nagaraja

[DJX995]

After working with Cisco, I just wanted to update this post with the information I received from them and the solution.

The root cause of this issue is explained below:
a.    The issue is caused by a timeout to an MLD join that device receives on one of its ports.
b.    Reboot will happen only if the MLD join is to one of the Link Local Multicast groups of which the switch is member. (this membership is added automatically under some IPv6 configurations. You can see this membership by using command “show bridge multicast address-table”.
c.    Examination of info from you leads us to believe that  MLD report to group  fe02::1:2 (All DHCP Agents) is the one that caused reboot.
d.    Device becomes member of this group due to command “ipv6 dhcp relay destination ” configured on VLAN 1.
e.    You reported that it is noticed sometimes when firewall server config is changed or rebooted, we assume that when this server comes up – it sends out an MLD report for the above group - fe02::1:2 .  (we can check this by capturing traffic on the server after reboot and checking if such message is sent).

A firmware with the related fix is scheduled for release in July.