topic Re: Notification Center in Cisco Catalyst Center 2.3.7.6 in Cisco Catalyst Center

Notification Center in Cisco Catalyst Center 2.3.7.6

Dominic Stalder — Thu, 29 Aug 2024 06:07:39 GMT

Hi guys

I just upgraded to Cisco Catalyst Center 2.3.7.6 and saw this "new" notification center. I can see a lot of error / warning / information events corresponding to the upgrade time yesterday day evening:

Now I have two questions:

1. Are those errors "still" true or is this an historical event notification? I don't think, that they are active, otherwise the Catalyst Center would not running "normally" like it does.

2. How can I dismiss or clear those historical events from the notification center?

Thanks and best regards

Dominic

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Mark Elsen — Thu, 29 Aug 2024 07:11:07 GMT

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Dominic Stalder — Thu, 29 Aug 2024 07:32:08 GMT

Hi M.

thanks, but I don't think it is in any way related to this bug.

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Mark Elsen — Thu, 29 Aug 2024 11:10:36 GMT

- This one : https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/tech_notes/b_cisco_dna_center_base_automation_services_troubleshooting_guide.html
comes from https://www.cisco.com/c/en/us/support/cloud-systems-management/dna-center/products-tech-notes-list.html
possibly something else in there could be useful

Or else invoke Cisco's TAC ,

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

maflesch — Thu, 29 Aug 2024 13:51:33 GMT

Hi Dominic,

As you stated, we have beefed up the notification center in 2.3.7.6. Unfortunately, there are still some kinks we are working out as some of the checks are a little aggressive. For the managed services check, it should do this every 10 minutes and will report if the the state is determined unhealthy each time. It will then report if the state moves to a healthy state and stop reporting until it moves back to an unhealthy state.

Unfortunately there is not a way to acknowledge them. They will clear up after a day usually. To confirm your system health, you can go to the Menu → System → System 360 → System Health → Tools → Validation Tool then choose Add and create a report for the system health.

Also, you can turn of these notification if you feel they are wrong by clicking on the gear icon and unchecking System Managed Services,

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Dominic Stalder — Thu, 29 Aug 2024 14:07:27 GMT

Hi maflesch

thanks a lot for the helpful information, just run the validation tool and it looks good. I will keep the notifcations enabled for the moment and will check tomorrow, if they are gone...

Thanks and best regards

Dominic

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Tomas de Leon — Thu, 29 Aug 2024 14:07:37 GMT

Dominic,

As you noticed after the upgrade, the notication center feature changed. This is phase 1 of the feature and there are already alot of improvements and enhancements that are coming in future releases. We have a list and if you would like to make any suggestions, please provide your comments using the "Make a Wish" tool in the Catalyst Center itself. Your feedback will be evaluated by the Product Management Team.

To answer you questions, the answer will be "it depends".

What I mean is that depending on the notification instance, the instance could be removed after some time or it may stay until the instance is addressed.

"Clearing or muting the notification instances" has been consistent feedback provided.

For the "Critical" notifications, I would suggest to look at the

/dna/systemSettings/system360/systemHealth

in the Catalyst Center UI. If the Critical Alert is there:

Use the "Validation Tool" and run a task for "Appliance Infrastructure Status, Upgrade Readiness Status, or all of the categories"
Collect a Support Bundle (from the UI) and collect "RCASupportBundle"
Open a Cisco TAC Case and the TAC engineer can triage your issues and provide you a plan of action.

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Dominic Stalder — Thu, 29 Aug 2024 14:09:26 GMT

Thanks also to you Tomas!

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Dominic Stalder — Fri, 30 Aug 2024 04:56:51 GMT

Good morning guys

Just wanted to let you know; I checked the notification center today and the alarms from two days ago are still schown:

But I have advised my colleagues to "ignore" those alarms.

But if there will be a solution for that, that would be great. I will "make a wish".

Best regards

Dominic

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

andrew.butterworth — Mon, 23 Sep 2024 09:03:33 GMT

I've upgraded the DNAC in our lab to CC 2.3.7.6 last week and have hit a similar issue. We have 20 errors all relating to Managed Services but with a 'PENDING_UPGRADE' in the message.

If you click on any of the Events, they expand out to this:

There are also 4 warnings - two regarding newer Trusted Certificate bundles being available and two relating to the CIMC adapter.

Any ideas how to resolve these

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

maflesch — Mon, 23 Sep 2024 13:13:10 GMT

Hi Andrew,

Unfortunately you cannot resolve this. It's a limitation to the notification center alerts that was added. We have raised the concerns with engineering and are looking to all the muting or resolving of alerts. However, this will not be available until a future release, with no set time yet. You could open a TAC case so that this case could be added to the list we already have, which will help push or show the importance of revamping this feature.

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

andrew.butterworth — Mon, 23 Sep 2024 16:08:30 GMT

OK, thanks for the update. What do they actually mean - 20 errors with 'PENDING_UPGRADE', following an upgrade of the software gives the impression part of the upgrade isn't complete.

The CIMC warnings are due to the firmware on the C220 M5 chassis being updated to a later release than what 2.3.7.6 was tested with. I'm happy to ignore this one, and I'm hoping I've solved the certificate warning by downloading the latest trusted certificate bundle.

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

maflesch — Thu, 26 Sep 2024 04:42:57 GMT

The alerts can be ignored. When you upgraded to 2.3.7.6 the system pushed out the new logic for the notification center while the upgrade was still going on. As such, the services reporting that they were "PENDING_UPGRADE" because at that time it was true. Unfortunately, there is no way to auto-resolve or ignore these alerts, so they persisted. That's part of the feature revamp we are pushing to engineering over this new logic.

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Eric R. Jones — Wed, 16 Oct 2024 22:29:38 GMT

Hello Dominic, I see you upgraded to 2.3.7.6. We having an issue with a bug that gets addressed in that version for physical deployments. We received information from Cisco stating "

It is currently available only for Virtual ESXi deployments, but Catalyst Center software release 2.3.7.6 is also available for physical devices to upgrade, We believe this issue has been addressed in newer release, but Catalyst Center release 2.3.7.6 is not Cisco suggested release based on software quality, stability and longevity." Have you experienced any issues besides the one your explaining here? We are on 2.3.7.4 and would like to clear this issue discovered in our scans.

We have had our Catalyst Center since February and haven't upgraded so wondering what the steps and possible pain points are?

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Eric R. Jones — Wed, 16 Oct 2024 22:47:17 GMT

Hello Dominic, I didn't think I was going to be able to read the release notes but I did. There are are 48 bugs and the top 4 have 3 that negatively affect us in a big way.

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Dominic Stalder — Fri, 18 Oct 2024 12:01:48 GMT

Hi Eric

sorry for my late response... But as you already noted and referenced the feedback from Cisco, this Cisco DNAC version of 2.3.7.6 is pretty buggy and I am not really happy, that I have upgraded to this version.

But because we don't use it that excessively or in a critical fashion; it does not hit us that hard and I can wait for the next, hopefully less buggy version.

Regards

Dominic

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

Eric R. Jones — Sun, 20 Oct 2024 21:25:14 GMT

Thank you for the reply. This provides more information when someone asks why we haven't addressed this particular CVE issue. Usually we just point out the documentation in the release notes but having a few testimonials addressing the issues at hand is always helpful.

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

JEFFREY SESSLER — Thu, 31 Oct 2024 22:13:19 GMT

I'm seeing similar things, but the errors just come back. It's as if the check fails and then recovers. Health check is clean but I've opened a TAC case just in case. I'm regretting upgrading to 2.3.6.7, and I keep seeing that there is a 2.3.6.9 referenced as fixing some stuff but I've yet to see that show up in release notes or for download.

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

maflesch — Fri, 01 Nov 2024 16:57:56 GMT

I don't know what you mean by 2.3.6.7 and 2.3.6.9. Fixes to the notification center should be in 2.3.7.7 or at the very least 2.3.7.9. Neither of these versions are out yet which is why you don't see them in the release notes.

Re: Notification Center in Cisco Catalyst Center 2.3.7.6

JEFFREY SESSLER — Fri, 01 Nov 2024 17:07:34 GMT

@maflesch Meaning that when I run across a bug that I may be hitting, it's often the case that 2.3.7.9 is the identified "first fixed." Do you have an ETA on 2.3.7.7 or 2.3.7.9? Has any thought been given to adopting more rapid/targeted releases (package level) in the same fashion that we have SMUs and APSP on the switch/wireless side? The SMU's and APSP's are great in that they target specific defects without the need to wait for the normal point releases and/or TAC special builds.