Talos Connectivity Problem

[Ditter]

Hi to all ,

i am getting many messages as the following:

Severity: critical
Module: Talos Communication
Description: 3 modules failed:

• * URLDB- Failed to retrieve beaker inventory
• * LSP- Failed to retrieve beaker inventory

My subscription is active (it expires in 2026).  

Any ideas about why is this happening?   Is it a problem that has to do with Talos?

Please note that this is the first time i get this message. 

The only change i did some days ago was to change the "Cached URLs Expire" which was set to never and i changed it to "week" but i do not think that my issue has something to do with it.

Any ideas,

Thanks, 

Ditter.

[ahollifield]

Several others in the community have reported this as well.  It appears to be a Talos problem at this time.

[Marvin Rhoads]

It's a problem on the TALOS side. Cisco have been notified and are looking into it.

Snort rule and URL database updates may be affected. It should not affect any traffic forwarding.

[HQuest]

root@fmc:/var/sf/beaker3# grep client_cert /etc/sf/beaker3/beaker3.cfg.template
client_cert = /var/sf/beaker3/securefirewall-dev-prod-01_prod.pem

root@fmc:/var/sf/beaker3# openssl x509 -text -in securefirewall-dev-prod-01_prod.pem
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 46240369 (0x2c19271)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, ST = California, L = San Jose, O = Cisco Systems Inc., OU = Security, CN = Keymaster CA 2
Validity
Not Before: Jan 30 22:32:39 2024 GMT
Not After : Mar 30 22:32:39 2025 GMT
Subject: CN = SFW76EVAL-prod-01, C = US, ST = California, L = San Jose, O = Cisco, OU = Security

Not really on Talos end but a screw up from Cisco. A locally deployed certificate expired yesterday and hasn't been renewed everywhere. Whoops.

[Marvin Rhoads]

My contact at Cisco advised:

"Yes. We are going to push an update to fix it. No user interaction. Need one more day"

[BACANEL]

Following this case as I have the same problem.

[erdyer]

This is a known issue and should be fixed soon.  In the meantime you can contact TAC for a temporary workaround.

[BACANEL]

Thank you, I have checked this morning and Cisco has not fixed it yet. I will reached out to TAC.

[Marvin Rhoads]

If the error is bothering you, you can fix it manually with the following commands from your FMC cli expert mode, root user (sudo su -):

expert
sudo su -

<enter password when prompted>

pmtool restartbyid talosAgent
pmtool restartbyid beaker3   
pmtool restartbyid fireamp
pmtool restartbyid CloudAgent

[HQuest]

Thanks Marvin, that is helpful.

I'm wondering, though, if this is sustainable and/or long term. The expired beaker3 certificate has been refreshed but with a short 4 days/96-hours lifetime. Is this workaround truly to give Cisco just enough time to push a long term fix?

root@fmc:~# openssl x509 -text -in /var/sf/beaker3/securefirewall-dev-prod-01_prod.pem
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 81794058 (0x4e0140a)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, ST = California, L = San Jose, O = Cisco Systems Inc., OU = Security, CN = Keymaster CA 2
Validity
Not Before: Apr 2 14:49:33 2025 GMT
Not After : Apr 6 14:49:33 2025 GMT
Subject: CN = a5898858-a7fd-11ee-94a3-3c06584886f3, O = Firepower Organization

[Marvin Rhoads]

I've been told those are short-term-by-design client side certificates. The fix will make it so that they are auto-renewed based on instructions in the payload received from Cisco (as had been working previously).

Update: BugID is published here https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwo63951

[BACANEL · Accepted solution]

Correct - when I reached out to TAC this is what I was told

CSCwo63951 - FMC Client side certificate used to communicate to Talos did not auto-renew correctlyThe Beaker3 process on the FMC is supposed to auto-update the client side certificate it is using to authenticate to Talos for downloads. This process is not auto-updating the certificate without being reloaded. You can workaround this issue by performing a manual reload of the
Beaker3(talos_agent) process with the following commands:
> expert$ sudo su -
#pmtool restartbyid talosAgent
# pmtool restartbyid beaker3

Please note this may take 10-15 minutes to resolve the health alarms that you receive. Additionally, please note that this workaround will request a new certificate that will expires in 5 days. You will need to re-perform this workaround every
5 days. Cisco will be releasing VDB 406 shortly that will renew this certificate for 1 year. Once this VDB has been released and installed on your FMC you will no longer need to perform the workaround every 5 days

[Replace7073]

Is there an ETA when VDB 406 will be released.

[Marvin Rhoads]

It was released on Monday 7 April 2025.

[Jon Are Endrerud]

Getting "Cisco Support Diagnostics Configuration failure" on the FTD's connected to FMC's after restarting, any tips ?