cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1745
Views
5
Helpful
2
Replies

Security Cloud Control Notifications in Splunk (Or other SIEMs)

AHack210
Cisco Employee
Cisco Employee

Introduction

Cisco Security Cloud Control Firewall Manager is a part of the Cisco Security Cloud Control platform where we can manage ASAs, FTDs, and many other Cisco security products.

SCC Firewall manager does have some "nice to have" notifications about events that might be critical to your operation. Some examples include:

  • AnyConnect Certificate Expiration
  • ASA Config Deployment
  • cdFMC Deploys
  • A new scheduled report is available from SAL
  • A scheduled ASA upgrade has started, failed, succeeded, etc

It would be advantageous for an enterprise or MSP to get notifications from all of their tenants and devices pushed to a centralized platform like Splunk or some other SIEM capable of ingesting HTTP based events (webhooks). Once ingested, the SIEM can do what SIEMs do and enrich, correlate, and even fire off webhooks or scripts of their own to trigger some process in response to the notification.

In this article, I will cover how to get SCC Firewall Manager notifications from any number of SCC organizations/tenants into Splunk. The reader can leverage this same knowledge to apply to any number of other SIEMs. While we will be using Splunk Enterprise as our example case, this same configuration will also work for Splunk Cloud consumers.

Note: I am NOT a Splunk expert and your local Splunk team will be the right people to configure your Splunk HEC, index, input types, reports, etc to best practices. I am providing the Splunk configuration below to assist those lab hackers like myself, who might be taking advantage of the free 6 month Splunk Enterprise license or the free Splunk Cloud offering for Cisco Security customers.

Requirements

  • Splunk Enterprise or Splunk Cloud
  • Splunk HTTP Events Collector (HEC) configured
  • Cisco SCC Firewall Administrator Access

 

Splunk Configuration

Configure the Splunk HEC

For my lab instance of Splunk Enterprise, the inputs.conf configuration looks like this (add the “local” directory if it does not exist):

/opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf

 

[http]
disabled = 0
useDeploymentServer = 0
enableSSL = 1
allowQueryStringAuth = true

The important part above is that we allow the Splunk HEC token to be passed as a parameter in the HTTP request string, rather than only as an HTTP header. The "allowQueryStringAuth = true" is the parameter that enables query string auth.

Now, restart the Splunk server so that your HEC settings are applied.

sudo /opt/splunk/bin/splunk restart

 

Create a Splunk Index

Screenshot 2025-07-15 at 5.11.39 PM.pngNow, we will create a Splunk index to hold the SCC Firewall Manager notifications.

Settings --> Indexes --> New Index

Accept the default values and name the new index scc_notifications.

 

 

 

Create a Splunk HEC Token

Next, let’s create a Splunk HEC token. This token is what will connect the SCC Firewall Manager notification messages to the new index that we just created.

Screenshot 2025-07-15 at 5.15.00 PM.png

In the Splunk dashboard, click Settings --> Data Inputs --> HTTP Event Collector --> New Token

Give it a meaningful name like scc_notify and click next.

In the “Select Allowed Indexes” dialog, select the index we created: scc_notifications, and then complete the wizard using the default values.

Copy the HEC token for the next steps.

 

SCC Notification Configuration

*Note: At the time of the writing of this article, the notification URL settings in the Cisco Security Cloud Control notifications panel at https://security.cisco.com does not allow the use of ports other than 80/443. However, a user can log into the standalone Firewall Manager application at https://us.manage.security.cisco.com (See notes below for other regions) which allows us to configure a custom URL/Port other than 443. The notification settings in the standalone portal are also applied to https://security.cisco.com. This caveat should be resolved soon so this is just a workaround until the Security Cloud Control platform can accommodate any TCP port and enable a test button.

7/23/2025 - The Security Cloud Control Platform at https://security.cisco.com now supports any TCP port for these notifications. This fix was recently published so there is a chance a user could have the old code cached. If a user has any issues getting the https://url.com:port syntax working, they should use an incognito web session or clear their cache and cookies.

 

Log into Security Cloud Control Firewall Manager, select the organization/tenant from which you wish to emit the notifications, and navigate to Administration --> Notification Settings.

Under “Service Integrations”, click the blue (+) and give your service integration a name. I am calling mine “Splunk-65150-Webhook” because I have a port forward on my firewall that is forwarding port TCP 65150 to my Splunk Server's HEC on TCP port 8088, which is the default port that the Splunk HEC service listens on.

In the “Service Type” select “Custom

In the URL, put the FQDN of your DNS record that points to the Splunk HEC service. In my case, the DNS record points to the outside IP address of my firewall, as I am using a port-forward NAT on the firewall’s public IP address.

The URL looks like this. Your mileage may vary depending on the port that you use, but you will get the general idea.

https://<my.fqdn.com>:<public_port>/services/collector/raw?token=<xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx>

Where <xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx> is replaced by the Splunk HEC token that we created above.

Screenshot 2025-07-15 at 5.48.38 PM.png

We are almost done! Let’s test this.

 

Testing

When you press the “test” button, SCC Firewall Manager will fire off a test webhook message to your HEC. You can verify that you received the message by logging into the Splunk Search & Reporting app and then showing messages from the index that we created by using the search: 

index="scc_notifications"

 

Now press the test button and refresh the Splunk search page. The following occurs:

1. The test sends json payload to your Splunk HEC similar to this:

{
  "uid": "81661e8b-a727-4f07-ab89-a3b1e38ea887",
  "timestamp": 1752609233572,
  "type": "ANNOUNCEMENT",
  "tenantUid": "fdc4e668-d568-43d3-b912-20e4f5a68e21",
  "details": {
    "message": "Hello, World!",
    "docsUrl": ""
  }
}

2. In SCC Firewall Manager, under the URL dialog, you will see the message “notifications.settings.webhooks.testSuccess” indicating that the TCP message was successfully delivered to an endpoint at the URL and port that you defined in the dialog.

Screenshot 2025-07-15 at 5.51.49 PM.png

 3. In Splunk, refresh your search and you should see this message in your index:

Screenshot 2025-07-15 at 2.59.05 PM.png

That’s all there is to this. From now on, any notification that you have selected in the notifications panel will be sent to your HEC and on to your Splunk index. From there, you can configure Splunk to do any number of things with this data, like firing off it’s own webhooks, dashboards, and reporting. If you are having issues getting the test message to transmit successfully, see my section below on troubleshooting.

Create a Splunk Report

If you have successfully received a test message and used the same index name that I did, the following search should work for you and will allow you to save the search as a report. Again, I am just a Splunk hacker, a real Splunk person may roll their eyes at my search, but this works for me. If you have suggestions for a more optimal search, by all means, feel free to leave an alternative search query in the comments!

index="scc_notifications" source="http:scc_notify" 
| spath output=timestamp path=timestamp | eval formatted_date=strftime(timestamp/1000, "%Y-%m-%d %H:%M:%S")
| spath details.deviceReferences{}.name output=deviceName
| spath details.deviceReferences{}.deviceType output=deviceType
| spath details.stateMachineInstance.stateMachineInstanceCondition output=stateMachineInstanceCondition
| spath details.stateMachineInstance.stateMachineType output=stateMachineType
| spath details.fmceEvent output=fmceEvent
| spath details.fmceEventState output=fmceEventState
| spath type output=type
| spath details.discoveryEvent output=discoveryEvent
| spath output=cert_expires path=details.certificateExpirationDate | eval certificateExpirationDate=strftime(cert_expires/1000, "%F %T")
| spath details.certificateType output=certificateType
| spath tenantName output=tenantName
| spath details.salQueryName output=salQueryName
| spath details.salStatus output=salStatus
| spath details.swcFileName output=fileName
| eval status=coalesce(stateMachineInstanceCondition, fmceEventState, salStatus)
| eval operation=coalesce(stateMachineType, fmceEvent)
| table formatted_date tenantName deviceType deviceName type status discoveryEvent certificateExpirationDate certificateType operation salQueryName fileName

Once you are satisfied that your query is producing expected results, click Save As --> Report

And now you have a report that will return the latest notifications sent by all of your configured organizations/tenants at your fingertips.

Screenshot 2025-07-15 at 3.19.51 PM.png

 

Troubleshooting

System Diagram

SCC-Webhook-Notifications-Splunk.jpg

Splunk

Make sure your HEC is enabled and listening on TCP 8088. From the splunk server's CLI:

netstat -alnt | grep 8088

Screenshot 2025-07-15 at 3.27.07 PM.png

Make sure that HEC is working locally by sending a test message using curl on your splunk server's CLI (where xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx is your HEC token).

curl -vvv -k --location 'https://127.0.0.1:8088/services/collector/raw?token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' \
--header 'Content-Type: application/json' \
--data '{"event": "msg=hello world, uid=1234abcdef", "foo":"bar"}'

A "200 OK" response with the json payload: {"text":"Success","code":0} indicates that the service is working correctly and that your token is being accepted as a query parameter.

 

Networking

On your ASA or FTD, use packet-tracer to assure that your port forward, NAT, and access-rules are allowing the webhook TCP session to reach your Splunk HEC server.
Reminder: ASA and FTD will use the POST-NAT IP address and port, so your access-policy should allow TCP to the server’s internal IP address and port 8088.

packet-tracer input outside tcp <some_internet_ip_address> 1099 <your_outside_ip> 65150 detail

NAT (port forward) is working as expected

Phase: 2
Type: UN-NAT
Subtype: static
Result: ALLOW
Elapsed time: 15360 ns
Config:
object network splunk
 nat (dmz,outside) static interface service tcp https 65150
Additional Information:
NAT divert to egress interface dmz(vrfid:0)
Untranslate <youroutsideip>/65150 to 10.0.0.10/8088

Access-Policy Allowed the traffic to 10.0.0.10/8088 on the DMZ Segment and the server responded to ARPs

Phase: 18
Type: ADJACENCY-LOOKUP
Subtype: Resolve Nexthop IP address to MAC
Result: ALLOW
Elapsed time: 3584 ns
Config:
Additional Information:
Found adjacency entry for Next-hop 10.0.0.10 on interface  dmz
Adjacency :Active
MAC address 2224.2222.d3c8 hits 3980048 reference 3880

Result:
input-interface: outside(vrfid:0)
input-status: up
input-line-status: up
output-interface: dmz(vrfid:0)
output-status: up
output-line-status: up
Action: allow
Time Taken: 271641 ns

And if all else fails, you can use packet capture or tcpdump to see if the traffic is reaching the Splunk HEC on port 8088 and getting return traffic.

 

 

Conclusion

In my next article, I plan to publish how to make Splunk react to "SAL Report completed" notifications in realtime,  download the report, and ingest the report data into Splunk automatically. Thanks for reading this far and for your support. I hope you found this walk-through helpful and, if you did, do me a favor and give me a "helpful" vote or a thumbs up.

2 Replies 2

Great observation. Yes, absolutely. Getting the event to Splunk or other systems is just the start. Meaningful notifications are critical vs "just noise". Stealthwatch was built around this idea. Of the millions of observations, give me a handful that I care about. Agentic AI will hopefully extend this even more by addressing the things it can on it's own and only bringing something to the attention of an engineer when it absolutely must. 

Marvin Rhoads
Hall of Fame
Hall of Fame

Nice article @AHack210 - thanks for sharing.

Review Cisco Networking for a $25 gift card