How to Turn a Data Overload into Actionable Intelligence

[AdvocateRick]

According to a 2020 IBM study, enterprises are using over 45 different security tools on average. That number feels like a bit of an understatement based on my own experience working with enterprise security organizations! With so many different sources of data floating around, turning the deluge of information into actionable intelligence can feel nearly impossible.

With the right approach, a data overload can quickly turn into a data gold mine. The rise of data lakes and data warehouses have given organizations the tools they need to build a one-stop, centralized location to view and interact with their data from all of their different security tools. Having all this data stored in a centralized location empowers organizations to build visual dashboards and advanced metrics that reach across differing data sets for painless custom reporting without the hassle of provisioning users with permissions in each tool. 

As Kenna (now Cisco Vulnerability Management) already consolidates vulnerability data from all your different scanners as a first step to providing actionable risk-based vulnerability insights, Kenna serves as a great foundation to building out a data lake or data warehouse for your security team and other internal stakeholders. 

Many Kenna customers are making use of the Kenna API to augment the out-of-the-box reporting provided in the Kenna GUI. Here are a few impressive use cases we've seen recently:

• A Fortune 50 technology firm is exporting all of their vulnerability data into a centralized database to allow business users to see a quick overview of their overall risk levels without needing access to Kenna.
• A large financial services customer is pulling all of their Kenna data into a data visualization tool to build custom dashboards for their executive team to present to the company's board of directors.
• A European bank is exporting their Kenna data into a data lake which contains all of the bank's security findings and asset metadata across hundreds of different tools and technologies.

The above use cases highlight the value of Kenna’s enriched vulnerability data, not just in the Kenna console itself, but also when combined and aggregated with other data sources.

These benefits and use cases just begin to scratch the surface of what’s possible when you extract data from Kenna. Once you have the data in hand, the sky's the limit of what you can build and accomplish.

Before you begin exporting your Kenna data into any external systems, it’s important to determine your scope by asking the following: 

• Are you only interested in reporting on open vulnerabilities on active assets? 
• Do you have a business requirement to report on all assets, including those that haven’t been scanned in years? 
• Are you using closed vulnerabilities to track remediation velocity? 

Once you have the scope of your data set defined then you have all you need to begin using the Kenna API to export all of your required data.

The rest of this blog post will assume you have a base level understanding of how to interact with the Kenna API. If you’re unfamiliar with the Kenna API, or even APIs in general, we recommend listening to our on-demand webinar “How to Start Using APIs in your Vulnerability Management”. 

Data exports generated with the Kenna API are processed asynchronously, meaning that you must send a request to generate the export, then perform a second request to download the data specified in the export request.

Kenna provides three different export models which mirror the three tabs in the Explore view in the Kenna GUI: Assets, Vulnerabilities, and Fixes. You’ll need to export all three models to replicate all of your Kenna data outside of the GUI. Assets are linked to their vulnerabilities by a field labeled Asset ID, while Vulnerabilities are linked to their Fixes by Fix ID.

By default, all three models are filtered to only include open vulnerabilities and active assets. Thankfully, Kenna provides the flexibility to target other statuses by passing a parameter into the body of the API request to generate an export. Let’s take a look at a few examples.

curl -L -X POST 'https://api.kennasecurity.com/data_exports' \ \
-H 'X-Risk-Token: <KENNA_API_KEY>’ \
-H 'Content-Type: application/json' \ \
--data-raw '{
  "export_settings" : {  
    "format": "jsonl",
    "model": "vulnerability"
  }
}’

At first glance you’ll likely notice that this is a data export POST request for vulnerabilities in the JSON lines format. However, you may not realize that since this request does not apply any overriding parameters, it will use the default filter of only including open vulnerabilities on active assets.

Now let’s say you’ve decided that you want to export ALL of your vulnerabilities, regardless of status or asset activity. The following export request will allow you to do just that.

curl -L -X POST 'https://api.kennasecurity.com/data_exports' \ \
-H 'X-Risk-Token: <KENNA_API_KEY>’ \
-H 'Content-Type: application/json' \ \
--data-raw '{
  "asset" : {
    "status" : [
      "active",
      "inactive" 
     ]
  },
  "status" : [
    "open",
    "closed",
    "risk accepted",
    "false positive"
  ],
  "export_settings" : {
    "format": "jsonl",
    "model": "vulnerability"
  }
}'

Due to the potentially large size of the data, all exports are highly compressed into a gzip format. Mac and Linux systems should be able to decompress these files natively, while applications such as 7zip can handle the task for Windows devices.

As large exports can take a while to generate, Kenna (now Cisco) provides an API endpoint to monitor the status of the export.

curl -L -X GET 'https://api.kennasecurity.com/data_exports/status?search_id=12345' \ \
-H 'X-Risk-Token: <KENNA_API_KEY>’

Once we get a response back such as message: "Export ready for download" then we can move on to the final step of downloading the compressed file.

curl -L -X GET 'https://api.kennasecurity.com/data_exports?search_id= 12345' \
-H 'X-Risk-Token: <KENNA_API_KEY>’

Once your initial export downloads are complete, it’s time to migrate to incremental exports to ensure your external data set stays up to date as your Kenna (now Cisco) data continues to change. We recommend running these incremental exports on a scheduled, daily basis for most customers.

By specifying a date and time range for your data export request, you can ensure that you’re only downloading the delta of what has changed since your last export. This keeps your scheduled exports minimal in size, which in turns reduces the time it takes to generate and download the data.

Here’s an example of an export request for assets that have had any changes in the past 24 hours:

curl -L -X POST 'https://api.kennasecurity.com/data_exports' \ \
-H 'X-Risk-Token: <KENNA_API_KEY>’ \
-H 'Content-Type: application/json' \ \
--data-raw '{
  "asset" : {
    "status" : [
      "inactive",
      "active"
    ]
  },
  "records_updated_since" : "now-24h",
  "export_settings" : {
    "format": "jsonl",
    "model": "asset"
  }
}'

In order to help our customers get started with setting up a thorough data exports process, we’ve prepared a Postman collection containing a few API request examples that we think will be helpful.

Be sure to check out our Acquiring Vulnerabilities per Asset blog discussing how to use the export APIs with Python once you’re ready to start building out an automated exports process. We also encourage you to check out the Kenna Security Postman Collection blog post to download our full repository of API sample requests.

We hope this information will speed you on your way to making the most out of your Kenna solution and vulnerability management program.

Jared Kalmus

Technical Team Lead

This blog was originally written for Kenna Security, which has been acquired by Cisco Systems.
Learn more about Cisco Vulnerability Management.