on - edited on by
jaredkalmus
- Do you need to clean up duplicate assets in your Cisco Vulnerability Management (formerly Kenna Security) Platform environment?
- How Primary Locators Work
- Asset Locators vs. Display Locators
- Determining if Your Locator Order Needs Updating
- How to Customize You Locator Order
- Using External ID and Prefixes as Your Primary Locator
- Using MAC Address as Your Primary Locator
- Two Common Asset Locator Scenarios and Fixes
Do you need to clean up duplicate assets in your Cisco Vulnerability Management (formerly Kenna Security) Platform environment?
This is a common request we hear from our customers and this article will explain how de-duplication of assets works, as well as some best practices for cleaning up and preventing duplicates. Understanding primary locators and how they work is at the heart of this issue.
As a Cisco Vulnerability Management customer, you may be bringing in multiple scans from multiple scanners and those scans may contain some of the same assets. When duplicates occur, it can lead to inaccurate results. When processing these scans, Cisco Vulnerability Management goes down a list of attributes to determine if an asset already exists in your environment. These attributes are what we call primary locators. Below is the default order of primary locators we use.
- Container identifier
- Image identifier
- EC2 identifier
- MAC address
- NetBIOS
- external IP address
- hostname
- URL
- file name
- fully qualified domain name (FQDN)
- internal IP address (RFC 1918)
- scanner-specific asset ID (eg Qualys host ID, Nexpose device-id)
- database
- application
How Primary Locators Work
Locator ordering (asset matching) is a step-system, based on “true”, “false” or “nil”.
TRUE: In the default locator ordering above, the first thing Cisco Vulnerability Management will do is check the Container identifier field on the incoming asset to see if it matches a Container identifier that you already have in Cisco Vulnerability Management (“true”). If the incoming asset matches an asset that is already there, Cisco Vulnerability Management will update any of the other locator fields that have new, non-nil values since the last connector run so that your data is current.
FALSE: If there is a Container identifier field on the incoming asset and it does not match an asset already in Cisco Vulnerability Management (“false”), we will create a new asset.
NIL: If there is no Container identifier on the incoming asset (“nil”), we will move to the next locator in the priority list and continue down the list until we find an incoming locator that has data on which to match (“true”) or not match (“false”).
Let's walk through an example:
Asset Locators vs. Display Locators
You may have noticed a column in Explore view, labeled "Locators", but we wanted to mention that it's not necessarily related to the primary locator that was used to identify that asset.
The display in that locator column is always static and, in the order listed below. You will not always see every attribute; it will only show locators that the scanner provided us.
DISPLAY_LOCATORS =
:file_locator,
:url_locator,
:hostname_locator,
:ip_address_locator,
:netbios_locator,
:ec2_locator,
:locator (primary)
A little tip: Using the dropdown arrow next to any asset will display the expanded view including all locator information we have on that specific asset from the scanner. It's a good way to see what exactly Cisco Vulnerability Management received and what information we need to de-duplicate on the asset.
Determining if Your Locator Order Needs Updating
To understand how primary locators play a part in your environment, you can use the filter on the right-hand panel. Under Asset Filters, there’s a dropdown for “Primary Locators”.
This list will tell you how many assets were de-duplicated by locator type. Now, we know that every organization’s environment looks different, which is why we allow you to customize the order of locators.
In thinking about how to customize your primary locator list, the most static attributes are the ones that should be at the very top. For example, if you have dynamic IP addresses, it wouldn’t be a good idea to have it at the top because you’d see a duplicate asset every time the IP changes.
How to Customize You Locator Order
Cisco's Customer Success team can help you assess the optimal custom locator order for your organization by analyzing which locators are most duplicated. Locator ordering can be set at two different levels:
- Connector level (often each connector needs different locator ordering)
- Client level/globally
Using External ID and Prefixes as Your Primary Locator
Many scanners provide an External ID on the asset which is unique and therefore often the best option to use as primary locator. This can be very helpful for de-duping assets. However, if you use more than one connector of the same type, different assets may be assigned the same External Id. Therefor, in order to prevent Cisco Vulnerability Management from incorrectly de-duping those assets, a prefix can be used.
The prefix is something that needs to be set on the back end by Support. You can name the prefix anything you like, but a good rule of thumb is to name it the same name as your Connector Name, or in the case of de-duping, a common name that makes sense.
Another use case for using external ID plus a prefix is when customers want to clean up and de-dupe assets that have been brought in by two or more of the same scanner type. In this scenario, the assets would have been located by something other than External ID which is causing duplicates. By adding the SAME prefix and using External ID to locate the assets, we will force a de-duplication of assets.
Using MAC Address as Your Primary Locator
A MAC address, or media access control address, is a unique identifier which can be used to track an asset despite changing IP addresses and host names. MAC addresses are stored as 12-digit hexadecimal values, however there is no defined industry standard as to how these values should be formatted. Some tools separate the octets of the 12 digits with colons, others use dashes, and we've even seen dots and commas used as delimiters across the various tools that can integrate with Cisco Vulnerability Management.
To help address this issue, Cisco Vulnerability Management developed a feature to normalize MAC addresses as they're imported. For a MAC address to be accepted by Cisco Vulnerability Management, the value must have 12 characters. Once a 12-character value has been provided, we will ensure that the octets are separated by colons, and that all characters are capitalized. This will avoid any issues with one scanner reporting MAC addresses in a different format than another scanner, leading to duplicate assets.
Two Common Asset Locator Scenarios and Fixes
Scenario 1: A Cisco Vulnerability Management user is spinning up assets in the cloud that are temporary. IP addresses are being recycled so they manually inactivate an asset when it is decommissioned. However, when that IP address is recycled, the asset does not come back.
Solution: Manually deactivating assets sets a permanent flag which will prevent the asset, as determined by the locator order, from being re-activated. The real problem is that assets are being located based on IP addresses. Use a credentialed scan to bring in more specific information for that asset. This way assets can recycle the IP but be identified by another locator field as long as that locator field is higher on the list than the IP address. For example, using “hostname” and moving it up the list or using “MAC address.”
Scenario 2: A Cisco Vulnerability Management user has transient VMWare machines or laptops in a DHCP environment that experience duplicate assets because of the reuse of IP addresses and newly assigned MAC addresses.
Solution: The current order of preference would cause a duplicate at “4. MAC address” as the MAC address is likely changed when the VM is re-created. Our suggestion would be to increase the priority for “Hostname” and decrease the priority of “MAC address” to ensure a new asset is not listed.
For more information on primary locators check out this help article.