cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5470
Views
10
Helpful
4
Replies

What is "Source ID" in Netflow V9 Packet Header

Deepak Ladha
Level 1
Level 1
Hi,
 
My question is regarding the "Source ID" field that appears in Netflow V.9 packet header. Following Cisco link (http://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.pdf) gives Source ID definition as -
 
"The Source ID field is a 32-bit value that is used to guarantee uniqueness for all flows exported from a particular device. (The Source ID field is the equivalent of the engine type and engine ID fields found in the NetFlow Version 5 and Version 8 headers). The format of this field is vendor specific. In the Cisco implementation, the first two bytes are reserved for future expansion, and will always be zero. Byte 3 provides uniqueness with respect to the routing engine on the exporting device. Byte 4 provides uniqueness with respect to the particular line card or Versatile Interface Processor on the exporting device."
 
I am using "Source ID" (combined with template id) to uniquely identify options templates exported by different routers. At our new lab setup where we have more than one routers configured to export Netflow, I observed that all the routers were exporting "Source ID" value as "0"(zero). It failed my assumption that I had formed based on definition from above Cisco doc.
 
 
I assumed -
 
SourceID    Template Id  Unique Key
------------     ---------------   --------------
source1       256              source1-256
source1       257              source1-257
source2       256              source2-256
source3       258              source3-258
 
 
But, I observed
 
SourceID    Template Id  Unique Key
------------     ---------------   --------------
0                  256              0-256
0                  257              0-257
0                  256              0-256
0                  258              0-258
 
Thus, same template id(256) from different routers(source1, source3) eventually form same unique key and breaks my code.
 
I would like to know if my interpretation that Source ID can be used to uniquely identify templates in this manner is correct or not ? 
Is "Source ID" user configurable attribute ? How does it comply to the definition given in above Cisco doc ?
 
Thanks,
Deepak
 
1 Accepted Solution

Accepted Solutions

Deepak,

Consider these quotations from the same RFC 3954:

 

Section 2: Terminology:

   Observation Point
   An Observation Point is a location in the network where IP packets
   can be observed; for example, one or a set of interfaces on a network
   device like a router.  Every Observation Point is associated with an
   Observation Domain.

   Observation Domain
   The set of Observation Points that is the largest aggregatable set of
   flow information at the network device with NetFlow services enabled
   is termed an Observation Domain.  For example, a router line card
   composed of several interfaces with each interface being an
   Observation Point.

 

Section 7: Template Management:

   A NetFlow Collector that receives Export Packets from several
   Observation Domains from the same Exporter MUST be aware that the
   uniqueness of the Template ID is not guaranteed across Observation
   Domains.

 

Section 9: The Collector Side:

   At any given time the Collector SHOULD maintain the following for all
   the current Template Records and Options Template Records: Exporter,
   Observation Domain, Template ID, Template Definition, Last Received.

   Note that the Observation Domain is identified by the Source ID field
   from the Export Packet.

 

So in other words, the Source ID is an identifier of the Observation Domain (and in fact, the IPFIX RFC calls this header field directly as Observation Domain ID). Template IDs are unique per Exporter and per Observation Domain, and if a single Exporter uses multiple templates in its different Observation Domains, the IDs of these templates could overlap even in a single Exporter. Observation Domain IDs (that is, Source IDs) identify only the internal structure of a single Exporter, and no provisions are done to preserve their uniqueness across multiple Exporters - for this, the source IP shall be used.

With respect to whether there can be multiple NetFlow instances on a single router, I am getting a feeling that with decentralized, distributed platforms, multiple linecards in a single router could run their own NetFlow analysis for data that pass through them, so each one provides a separate NetFlow collection. Thus, each linecard or each feature card doing its own NetFlow analysis should be assigned its own unique Observation Domain ID.

If it is not user configurable then system should automatically form the value based on router engine and line card. But what I have observed, at more than one routers, is that this value is always 0(zero).

I believe this is strongly dependent to the hardware construction of the router. As a remotely-related example, old 2600 series routers had two WIC slots. If you inserted two WIC-2T modules into these slots, you'd expect that they would be numbered Serial0/0, Serial0/1, Serial1/0, Serial1/1. Very surprisingly, however, these routers considered both slots to be internally connected to a single bus, and the interfaces were named Serial0/0, Serial0/1, Serial0/2 and Serial0/3 - as if they all were installed in a single slot '0'. Something similar may happen to the Observation Domains and their IDs. You would believe that each single linecard constituted a separate Observation Domain. However, the reality may be different, and the whole router can act as a single Observation Domain to the outside world. It's just the way it is constructed - and programmed.

It is not clear why Cisco doc says that one should use both "Source ID" and "Source IP Address" to properly distinguish between flows.

I think it's a poor wording in the RFC. I think what they want to say is that if you use the duplet <Source IP, Source ID> to distinguish between flows, then you're fine both for multiple flows from the same Exporter, and for multiple flows from different Exporters.

Moreover, isn't "Source IP Address" good enough to distinguish between flows from different sources ?

If an Exporter could truly be partitioned into multiple Observation Domains then the source IP would not be sufficient. I am just making up examples with no real-life backup here, but think of, say, a multi-chassis router with each chassis being one Observation Domain, or each linecard of a distributed switch being a standalone Observation Domain, or one router virtualized to several different contexts and virtual routers, each of them being a unique Observation Domain, reporting about the flows using the same source IP... I think you get the point.

I would put it this way... The existence of Source ID in NetFlow v9 (and Observation Domain ID in IPFIX) allows these protocols to nicely cope with situations in which a single physical device can be partitioned into several Observation Domains and perform independent reporting on them using a single source IP. However, the fact that these protocols have this ability does not mean that each and every device, even a Cisco router/switch, must necessarily make use of it.

Best regards,
Peter

View solution in original post

4 Replies 4

Peter Paluch
Cisco Employee
Cisco Employee

Hi Deepak,

I am not personally well-versed with NetFlow but according to your observations and the Cisco doc you referenced, it appears to me that the Source ID only guarantees uniqueness between flows exported from a single device. It does not try to guarantee uniqueness between flows exported from multiple devices. It appears that the Source ID only tries to capture the subsystem in a Cisco router or a switch that performed the flow observation. Here is what RFC 3954 has to say about this field:

   Source ID
         A 32-bit value that identifies the Exporter Observation Domain.
         NetFlow Collectors SHOULD use the combination of the source IP
         address and the Source ID field to separate different export
         streams originating from the same Exporter.

Regarding the setting of the Source ID, it does not seem to be configurable. There are some threads out there on internet asking for the same thing, and arriving at the same conclusion, e.g.:

http://www.gossamer-threads.com/lists/cisco/nsp/167725

It seems that you should use both the source IP address of the flow exports and source ID to properly distinguish between flows exported from individual collecting devices.

My 2 cents...

Best regards,
Peter

Thanks Peter for your opinion and sharing the link on "Source ID" discussion.

I am still not sure how the value is set in this field. If it is not user configurable then system should automatically form the value based on router engine and line card. But what I have observed, at more than one routers, is that this value is always 0(zero).

It is not clear why Cisco doc says that one should use both "Source ID" and "Source IP Address" to properly distinguish between flows. If the "Source ID" is always zero then clubbing or not clubbing it with "Source IP Address" shall not make any difference. Moreover, isn't "Source IP Address" good enough to distinguish between flows from different sources ? Does it mean that one device (router) can have more than one instances of Netflow ? 

Any thoughts ?

Thanks,

Deepak

Deepak,

Consider these quotations from the same RFC 3954:

 

Section 2: Terminology:

   Observation Point
   An Observation Point is a location in the network where IP packets
   can be observed; for example, one or a set of interfaces on a network
   device like a router.  Every Observation Point is associated with an
   Observation Domain.

   Observation Domain
   The set of Observation Points that is the largest aggregatable set of
   flow information at the network device with NetFlow services enabled
   is termed an Observation Domain.  For example, a router line card
   composed of several interfaces with each interface being an
   Observation Point.

 

Section 7: Template Management:

   A NetFlow Collector that receives Export Packets from several
   Observation Domains from the same Exporter MUST be aware that the
   uniqueness of the Template ID is not guaranteed across Observation
   Domains.

 

Section 9: The Collector Side:

   At any given time the Collector SHOULD maintain the following for all
   the current Template Records and Options Template Records: Exporter,
   Observation Domain, Template ID, Template Definition, Last Received.

   Note that the Observation Domain is identified by the Source ID field
   from the Export Packet.

 

So in other words, the Source ID is an identifier of the Observation Domain (and in fact, the IPFIX RFC calls this header field directly as Observation Domain ID). Template IDs are unique per Exporter and per Observation Domain, and if a single Exporter uses multiple templates in its different Observation Domains, the IDs of these templates could overlap even in a single Exporter. Observation Domain IDs (that is, Source IDs) identify only the internal structure of a single Exporter, and no provisions are done to preserve their uniqueness across multiple Exporters - for this, the source IP shall be used.

With respect to whether there can be multiple NetFlow instances on a single router, I am getting a feeling that with decentralized, distributed platforms, multiple linecards in a single router could run their own NetFlow analysis for data that pass through them, so each one provides a separate NetFlow collection. Thus, each linecard or each feature card doing its own NetFlow analysis should be assigned its own unique Observation Domain ID.

If it is not user configurable then system should automatically form the value based on router engine and line card. But what I have observed, at more than one routers, is that this value is always 0(zero).

I believe this is strongly dependent to the hardware construction of the router. As a remotely-related example, old 2600 series routers had two WIC slots. If you inserted two WIC-2T modules into these slots, you'd expect that they would be numbered Serial0/0, Serial0/1, Serial1/0, Serial1/1. Very surprisingly, however, these routers considered both slots to be internally connected to a single bus, and the interfaces were named Serial0/0, Serial0/1, Serial0/2 and Serial0/3 - as if they all were installed in a single slot '0'. Something similar may happen to the Observation Domains and their IDs. You would believe that each single linecard constituted a separate Observation Domain. However, the reality may be different, and the whole router can act as a single Observation Domain to the outside world. It's just the way it is constructed - and programmed.

It is not clear why Cisco doc says that one should use both "Source ID" and "Source IP Address" to properly distinguish between flows.

I think it's a poor wording in the RFC. I think what they want to say is that if you use the duplet <Source IP, Source ID> to distinguish between flows, then you're fine both for multiple flows from the same Exporter, and for multiple flows from different Exporters.

Moreover, isn't "Source IP Address" good enough to distinguish between flows from different sources ?

If an Exporter could truly be partitioned into multiple Observation Domains then the source IP would not be sufficient. I am just making up examples with no real-life backup here, but think of, say, a multi-chassis router with each chassis being one Observation Domain, or each linecard of a distributed switch being a standalone Observation Domain, or one router virtualized to several different contexts and virtual routers, each of them being a unique Observation Domain, reporting about the flows using the same source IP... I think you get the point.

I would put it this way... The existence of Source ID in NetFlow v9 (and Observation Domain ID in IPFIX) allows these protocols to nicely cope with situations in which a single physical device can be partitioned into several Observation Domains and perform independent reporting on them using a single source IP. However, the fact that these protocols have this ability does not mean that each and every device, even a Cisco router/switch, must necessarily make use of it.

Best regards,
Peter

Thank you Peter for such detailed explanation.

I hope Cisco Netflow docs will cover such details in future revisions to help end user understand the concepts precisely.

Regards,

Deepak