Flow Detail Records Overview

The Flow Detail Records (FDR) feature is a method for gathering and processing per-flow statistics. When FDR is enabled, the PacketShaper will become an emitter, periodically pushing data to a remote system called a collector. The unit will emit records that contain details of all flows that go through the PacketShaper to a collector, such as Packeteer’s ReportCenter 3. These records are called flow detail records.

PacketWise can look at a flow, identify its application or protocol, gather statistics about this flow, include this information in the Packeteer flow detail record, and then send the FDRs to a collector. In the collector’s report generator, you can view reports to summarize and analyze the data.

Packeteer’s FDR solution offers:

• Enhanced troubleshooting and forensic capabilities. ReportCenter 3.x’s reports can aid in troubleshooting network problems and help determine the source of a DoS attack. For example, ReportCenter 3.x’s Flow Detail reports let you see the busiest hosts on your network and drill down to see the applications used and destination addresses contacted. Or, you can view a port-based report to see a list of the busiest ports and then drill down to see the hosts that generated traffic through each port.
  
  
• Integration with accounting/billing programs. Broadband service providers can bill customers by application usage and, if desired, have different billing rates for different types of applications (such as P2P, VoIP, email, and web surfing). Or, enterprises can track each department’s application usage and bill them accordingly.
  
  
• Statistics of latency, loss, and utilization of VoIP flows. You can use these statistics to monitor and analyze the effectiveness of VoIP on your network.
  
  
• Historical Top Talker and Top Listener data. In ReportCenter 3.x, you can specify how long you want to keep the data; for example, you may want to keep hourly data for six weeks and monthly data for two years. You also have an increased number of Top Talkers/Listeners (PacketWise offers a maximum of 12 per traffic class; in ReportCenter 3.x, this value is user specified.)
  
  
• Cisco NetFlow v5 compatibility. PacketWise can emit FDRs to NetFlow-compatible collectors and analyzers, such as Evident Billing Software and Cisco Collector.

You can also specify classes for which flow detail records are emitted by using the setup flowrecords filters command in the CLI.

What Type of Information is in a Flow Detail Record?

Generally, a flow detail record (FDR) contains information about a TCP or non-TCP flow, such as source and destination IP addresses, the size of the flow (in terms of packets and bytes), and when the flow was sent. The specific fields of information vary according to the type of record format. PacketWise offers three different record types: Packeteer-1, Packeteer-2, and NetFlow-5. The NetFlow-5 record type identifies the flow’s Layer 4 protocol (such as TCP, UDP, or ICMP) and IP ToS/Diffserv. The Packeteer-2 format contains all the NetFlow fields as well as Packeteer-specific data, for example: the traffic class into which the flow was classified, type of policy, number of retransmitted bytes, Response Time Measurement (RTM) data, packet exchange time, and VoIP statistics for RTCP VoIP streams. The Packeteer-1 format can be emitted, but collectors for this format are not currently available.

For a list and description of all fields contained in in the NetFlow-5 record type, see NetFlow v5 Record Format.

Flow Detail Record Collectors

A flow detail record collector is a software application, such as Packeteer ReportCenter, that accumulates the data from an FDR emitter (PacketShaper). Most collectors do much more than gather the data — they also massage and present the information in a meaningful way in reports and graphs.

Use the Packeteer-2 format to send flow detail records to the ReportCenter 3.x collector. Use the NetFlow-5 record type to send flow detail records to a NetFlow v5 collector.

In the example below, ReportCenter has been set up as a collector that uses the Packeteer-2 format, and Cisco Collector and Evident Billing Software have been defined as collectors of the NetFlow-5 format.

How Often are Flow Detail Records Emitted?

PacketWise can emit flow detail records either at the end of a flow (default) or at a set interval (optional):

• End-of-flow FDRs
• Intermediate FDRs

Note: Changes made to FDR behavior impact the records emitted to all FDR collectors. For example, you cannot configure your unit to emit intermediate FDRs only to a NetFlow-5 collector, while sending end-of-flow FDRs to a Packeteer-2 collector, such as ReportCenter (which cannot process intermediate FDRs).

End-of-flow FDRs

For TCP flows, two flow records (one for the start of the flow, one for the end) are sent when the TCP connection is closed. In the unusual case when connections remain open for a long period of time without any activity, PacketWise will eventually reclaim the resources and close the connection; the flow records will be created at that time.

For non-TCP flows, flow records are generally created one hour after PacketWise sees the last packet for the flow. Exceptions are transactional non-TCP flows, such as a DNS lookup over UDP or an ICMP ping. For these types of flows, the flow record is created when the transaction is completed.

Flow detail records are bundled into UDP packets before they are emitted to the collector. Because of this bundling process, there is a short delay from the time flow detail records are created until the UDP packet is emitted. On a busy PacketShaper, this delay is typically less than a second.

UDP Flow Record Packet for NetFlow v5

Note: If flow recording is on and then turned off, any unsent flow records are bundled into a UDP packet and emitted to the collector as recording is turned off.

At the top of each UDP flow record packet is a header. The header provides information about the records in the UDP packet, such as the number of flow records in the packet. For a list and description of all fields contained in the NetFlow header, see UDP header for NetFlow v5 packets.

Intermediate FDRs

When the intermediate FDR feature is enabled, FDRs are emitted at a configurable interval throughout long TCP flows as well as at the end of flows. This allows a suitably-instrumented collector, such as Cisco-based Netflow-5 collectors, to report flow data during long-lived flows. For information on how to enable intermediate FDRs, see Enable Intermediate FDRs.

Note: ReportCenter v3.1 and earlier do not support intermediate FDRs.

Classification of Flow Detail Records

Flow detail record packets are automatically classified into two predefined classes:

• FlowRecords — includes Packeteer-1 and Packeteer-2 FDRs
• NetFlowV5 — includes NetFlow v5 FDRs emitted by the PacketShaper

The appropriate class is automatically created as soon as a collector is enabled. The FDR-related classes are child classes of Outbound/Localhost because FDRs are generated by PacketShaper (the local host). For example, if a Packeteer-2 collector is defined and enabled, an Outbound/Localhost/FlowRecords class is created; this class tracks all FDR traffic emitted to the defined Packeteer-2 collector.

The service names correspond to the class names: FlowRecords and NetFlowV5. If you delete the classes that were automatically created, you can either manually recreate them, specifying FlowRecords or NetFlowV5 for the service, or reset the unit (the classes will be automatically recreated after a reset).

Feature Requirements and Limitations

The Packeteer flow detail record feature has the following requirements:

• PacketWise v7.0.0 or above
• PacketShaper 1200, 1400, 1550, 2500, 3500, 6500, 7500, 8500, 9500, or 10000 models
• 256 MB minimum memory

The FDR feature has several differences from Cisco's NetFlow:

• PacketWise does not support aggregation (for example, aggregate all flows to the same destination IP and report them as one flow) or sampling (collect details on every nth packet).
• For short-lived TCP flows, PacketWise reports the flow data as soon as the connection is closed. NetFlow typically reports it about 15 seconds later.
• For short UDP flows (such as a 30-second Voice over IP conversation), PacketWise closes the flow and reports it after one hour (when PacketWise times out non-transactional UDP flows). NetFlow reports the flow in about 15 seconds.

See also:

FDR Notes

FDR Troubleshooting

Define Flow Detail Record Collectors

Enable Intermediate FDRs