Rapid PCAP Analysis with Analytics

In this post I show you how one can use the built-in security analytics in Immediate Insight to help you rapidly review a large PCAP file for anomalies. Immediate Insight is security analytics for data discovery – purpose-built for incident response, investigation, and triage.

1) Collect the PCAP. Here I am using my Fortinet FortiWifi 90D’s built-in packet capture functionality – you can also use tshark or whatever your favorite packet capture methodology du jour is. Immediate Insight does not use parsers – its natural language system can take in any human-readable data (PCAP, PST files, PDF, Microsoft Office documents and spreadsheets, configuration files, IP reputation files – the list goes on).


You can even stream PCAP in real-time to Immediate Insight:


2) Use Immediate Insight’s Drag and Drop Import functionality to ingest the PCAP quickly and easily – tag the data with a unique term (I used fwf90dpcap). This tag will be used to search and use the built-in analytics in Immediate Insight only on this specific file (you can also, of course, review all of your data together – including this file). Be sure to review the Import Time settings (set it to the current time – or now – or set it to the time stamps found within whatever file you are dragging into the system) to make sure they match the overall purpose of your investigation (the Drag and Drop Import functionality is primarily designed to supplement the real-time data coming into your system – such as syslog, flat-files, NetFlow/IPFIX/sFlow, and much more).


3) Search on your tag (fwf90dpcap in this case).


The default Search Results page will display very much like what you’re used to with a SIEM. However, one of the main goals of Immediate Insight is to encourage exploration and the discovery of the unknowns (with support for both structured and unstructured data) – I will be focusing on some of the analytics components in this post.

Clicking on Explore > Entities will take you to Immediate Insight’s Association Analytics featuring a look at both frequent and infrequent entities (things like URLs and IP addresses), locations (Immediate Insight does automatic geolocation enrichment at time of data ingestion), addresses (IP addresses), networks, sources (where the data came from – i.e. syslog), and names.

Furthermore, clicking on Analytics > Most Common will take you to Immediate Insight’s Event Clusters functionality. This is an incredible tool for going through large datasets or finding the unknown unknowns. It takes the entirety of the dataset that you’re reviewing and finds the commonality in the data – grouping similar data into buckets so a human can rapidly determine what is happening. In the image below – you can see that a very complex event has been boiled down to its essence for the purpose of human analysis.


You can click on the black and white circle to highlight unusual terms – adding further ease to the review. You can also flip the data over and look at the unusual items (versus the common, by default).

This is just the tip of the iceberg regarding the capabilities of Immediate Insight from FireMon. The free, Community Edition is full-featured and supports up to 25 million events – it’s a VMware OVA that you can deploy within 10 minutes in VMware Workstation, Fusion, ESX, or vSphere. Give it a try. Come back when you’re ready to take advantage of our nearly infinite horizontal clustering capabilities for big data. Contact me if you’d like to see a demo, would like assistance working with your visibility and data ingestion use cases, or if you just plain want to geek out on this cool software.