In Part 1, we built the case that SIEMs are ineffective for threat hunting, based on the following reasons:
- Too slow
- Too dumb
- Don’t scale
- Don’t provide the right visibility
- Too costly
To date, SIEM vendors have not provided the market with the functions needed for producing world-class threat hunting. Again, threat hunting is a method. In order to follow this method, we have to have tools that accelerate and amplify our human work, rather than using technologies that brush aside our method in favor of operating within their paradigm. Too many threat hunting programs are sputtering because we continue to believe that the method should conform to the technology, but that gets things backwards.
What we need for threat hunting is:
- Real-time analysis
- Parsing-free ingest
- Cost-effective data storage
- Open-ended search
- Big Data scale
- Self-defined indicators of compromise, based on evidence within the data
SIEMs have become a high-priced data store for logs and compliance. Analysis is woefully absent. I am asked often, “How is FireMon’s Immediate Insight different from SIEM?” Let me explain how.
First, we begin with recognizing that the human brain is the best analytics engine in the known universe. Immediate Insight makes the human more effective at working with data to detect the unknown and hunt for threats.
Each aspect of Immediate Insight is built with this belief in mind.
- Natural Language Extraction
Using NLE, we can tokenize every portion of a dataset without any need for parsing. This is precisely how Google has indexed the vast Internet without prior knowledge of the content it consumes. These are Big Data principles, and the security challenge is a Big Data challenge.
- Open-Ended Search
Using Immediate Insight, hunters do not have to conform to a query language or closed restrictions – just type what you want to find. Most start with a known indicator of compromise (you subscribe to dozens of them). Immediate Insight finds those IOCs in your network, but goes beyond that with associative analytics mapping the relationships of entities, users, destinations, machines and chatter connected to that IOC.Second, hunters create their own self-defined IOCs by leveraging their neural tissue. If I see A in this quantity, B in this quantity, C in this quantity, that is an 80% probability of an early indication of compromise. No one else gives hunters that flexibility.
- Metadata Enrichment
Immediate Insight applies new attributes to data that it didn’t have when ingested. Let’s take an IP for example. A string of numbers and dots doesn’t really tell us much, but when you add metadata to it (e.g. geo-location, reputation, users associated, CMDB details, and so on), you can get more information about that entity, AND expand its relationships. The data comes to life.
- Not Another Data Store
Immediate Insight is your analysis engine, not another data store (e.g., SIEM). Immediate Insight makes use of the data you have already collected to ensure you can hunt effectively within that data. Many organizations are building Big Data Lakes via Hadoop and use Immediate Insight as the analysis that sits atop such a data store. You don’t need another repository and another meter running for data warehousing.
Immediate Insight’s underlying technology is Elasticsearch, and right there in the name is our scale function – elastic. Deploying Immediate Insight into islands or federated clusters is simple – literally 5-10 minutes for set up – which gives hunters the free time needed for hunting instead of appeasing another cranky app.
In this two-part diatribe, I have made the case that SIEMs are glorified databases for a security context, that analysis must start with the human in mind, that solutions must scale to meet the demands of evolving tactics, techniques, and procedures (TTPs) of adversaries, and that a more appropriate way forward is applying Big Data principles to threat hunting.
At FireMon, we unapologetically believe the human is the most essential part of any security program. We want humans to have frictionless ways to work with data, be more productive, secure their environments, and apply their own methods to their tools. Immediate Insight is the fastest, best way to have world-class threat hunting – because it welcomes the hunting method, rather than forcing conformity to a high-priced database.