How important are security analytics in today's complex security architectures? What are the benefits?
It has become a near 'mission impossible' to totally prevent breaches because of the increasingly large and complex environment security professionals are tasked with protecting. We’re even to the point where many organizations already assume they have been successfully breached by advanced persistent attacks, and in this difficult state of affairs, security analytics are extremely important to help us learn everything we can about our environments and the threats they face.
Analytics can help identify and manage breaches in a timely manner to significantly reduce the ultimate cost that malicious activity will have on a business or other organization.
For nearly a decade, IBM has been tracking the costs of a data breach, and its most recent report found that the cost per stolen or lost record and the average total cost of a breach are both on the rise. In addition, the report found that fewer customers are remaining loyal after a breach.
The importance of security analytics are directly proportional to how much a breach will cost an organization, and in the current environment, they’re becoming essential. Amid the perpetual race of hackers looking to break through a perimeter versus security professionals moving to patch the newfound vulnerabilities – and the cycle beginning over again – security analytics have become invaluable.
What are the most significant challenges involved in getting usable information from massive data sets?
Often the fingerprints of a successful breach are only visible in massive sets of machine data being generated by web proxies or network flow collectors. However, getting usable and actionable information from these data sets has significant challenges. First and foremost, the tools and techniques used to collect, store and search this data must scale to the size of the data. This may seem fairly obvious, but again, because the size and complexity of the average environment is getting so big, it bears repeating.
When the data in question comes from sources such as web proxy servers, the fact that almost all the data within these massive data sets relates to non-malicious, standard business activity is another significant challenge to consider. Differentiating malicious activity from non-malicious activity is extremely difficult as there may only be a small handful of malicious activities each day that are hidden in the billions of interactions that take place every minute.
Traditional methods of extracting usable information from this data involves searching for known signatures of an attack. Unfortunately, advanced hackers and criminal enterprises know enough to modify the threat signature so as to avoid detection. In the end, however, the attack is going to generate outlier behaviors, so a complementary approach to signature and rule based-intrusion detection is analyzing internal and outgoing traffic for statistically unusual behavior.
However, the level of statistical analysis required far exceeds the capabilities of even the more advanced security architects or analysts. For instance, there are generally statistically unusual interactions happening all the time in a typical organization. Trying to scan for unusual websites visited by employees of a large enterprise can generate thousands of false alerts a day.