Metrics: Valuable security indicator or noise?
by Dwayne Melancon - CTO at Tripwire - Thursday, 18 July 2013.
Instead, ask for metrics and indicators that demonstrate success measured against achievable goals and continual incremental improvement. Good examples are:
  • Percentage of breaches that have resulted in loss
  • Mean time to detect and remediate breaches
  • Reduction in the risk of injury incidents detected
  • How often is the infrastructure offline: for how long, what caused the outage, what could be changed to reduce outages
  • Are processes being adhered to
  • Are security practices being circumvented: which ones, by whom, what alternatives could be introduced, what actions were taken to deter future infractions.
Metric top tips

Sxactly which metrics will be useful to any organisation is personal as it’s determined by the business’ goals. However, the principles remain the same:

Set the priority framework: From the outset, everyone within the business should understand what needs to be done to meet the organisation’s objectives. The metrics collected are to verify how well this is being met – or not! This ultimately helps focus efforts on what few things can be done today, to make the most progress towards the end goal. There will always be too much to do – priorities enable staff to make good decisions that align with the priorities of the business.

Perfection takes time: While the end goal may be perfection, a few mini targets of continuous improvement along the road will help build confidence.

Wheat or chaff: Rather than getting into granular detail, IT should be able to quickly and easily abstract the salient stats based on the mission. Solutions exist that automate the process to collect metrics which are then measured against rules. Results are then flagged - with red, yellow or green indicators, so performance can be determined at a glance showing where the organisation is on track and highlighting what requires immediate attention.

Use them or lose them: If IT are regularly producing a report that is not being used, or deemed to offer little value, then why let them continue? Either the statistics need to be presented in another useful format or not collected at all.

Ultimately, when it comes to metrics, don’t allow your IT team to hide behind complex pseudo-science or bamboozle you with stats. Show you’ve confidence in your IT team by asking what the end result is, what the requested investment will deliver, how it helps with the end goal of driving revenue growth or profitability and even how it will make customers happy. If you don’t understand the response make them explain it in a language everyone recognizes. They can speak English – although sometimes it has to be forced!

IT has the intelligence available at their fingertips – you just have to ask them the right questions to get it.


Harnessing artificial intelligence to build an army of virtual analysts

PatternEx, a startup that gathered a team of AI researcher from MIT CSAIL as well as security and distributed systems experts, is poised to shake up things in the user and entity behavior analytics market.

Weekly newsletter

Reading our newsletter every Monday will keep you up-to-date with security news.

Daily digest

Receive a daily digest of the latest security news.

Thu, Feb 4th