Reliance on manual processes is stifling mature security metrics

The new 2020 Financial Services Security Metrics Report found that security leaders are aiming for better ‘metric maturity’ over the next 12 months. Over half of respondents described the maturity of their security programme as basic, elementary, or intermediate. However, a reliance on manual processes around security programmes appears to be holding them back. 

Let’s dig a little further into the details here, taking a look at what we mean by ‘metric maturity’, and what organisations can do to improve it. 

Security leaders are aiming for better ‘metric maturity’ 

When it comes to metrics programmes, there isn’t really a standard basic template. While there are certainly frameworks to build on, they tend to cover a huge number of security domains which makes it extremely difficult to cover everything effectively. 

But in some ways that’s appropriate, because all organisations are different and will therefore have different wants and needs. In fact, the report suggests that knowing what metric to use can be a major hurdle, with 16% of respondents saying this is their top challenge when it comes to producing security metrics. It really is a tough game. 

Our CEO Nik used this classification to measure the maturity of a metrics programme for his last talk at FS-ISAC in 2019, and came up with a relatively simple 5 stage interpretation: 

The report points out that about half of respondents would describe their security programme as Basic, Elementary, or Intermediate. So, let’s look at the early stages. 

There was a time not that long ago that many security leaders reported using little more than subjective attestations, gut feel, and a visit from one of the Big Four. Due to increased threat levels and a shift towards the importance of cybersecurity in boardrooms across the world, this is mostly a thing of the past. 

The Elementary level is like a strobe light, giving snapshots on a periodic basis. While this is more valuable than the basic level, the key issue is that users are unaware of security posture in between snapshots. It’s possible that a metric went from x to y then off to pqr then back round to x. Sure, it started and ended with x, but there would be no visibility or awareness that the organisation was at risk from pqr.  

The Intermediate stage is the introduction of basic automation. When a security programme is at this stage, it is likely still suffering the same challenges as previously, but perhaps to a lesser extent – reliance on manual processes, lack of visibility, trust in the data, etc. 

The report found that 65% of security leaders want to improve their metrics maturity to an Upper Intermediate or Advanced stage by next year. Given the above that half of programmes are no better than Intermediate, that means that security reporting and metrics will need to improve quickly and effectively in order for organisations to reach this goal. 

What’s holding them back? 

The report found that the majority of respondents are using tools that come with their own challenges. 

Here’s a breakdown: 

These methods are often subject to manual processes, dependent on ninjas, susceptible to human error, and take up valuable time and resources. Despite this, they are the norm. Reliance on these processes is stifling advanced security metrics. 

Without a platform that can be relied on as a ‘single source of truth’, data is often siloed in individual or disjointed tools. This means that security teams are forced to work individually between tools. 

If we look back at the stages of maturity, the key element in jumping from stage to stage is increasing levels of automation. In order to continue to Upper Intermediate and Advanced metrics programmes, there are several key processes that need to be continuously automated, such as entity resolution and data triangulation. This allows the automatic and continuous collection and correlation of data to produce the metrics and measures that key stakeholders need to see (what we at Panaseer call Continuous Controls Monitoring). 

And in the future, security teams will eventually be able to look to the Advanced stage for a programme that not only automates the metrics but predicts gaps in coverage and automates remedial action as well. Although that may be a little Skynet for now. 

If you’re looking for more on how to implement or mature a security metrics programme, check out this excellent guide written by our Lead Data Scientist, Leila Powell: https://panaseer.com/business-blog/principles-for-implementing-security-metrics/.