New feature: Cybersecurity Controls Scorecard

In short, we’ve created a single score that shows your organization’s security controls performance, rolling up controls performance of key initiatives. It gives CISOs and other security leaders a single pane of glass to both manage the organization’s security posture and report it to any number of stakeholders, such as the board, risk committees, or regulators in a simple-to-understand summary of the organization’s cybersecurity controls.

In this blog, we’re going to look at:

• Why we’re rethinking the approach to scorecards;
• What the right solution looks like;
• And our vision for the Cybersecurity Controls Scorecard.

Why we’re rethinking the approach to scorecards

CISOs are now under a huge amount of scrutiny from both internal and external stakeholders. It’s not just reporting to regulators, but also a range of internal stakeholders that are interested in enterprise risk: the board, the executive risk committee, and operational risk committees. They all want the same information on the big exam topics right now: cyber resiliency, cyber posture and cyber hygiene. These are fundamentally measured by the efficacy of security controls.

There are a few kinds of approach.

Some CISOs report a bunch of important security metrics to the board and other non-technical stakeholders. But one of the CISOs we recently spoke with at I4 said: “Showing up with pages of metrics is the fastest way to get leveled”. Non-technical audiences don’t want to see all your controls data, even in shiny dashboards. It’s just too much.

Then there’s a more business friendly approach. Some CISOs use out-side security rating scorecards that are easy to generate and consume. But these scorecards are insufficient to manage risk. They use superficial signals from the outside of an organization to infer cyber risk. This can be helpful for generating a quick, directionally correct view of cyber risk, but will not stand up to the long-term scrutiny required for board and exec reporting.

Some try to build something themselves, but there are various pitfalls to the DIY approach. One of our customers said that before they had Panaseer, they had dashboards that had been built by interns over the course of several years, and now nobody knew how to maintain or update them.

These approaches are basically broken. Over half of respondents in our 2024 Security Leaders Report said audit was a manual process, with security teams spending 59% of their time on manual reporting.

What does the right solution look like?

We’ve spoken to dozens of CISOs and other security professionals responsible for security reporting, and they’ve made it clear that the right solution has a cascading set of reporting requirements.

Most importantly, a single data set that will allow them to manage their team without rampant data trust issues, while simultaneously upholding accountability for the security team and wider business. Not just data trust, but “data truth”, is essential. This is the “data truth” that the CISO wants to be held accountable against.

This data truth is the basis for security measurement, which is crucial for ascertaining compliance. Both to your own internal thresholds, and to external pressures such as regulators and frameworks.

It must also allow for simplified metrics for non-technical stakeholders, who don’t think in terms of EDR coverage or vulnerability SLAs. They want simple scores for security concepts couched in terms of the context of the business – business units, services, or applications.

They also need long-term trends. Particularly for the regulators and the board, they think quarter over quarter, year over year. It’s not about whether you pass or fail individual controls, it’s about whether things are improving and you’re reducing risk.

It also needs to be validated against third party standards. That means mapping to security standards like NIST CSF or CIS.

Ultimately, you need to use the data to improve security and reduce risk, meaning it needs to be explorable to findings that are genuinely actionable. From overall performance score, through your initiatives and controls, down to groups of devices or vulnerabilities in breach of policy that allows you to take action.

Cybersecurity Controls Scorecard: The vision

In this section we want to give you a taste of the long-term vision.

Note: this image is a prototype. The current beta version has some of the capabilities and looks slightly different.

Cybersecurity Controls Scorecard prototype

Figure 1: Cybersecurity Controls Scorecard prototype

The Cybersecurity Controls Scorecard (Figure 1) is not just a simple communication of cyber posture like you might see from other scorecards. It’s based on real data truth that the CISO wants to be held accountable to. It’s communicating the reality of the organization’s total security posture on that exact day. But in a simplified way.

Envision a CISO who is preparing for their quarterly report to the board. They can use this scorecard to tell the story of their security program. It effectively summarizes the overall security posture of the organization in a simple way that is easy to understand for non-technical stakeholders. While 50% is not an amazing score, the trend graph demonstrates significant improvement over time.

The overall score is informed by the scores for individual security initiatives. Security initiatives are defined by the CISO and are typically based on controls domains (such as vulnerability management) or threats (e.g. ransomware). While slightly technical, these security initiatives are within range for a board. We’re not showing them EDR coverage, vulnerability remediation SLAs, or mean time to detect.

The third section of Figure 1 provides a Red/Amber/Green (RAG) map of controls performance across the organization, highlighting hotspots and success. These are breakdowns of security controls performance based on different parts of the business – business units, operating companies, services, geographies, etc. This information is useful at the board and executive levels but is really the focus area of operational risk committees.

While this may not be the main event in a board report, it is extremely powerful for the CISO’s day-to-day decision-making around prioritizing which areas of the business need attention and remediation. As it is automated, it saves the CISO and their team a huge amount of time that would otherwise be spent gathering all that information together.

If the CISO is interested in a particular score, they can drill down for more information or click through to a dashboard that shows the individual metrics and control failures that are creating the scores.

Figure 2: Example dashboard

Figure 2 is an example of such a dashboard, showing a score and the controls for an individual security initiative. The score is calculated based on compliance to risk thresholds set for individual controls.

Use cases

Ultimately, the Cybersecurity Controls Scorecard is going to provide simplified reporting of security posture that’s digestible for non-technical audiences. This will help create better communication with the board and other key stakeholders, fostering trust and support for the security function. It will provide the basis for accountability for the CISO, that is crucially built on genuine, up-to-date data truth. It will help security teams to get on the front foot, positioning their program and its performance rather than just responding to the barrage of questions on cyber resiliency, cyber hygiene, and security posture.

It will provide single pane of glass for the CISO to manage the organization’s security posture, driven by automation. This is going to help save CISOs and their teams a huge amount of time on reporting security risk.

Crucially, it will also help security teams to improve their prioritization. They will be to identify areas of the business that lack cyber controls and will be able to make decisions that have the most impact on risk reduction. All based on a single view.

The final word

As noted, the Cybersecurity Controls Scorecard is currently in beta. Select customers and prospects are getting access to an early version. If you would like to be one of those select prospects, feel free to request a demo.

Otherwise, look out for another future blog which will focus more on the live capability when the Cybersecurity Controls Scorecard is generally available.