Aligning metrics to security controls frameworks

Security controls frameworks are an essential aspect of any security programme, providing guidelines for the controls an organisation must implement in order to effectively defend themselves. In this Metric of the Month, we explore what makes a good framework, the challenges around compliance, ransomware, automation, and more, as we interview a world-leading expert on security controls frameworks.

We are joined by Phyllis Lee. After a 25-year career in government agencies, Phyllis has turned to providing Controls guidance for the global security community as Senior Director for the CIS Critical Security Controls. She has been instrumental in CIS’s latest Controls offerings: CIS Critical Security Controls v8, Implementation Groups, and Controls Assessment Specification.

What makes a good security controls framework?

For most organisations, security is a bolt-on afterwards. Every organisation has limited resources, so you really need to align yourself with a security framework that helps your organisation prioritise within your business needs.

There are several cybersecurity frameworks out there, and organisations often choose more than one or combine several to create something more tailored to their business. When building a framework though, what makes it effective?

“A good framework, in my opinion,” says Phyllis, “is achievable, provides prioritisation, and shows the efficacy of what you are doing.”

“It’s important to help organisations with questions like ‘what do I need to do?’, ‘what do I need to do first?, and then support implementation. For most organisations, security is a bolt-on afterwards. Every organisation has limited resources, so you really need to align yourself with a security framework that helps your organisation prioritise within your business needs.”

There is an element of uncertainty around benchmarks, baselines and thresholds – many are unwilling to share or divulge on this topic.

“It’s okay to have a low bar at first. We need to crawl, walk, run. It’s important to get the essentials in place first. A lot of people use the word ‘basics,’ but I don’t think that’s the right word. Basic implies easy. And the essentials are not easy. Once everybody is crawling, then hopefully one day you can walk and run.”

This is partly the reason CIS is recommending Implementation Groups (IGs) – Controls grouped by priority and difficulty of implementation. It’s not just about difficulty though, it’s about achieving reasonable levels of protection. Achieving each IG also indicates that the organisation would have appropriate Controls to defend against various attack techniques, which CIS was able to validate when developing their Community Defense Model (CDM) – but more on that later.

The first, IG1, focuses on essential cyber hygiene – enterprise asset inventory, software asset inventory, and data protection.

“I don’t want to be glib and say ‘do these three things,’ but they are essential. You have to know what’s on the network. You have to know what software you’re running. And you have to know where your sensitive data is.”

Security Control frameworks can often be generic, so you have no idea what it means to implement successfully. It’s unclear what they’re asking for.

The challenges with security framework compliance

Some security controls frameworks can be extremely difficult to enact and comply with, especially the bigger frameworks with hundreds or thousands of controls.

“Security Control frameworks can often be generic, so you have no idea what it means to implement successfully. It’s unclear what they’re asking for, so I feel badly for especially small/medium enterprises that don’t have the experience, because it’s a bit of a guessing game.”

Phyllis mentioned that ‘loose language’ is the norm.

“In the security community, you don’t want to be the one that says ‘this is okay,’ and then something bad happens. So, we often default to language that can give you an out: ‘as needed,’ ‘whatever your industry requires,’ etc. Really what they are looking for from the framework is guidance.”

This lack of clarity and need for guidance is a challenge that Phyllis reflected on in the creation of version 8 of the CIS Critical Security Controls framework and the Controls Assessment Specification, which provides more specific, prescriptive guidance for organisations that already have tooling in place. It’s a relatively new concept for a framework to give such detail in advising organisations what to measure and what good looks like.

Phyllis highlights a common question that acted as a driver for this initiative: ‘How do I know if I implemented that successfully?’

Controls Assessment Specification provides detailed metrics on Controls, with example SLAs – whether weekly, monthly, etc.

“Big banks are subject to lots of regulatory requirements, have a dedicated staff and tooling, so they can measure more frequently. Controls Assessment Specification is mainly aimed at the small and medium enterprises who need the most guidance.”

“The CIS Controls are not a mandatory regulation, yet it’s so widely adopted.” If the framework is mandatory but difficult to implement, it becomes problematic when the auditor comes by. “Of course, you have to implement the frameworks that you’re subject to for regulatory reasons, but if the regulatory framework isn’t clear, you can only hope you meet what the auditor is expecting you to implement.”

The authority of a security controls framework

Part of the reason that the CIS Critical Security Controls are so widely adopted is because of the authority it holds in the security community, and the trust that the community has in its publications. As security practitioners, we look to CIS and the Controls as one of the gold standard control frameworks. This is something Phyllis takes very seriously.

“I am amazed by the authority that CIS has. When I was working in government, no one ever really trusted anything we said. But when I came here to CIS, I can provide some advice and everyone says ‘great.’”

On one hand, it’s great to get that recognition and to be a trusted source, but it also puts pressure on Phyllis and the team at CIS to really get the framework right, because so many organisations are reliant on it.

A community feedback loop is essential to the success and authority of a security control framework.

“We’re very open and transparent with the community. We welcome valid feedback and try to take it onboard and provide a course correction. Take the example of NTP servers. The IETF specifications said everyone needs three NTP servers, but as I talked to the community, no one used three NTP servers. Nobody. I talked to people from Fortune 100 companies and small medium enterprises. Everyone just defaulted to two. I read the specification and the latest version asks for four.”

“But what’s the intent? If one fails then you still have DNS working. But when not even the high-budget tier-one companies have more than two, we realised we needed to roll it back to two. Even though it’s out of compliance with IETF, it’s achievable rather than aspirational.”

Is there a way to use frameworks to protect against ransomware?

Ransomware is a hot topic, and in part contributes to the importance of the Community Defense Model (CDM). “Ransomware has taught us that we need to be more vigilant,” says Phyllis.

“We have 18 Controls supported by 153 Safeguards mapped, where appropriate, to MITRE ATT&CK techniques and sub-techniques that comprise attack patterns, or the steps of an attack. That way, if security teams implement x Safeguard, they know they are defending against y (sub-)technique.”

“Then we identified the top five attack types from reports such as the Verizon DBIR, including malware and ransomware, and created the corresponding attack patterns. The result of that is that we are able to provide a mitigation mapping of Safeguards for specific threats such as ransomware.”

This mapping links back to the aforementioned Implementation Groups, so that users can understand which techniques they are protected against at each Implementation Group.

How will frameworks evolve as automation becomes more prevalent in cybersecurity?

“I think frameworks need to evolve. We’re killing organisations with multiple compliance frameworks that they’re subject to. They’re spending millions of dollars to comply to regulations.”

The author of a framework should be the authoritative source on how you measure success in that framework. We all need to step up and take that responsibility.

“I’ve been trying to figure out how to solve this multi-framework problem. How can we as CIS and we as a community provide solutions for end organisations?”

“There is a view that more regulation is better because everyone needs insights, because everyone’s failing at cyber. So, we need another framework, we need another regulation, and we need another law. At a certain point, it becomes too unwieldy.”

“I think we need to find a standard way to represent the Controls, the supporting Safeguards, and automated assessments.”

It’s not just about cleaning up once a year before an audit. Automation is key. You need to be continuously monitoring Controls to make sure that your metrics are within your threshold and you’re doing what you need to do to stay compliant and stay secure.

“The author of a framework should be the authoritative source on how you measure success in that framework. We all need to step up and take that responsibility.”

How does Continuous Controls Monitoring support a security control framework?

When an organisation has a security controls programme in place, they need to make sure it’s actually effective. ‘People really want that automation, they want to know how well they measured against the Controls. As you are patching, updating, and adding new software, you need to measure drift over time. It’s not just about cleaning up once a year before an audit. Automation is key. You need to be continuously monitoring Controls to make sure that your metrics are within your threshold and you’re doing what you need to do to stay compliant and stay secure.’

Security controls frameworks are essential to any security programme at any level of maturity. Not simply as a guideline for controls, but as support for what to measure, how to improve, and as a way of helping organisations achieve best practices at any level. Find out more about how Panaseer can help your organisation by requesting a demo.

If you’re enjoying our Metric of the Month content, subscribe to our newsletter.