What are the most important application security metrics?

In this article, we look at two important application security metrics to evaluate to ensure the right controls are in place for an organisation’s critical applications - vulnerable code and application access. We welcome back James Doggett, a senior security and technology risk expert who’s seen it all. With decades of experience in senior roles at organisations like EY, JP Morgan, Kaiser Permanente and AIG, Jim’s enterprise security wisdom has been a great source of support.

As a security professional, I’m always going to assume that threat actors are going to get past the first line of defence. And our applications act as a second line, so we need to ensure they are as secure as they can be.

Why is application security important?

When we first discussed the topic, Jim mentioned right at the top that it’s particularly pertinent:   "A threat actor’s first task is to break in somehow, but that doesn’t really get them a lot unless they got lucky by hacking somebody who had something very valuable on their computer. Then they are looking to escalate privileges and laterally move across the network of the company they broke into. And that lateral always ends up trying to break into some type of application. As a security professional, I’m always going to assume that threat actors are going to get past the first line of defence. And our applications act as a second line, so we need to ensure they are as secure as they can be." Unauthorised access to an application can be particularly damaging, so it needs to be a top priority for security leaders:  "If a threat actor breaks into an application directly, they can use the application itself to steal data. For example, an attacker breaks into a medical record application looking for a specific person, like a politician or movie star. It’s easier to just look up their chart using the functionality built into the app than coming from underneath and parsing through a database that could be cryptic. It makes their job a little easier."

Identify critical applications

In order to effectively evaluate application security metrics, Jim says the first thing to do is identify the critical applications. Modern organisations use a huge number of applications, whether they are built in-house or purchased. It’s essential to understand the relative importance of each application to the business, starting with understanding which are the mission-critical apps. For example, a medical record system in healthcare or part of a trading system in an exchange. This is also important when it comes to compliance, as regulators will need to know that your critical applications have appropriate controls in place and are being reviewed.   "Today, many companies have too many applications to focus heavily on all of them separately. So, they need to apply risk disciplines to the apps and prioritize those which are most important to the company and implement more stringent controls and assessments around those identified. While doing this, include applications that have not been internally developed. Purchased applications may also be critical to the company and could have weak security, vulnerabilities, or malicious code embedded, such as Solarwinds."

What are the most important application security metrics?

There are two key application security metrics to measure, says Jim. The first is vulnerabilities in the code. The second is high-risk privileged access to the application.

It’s got to be risk-based. You can’t fix everything all the time. You’ve almost always got more vulnerabilities than you can actually patch.

Application security metric 1: Vulnerabilities in the code

With the large number of applications in an enterprise, carrying a significant number of vulnerabilities that can be addressed, prioritisation is critically important for most organisations. Jim discussed vulnerability prioritisation in his previous Metric of the Month article. "It’s got to be risk-based’, says Jim. ‘You can’t fix everything all the time. You’ve almost always got more vulnerabilities than you can actually patch.’ However, remediating vulnerabilities on applications is slightly more straightforward than vulnerabilities on systems: ‘A system level vulnerability could affect a large number of applications, but an application vulnerability only affects the application itself, so there’s only one thing to remediate." With system vulnerabilities, organisations tend to scan daily or weekly – with regularity and often automation. ‘But with applications it’s a little more complex. You have to go into individual applications to check for newly available patches. The good news is patches are less common at the application level.’  That means it is important that you measure the coverage of your scanning tools – are all the critical applications being scanned? And the regularity of that scanning. Are those scans happening when they should be, based on an established SLA?  "There was a time when we would hire people to come in and look at certain applications, though now this is done by specific tools, and we would end up with a long list or report of all these things that need to be fixed. So, you have a spreadsheet of 30 things you need to fix on one application. You can prioritise that list relatively easily. But when you do that across every application in the enterprise, it becomes extremely cumbersome. " "That’s where you need a tool like Continuous Controls Monitoring to not only automate much of that process, but also to help understand the priority of those fixes across all applications, and crucially, to ensure that they’ve actually been fixed. As well as understanding and ensuring that applications are being scanned, and scanned when they should be." Here's a dashboard that highlights example metrics:

Application security metric 2: Privileged access to the application

The second key application security metric is about ensuring the correct access to that application. While this could be construed as part of Privileged Access Management, Jim notes it is still an important part of application security: "We should be broadening our focus and consider all aspects of application security for those apps most important to the company." "Access rights can be messy and intricate with people leaving or moving within the organisation. If the app has been written so that access relies on Active Directory, removing it is relatively simple. But that isn’t always the case, so you usually need to go app by app to remove access," says Jim. "Historically I have found that while people at the network level are pretty good at controlling network access, when you get to the app level, it still tends to grow over time. Sometimes when people leave companies, they're very quick to get rid of the network level access but not the application level access. With people changing roles within a company, they may accrue more and more access over time. And it’s much more common to give access than take it away."

We found there were over 200 people that had unnecessary sensitive access to an application and he was absolutely shocked.

There are levels of access, of course. Privileged access is essential for some roles, but following the principle of least privilege, it is important to keep it only for essential roles. ‘It’s particularly important for more sensitive privileged access, like sysadmins or people who are able to edit data. If we take the medical record example, do you want someone who isn’t a medical practitioner to be able to update a medical chart?' "In an access education exercise for senior management, we spoke to the CFO and evaluated the general ledger system he was responsible for. We found there were over 200 people that had unnecessary sensitive access to an application and he was absolutely shocked. They could basically post any transaction in the system and bypass segregation of duties. From that point on, he wanted a monthly list of privileged users because he understood the risks of something bad happening on his watch." Here's a dashboard that highlights an example metric:

Anytime you can start to merge different control areas to provide a better picture of how much risk there is, I think it is a fantastic idea.

How can Continuous Controls Monitoring help measure the key application security metrics?

According to Jim, it’s key for application security metrics to be centred on their criticality and value to the business. It’s difficult for many organisations to understand that context and that is where a Continuous Controls Monitoring platform can help. "Anytime you can start to merge different control areas to provide a better picture of how much risk there is, I think it is a fantastic idea,’ says Jim. CCM can help to overlay these two key application security metrics – vulnerabilities and access. Jim provides an example: You identify some application vulnerabilities. You’d also like to know if they’re on a critical application. And it would also be great to know who the critical users are for that application. Are any of those people prone to failing phishing tests? ‘As you start to combine that all together, you get a pretty cool picture of your risky applications and your risky people." The frequency with which you can assess these metrics is also critically important. If it’s a highly manual or time-consuming effort, the metrics start to lose their value given that an application’s vulnerability status is subject to rapid changes.  Jim also highlighted the value of automating security metrics and measurement. Automation removes the considerable manual effort around collating, aggregating, and reporting security metrics, as well as eliminating the potential for human error or bias. It supports consistency, accuracy and data trust.  Similarly, Jim notes: ‘While application vulnerability scanning is less frequent than system vulnerability scanning, it is still essential. If done in a non-automated way, it can be forgotten or pushed down the priority list, and you can easily miss your scanning SLAs.’  We’ll leave you with an idea of what some of those overlayed metrics might look like in a mock dashboard.

 Application security metrics are an integral part of any security measurement programme. It's essential for security teams to ensure their applications are scanned for vulnerabilities with regularity, and that those vulnerabilities are remediated. It's also extremely important to keep on top of the access to these applications as that can pose a profound risk to the organisation in the event of a breach.  

If you're enjoying our Metric of the Month content, subscribe below.