Goals, Questions, Metrics

This week we take a short trip back into the world of data science to introduce a framework we’ve adopted to support the delivery of Security Insights to our customers. Goals, Questions, Metrics (GQM) is a measurement model that uses business level objectives (in our case the pursuit of better security) to drive the identification of the right metrics to help organisations to measure progress against those objectives.I came across the GQM framework after a presentation by Alex Hutton and David Mortman at RSA last year and doing some further research into the subject I was struck by its simplicity and effectiveness:

• Goals: The definition of what we want to accomplish
• Questions: Contextualise the goal and help to understand how they can be achieved
• Metrics: The quantitative measurement that helps to answer the question

The reason I found it so compelling was that in discussions with various CISOs and their teams I was constantly hearing ideas for metrics that left me thinking so what? What do you learn from that? How does knowing that make you more secure? However, by applying the GQM framework to each metric in turn it became easy to weed out poor metrics or in some cases it caused us to reframe or amend a metric such that it fit the framework and aligned to a particular goal.Here is a simple example of GQM in action:

• Goal: Comprehensive coverage of vulnerability scanning.
• Question: What proportion of the estate is currently scanned by the vulnerability scanner?
• Metric: Percentage of estate scanned

What makes a good metric is a subject for a future post and don’t underestimate the difficulties in picking metrics that provide a meaningful measure of security performance. However, keeping the objective in mind and selecting a set of questions that are meaningful to the CISO helps security teams focus on what really matters.We are continuing to explore this framework and possible extensions to it and will report on the success of this work in future posts. For now, I encourage you to take a look at the links above and familiarise yourself with the GQM framework.