1st Line vs. 2nd Line — FIGHT!
August 29, 2017
Ok, ok — while it’s not quite that bad, there are growing tensions between 1st line and 2nd line around risk reporting. Witness: a quote from the CISO of a top UK financial…
“When my CRO [Chief Risk Officer] asks me a question, I have to pass it to Operations to get an answer. What comes back is always ‘Here’s what we can tell you’, which is never what I need to know. I speak to my peers — we all have the same problem.”
So, what’s at the root of this issue? As we’ve talked with 1st and 2nd line teams, two themes keep repeating.
[TL;DR — Risk and security teams need to work together, and ideally with a community of peers in other firms, to understand and tackle the problems associated with measuring risk and security performance … and not just throw the issue over the fence.]
1. It’s hard to turn data from point products into useful management information (MI)
Rather than me write about this, I’m going to be lazy and copy what a CISO told us in an off the record chat about their team’s experience.
“On topics like vulnerability, most people take lists of numbers to their execs, because that’s what you get from products. But all the questions I have in my position can’t be answered by a GUI. I can’t make sense of the data from tools to say ‘Are we happy or not? Are we dealing with things at the right pace?’ I can’t use it to understand how to focus our efforts; to know what is a systemic problem; where I need to change a process or our tooling. It’s really awful what you get.
“So you always end up needing to circumvent the GUI. You go to the API, and try to understand the 20 or whatever data fields. And soon you realise ‘oh, this is a lot of data’ and you can’t deal with that in traditional tools to get the trending you need.
“But even before that, we’d start producing analysis, and somewhere we’d find a particular edge-case. So we’d investigate it, and it would invalidate everything we’d done. Then we’d try again, and the same happens. There are only so many cycles of that you can run when your CIO is asking ‘Why are these numbers like this?’ This stuff sounds so simple. But just getting to a level where you know what going on even with basic hygiene stuff — it’s really hard. But it’s so important, even if you’re not happy with what it tells you.”
2. It’s often necessary to trust, without being able to verify, that data underlying MI is accurate
I’m going to be lazy again. Here’s a different CISO talking about this…
“Everyone pretends their asset inventory is fine until you show them it’s not. Yes, it’s not necessarily the most sexy thing, it’s not at the top of list. But if we’re honest, is what’s at the top really the most important … or just the most visible?
“So when you need to get an answer about some area of security, bcause that foundation is missing, you’ll go talk to the <redacted> team who manage <redacted control>, and they’ll give you data and say ‘This is everything, we’re totally confident this is 100% of the assets relevant to your question’. And if you have no real way to sense check that, you have to assume a level of trust in that data. But say you go to the <other redacted> team and take their data and cross check those two data sets … oh hang on, you find a ton of contradictions. Suddenly that ‘100%’ has shifted to look more like ‘80%’ — and now you’re left with the question Hm, what are those assets? Where are they? What business service do they support? Why aren’t they in the first data set? And what’s missing from both?
“Often you find you’re in a position where you have lots of ‘silver sources’ but no ‘golden source’, no single version of the truth. And the silver sources you have present multiple versions of the truth. So which one do you go with? Which do you trust more on any given day?
“So let’s say I find a material issue and I take it to my CIO … and he asks ‘How big is this problem, where else does it occur across our estate?’ If my answer is ‘We’re not in a position to know that’, well, that won’t go down well. It won’t inspire confidence.”
For a more technical dive into these themes (and a few others!) check out this talk by Dr @lc_powell of the Panaseer Data Science Team.
If what you’ve read resonates, or you’d like to find out how @panaseer_teamis building a community of customers to tackle problems get in touch now
{Bonus postscript bit}
It’s no coincidence that issues like the ones mentioned above are coming into sharp focus as regulatory scrutiny into ‘cyber’ is evolving. For example, whereas regulators used to focus on whether control mechanisms were in place to monitor or secure assets, they are now asking firms to show how effective those mechanisms are. (This also goes beyond point-in-time assessments, which is why job titles that mention things like ‘continuous controls assessment’ are popping up more frequently.)
To complicate things, firms aren’t being asked to show they have their finger on the pulse of risk exposure just because regulators care about them specifically. It’s also about the risk they could pose to the ecosystem they plug into. In finance, this means ‘all the other banks’.