Skip to main content

Build vs buy framework for Continuous Controls Monitoring

June 21, 2024

Neil Hooper

To build, or not to build, that is the question.

“Build vs buy” is an increasingly important decision for companies looking to implement a Continuous Controls Monitoring (CCM) solution. Organizations large and small find themselves with a growing number of standards and compliance regulations that require disparate systems and processes in place to support them.

The daunting task of managing large cyber risk inventories manually, via spreadsheets and homegrown databases, is a decidedly futile challenge for already overwhelmed security managers. Leading analyst Jie Zhang of Gartner highlights CCM as a solution: “To address this issue, leaders in security and risk management must adopt technology such as Continuous Controls Monitoring. It automates the monitoring of cybersecurity controls’ effectiveness and aids in gathering relevant information in almost real-time.”

Inevitably, companies find themselves faced with a decision. Is it more beneficial to build a custom CCM management system to meet specific needs? Or purchase an existing commercially available tool with comprehensive functionality already built in?

If a cybersecurity team wants to build their own CCM function, there are a lot of difficult challenges to address. Let’s explore the benefits and challenges of build vs buy in this article.

Recommended decision framework

A generally accepted framework to evaluate build vs buy across costs and strategic value is broken down with core decision factors:

  • Risk
  • Maintenance
  • Upgrades
  • Integration
  • Time-to-value.

These are some of the challenges that any business faces trying to build their own platform, with answers that are relevant to CCM based on these dimensions. Additional detailed CCM-based program challenges are covered in the next section.


- Build

+ Buy


Custom development

The entire BI platform must be coded from scratch.

Configuration possible to customize application for unique reporting needs.

+ Buy

Reduced likelihood of major development problems and also easier ability to support requirements.

Implementation timeline

Primary critical path to go live.


+ Buy

Packaged applications can be implemented faster than if the solution were built from scratch.

Quality assurance

Onsite and minimal
Minimal testing increases risk and impacts client relationships.

Tested twice
Applications tested once in the lab by the manufacturer and again in the marketplace by customers.

+ Buy

Rigorous quality assurance procedures ensure applications are ready for enterprise adoption.

The Build option introduces risk around developing from scratch without the benefit of numerous deployment learnings, unknown timeline extenders, and the lack of fully staffed QA teams continually keeping solutions current.


Considerations include: Continuous improvement, quality management, knowledge transfer, and problem resolution.

- Build

+ Buy


Continuous improvement

  • Manual upgrades

  • Manual changes to code for business process and/or organizational changes

  •   Release upgrade and migration software

  •   Source code and tools to manage custom code and version control

  •   Support packages for incremental corrections

+ Buy

  • Enhanced robustness and minimized potential for failures and shutdowns

  • Easier to enhance business processes

Quality management

  • Manual intervention based on experience of implementer/user staff

  • Proactive safeguarding of system and services

  • Implementation tools and methodologies

+ Buy

  • Reduced number and duration of disruptions, failures or shutdowns

  • Improved monitoring of the technical throughput of end-to-end business processes throughout the landscape

Knowledge transfer

  • Manual transfer of information

  • Based on the willingness of implementation team

  • Integration of content, tools, and implementation and operation methodologies

+ Buy

  • Eliminates the need to create content, templates and documentation needed for implementation and on-going operations

The Buy option explicitly includes maintenance as business-as-usual, where the Build option relies on ongoing maintenance outside of core competencies. But I’d also question whether the Build core competency maintaining solutions is connected to multiple sources and continually adding new technologies.


Considerations include; long term infrastructure required to keep pace with continuous change.

- Build

+ Buy




  • Upgrade costs are primarily driven by custom code conformance to standards (30-50% of overall upgrade in custom code environments)

  • Because the entire application portfolio is coded from scratch, upgrades are challenging

  • Many customers opt to reimplement completely at time of upgrade

Defined and easier

  • Dedicated sizeable group of developers focused on future requirements

  • Developer’s core competence is enterprise software development

+ Buy

  • Improved ability to leverage new technology and stay current

  • Less costly to refresh technology

The Build option adds costs for every ongoing upgrade, known and unknown. The Buy option includes this as business-as-usual.


Considerations include; Total Cost of Ownership, Flexibility, Benefits, and Risk.

- Build

+ Buy


Total cost of ownership

  • Several components would have to be purchased from different venfors and then integrated together, possibly requiring that new skills be obtained to work with the separate components.

  • Requires the implementing firm to absorb the total expense on its own

  • The individual components are already integrated

  • While packaged applications may also require that staff acquire new skills, the task should be less of an issue with a single, packaged solution

+ Buy

  • Ongoing maintenance and support cost should be lower due to the smaller number of individual software components that would be used

  • Vendors are able to leverage the development costs over a large number of customer organizations


  • Is a more custom approach and likely that meeting new integration requirements in the future would require more effort than under the buy scenario

  • Has an inherent advantage since integration software is designed to provide solutions across a wide variety of organizational applications

+ Buy

  • The packaged application products can be used to implements the initial project(s) to achieve the anticipated return on investment (ROI), and for other integration projects that may arise in the future


  • Equal quality results can be achieved by using either approach, so there should be no difference in the application functionality that could be provided under either the build or buy approach

  • Time required to implement the system should be significantly less since much of the coding and component linking has already been completed (and rigorously tested)

  • Many application integration vendors are starting to include vertical industry templates

+ Buy

  • Should normally generate the targeted benefits to the organization more quickly than the build approach


  • More custom work creates future support risks

  • Implementations take longer and the likelihood of project failure is higher

  • There is less custom work, which reduces the likelihood of major development problems and also eases future support requirements

  • Packaged application solutions reduce risk since their implementations can be completed faster than if the solution were built from the ground up

  • This is important from a risk perspective due to the direct link between project failure and the time it takes to implement

+ Buy

  • Buying a packaged integration solution is less risky because organizations can choose packaged solutions that have already been widely tested in the marketplace

  • B2B integration projects are more likely to be accepted by key trading partners if they rely on a proven packaged integration solution rather than a custom-built application

External and internal integration remains a constant focus of any CCM project. The Build option carries the burden of building integration with ongoing risk and cost. The Buy option includes economies of scale of learnings across a global installed base, reducing risks and costs.

Time to value

Considerations include; Implementation Timeline, and Total Cost of Ownership.

- Build

+ Buy


Implementation timeline


  • Implementations take longer and the likelihood of project failure is higher


  • Packaged application solutions require less custom development can be implemented by configuration which can be faster

  • This is important from a risk perspective due to the direct link between project failure and the time it takes to implement

+ Buy

  • Packaged applications can be implemented faster than if the solution were built from scratch

Total cost of ownership


  • More custom work creates future support risks

  • Maintenance costs can be higher due to higher number of integration points

  • Support costs can be higher due to lack of adequate knowledge transfer

  • Upgrade costs can be higher due to extra testing requirements due to custom development


  • There is less custom work, which reduces the likelihood of major development problems and also eases future support requirements

+ Buy

  • Packaged applications are seamlessly integrated allowing synchronization of the entire value chain

  • Elimination of integration, seamless upgrades and extensive knowledge base and support lowers total cost of ownership

The Build option carries the risks and costs associated with custom work, ultimately and consistently delaying time to value. While custom work can seem appealing in the planning phase, the ultimate time to value is generally a more appealing driver of a program.

The Buy option replaces custom work with configuration of settings, attributes, reports, and layouts in an efficient manner, reducing time to value. Finally, the Buy option includes a Customer Success Program dedicated to time-to-value throughout the life of the program.

The speed of change calls for expert knowledge

In this section, we will address the challenges more specific to CCM that companies will need to overcome if they want to build effectively. To quote our customers who have attempted the Build option:

“We built it ourselves. It was a monstrous beast. Never again. We couldn’t keep up with stakeholder requirements.”

“Panaseer proposed a solution in 2021 and we decided to build our own. It probably cost twice as much as the Panaseer quote. And we now see, three years on, that we have built an inferior version of what Panaseer had in 2021. Looking at what you have now, the cost and value gap is huge.”

Here are some of the many challenges:

First, the speed of change calls for expert knowledge. It’s also worth noting that the cybersecurity landscape is constantly changing, both in terms of the specific estates of organizations and the wider industry. That’s particularly relevant to the CCM space, because a CCM tool must adapt to those changes. For example, CCM should be able to help an organization with changes such as:

  • addressing previously unknown zero-day risks;
  • addressing audit findings and weaknesses;
  • and changes in infrastructure due to M&A activities.

These changes require CCM best practice knowledge to manage effectively. As such, Panaseer recommends partnering with a firm that not only has out-of-the-box capabilities but has managed services staff ready to assist with the rapid adoption of changes brought forward by new challenges. The Build option is challenging to staff for future scenarios requiring talented subject matter expertise to continually evolve the program.

The second is data ingest. According to ISACA, the most considerable block to making data-driven decisions is “poor quality information”. One of the main goals of CCM is to address this issue. To be effective, CCM needs to take in data from many sources across security and the wider business. We’ve found when we replace Build option solutions, APIs have often been misunderstood or there are issues with permissions and configurations. This leads to the wrong data being ingested and therefore less trustworthy results.

The solutions

And here’s what you do with that data. Most security functions lack dedicated data scientists. But good data science is essential to good CCM. Normalization of data is critical. Specifically, the entity resolution process, which is extremely complex (but made to seem less so in our blog about it).

Reliable, trustworthy data is essential to CCM, otherwise, what’s the point? You need to be able to prove data lineage, you want transparency and inspectability. From the highest-level scorecard to the most granular detail about a single record. You want the same data being used, and trusted, by all stakeholders.

Another part of what makes CCM so powerful across the organization is the way it supports collaboration. Features like the scorecard allow non-technical users to engage with the platform. A Build option solution often struggles to enable non-technical users as it struggles to translate complex cybersecurity concepts into the language of the business.

Adding Panaseer flavor to the decision-making framework

Large organizations will often consider or start to build their own tools when existing vendors’ solutions are perceived to lack specificity, maturity and functionality. In a maturing space such as CCM, we encourage executives to evaluate a build vs buy decision. But, in our experience, at least for Continuous Controls Monitoring, “build” has often been an unsuccessful strategy. Time and again, we have heard companies return a year or two after initial discussions because their in-house project became too expensive or didn’t function as expected. There are many advantages for a buy decision, but even still there are remaining questions.

Our platform

Panaseer’s CCM platform has evolved through the experience of many large global implementations. Through this vast experience, we’ve built a range of features and nuances that wouldn’t necessarily have been identified during the development of in-house tools. This lowers risk and maintenance, offers upgrades and integration, and delivers time-to-value.

Specific to Panaseer, we have further broken down criteria to help guide the decision-making process for this strategic long-term decision.



Enterprise platform

Security audits and assurance are required on an ongoing basis for security purposes. SSO, access control, data retention, encryption, and high availability all require development, investments, and ongoing costs.

ISO 27001 certified security solution with ongoing external audits to ensure enterprise security. Includes SSO integration, solution wide role-based access control, encryption at rest and in transit, data retention and governance with cloud provider, high availability and resilience.

Cost of ownership

There are ongoing costs that are frequently higher than anticipated to the point of indterminable. These may include development, maintenance and upgrades.

Cost-effective. Typically only a fraction of the overall cost of an internal development program with forecastable ongoing investments.

Regulatory frameworks

Ongoing continual commitment to mapping to changes in regulatory frameworks.

Mapping to frameworks such as CIS 8 and NIST CSF 2.0, with a team dedicated to ongoing mapping as frameworks continually evolve and issue new releases.

Development time

Can significantly increase delays and affect associated projects and initiatives. Developers often underestimate time and resources required.

Immediately available. To-date over 9 years of development have gone into Panaseer. Includes data platform with secure access to open APIs, and over 200 pre-packaged cybersecurity metrics.

Features and functionality

Typically yields a first generation tool with limited features, potential software bugs, and unanticipated logic issues

Start with a proven, third generation tool already deployed in hundreds of locations.

Unique requirements

Provides limited specialized functionality.

Highly flexible and configurable tool that will meet a wide cross section of specialized company requirements.


Heavily reliant on abilities and availability of a few development person(s).

Backed by a widely respected software company with dedicated development and support teams.


Unlikely to have combination of high-level expertise in assessment methodologies, concepts and programming skills.

Highest level of programming skills leveraged by thought leadership in assessment methodologies and concepts.

Enhancements and support

Timelines dependent on resource constraints and programmer’s ability.

New updates, features and enhancements regularly released and included with Panaseer’s ongoing support and maintenance program.

Content creation

Timelines dependent on resource constraints.

Panaseer has a dedicated team specifically focused on content and template development.


Limited. The tool is often only usable by a handful of individuals with the unique skills required.

Enhancements to Panaseer and its content is driven by growing list of clients and an ecosystem of consulting partners.

To build, or not to build, what is the answer?

Panaseer has invested over ten years in infrastructure and capabilities with many iterative improvements. Enterprise scale provides ongoing improvement and the power of a large customer community.

For long-term continuous and automated processes, the build vs buy decision requires careful consideration, and we hope this analysis will assist the decision-making process!

But buy, obviously.