Who should review your algorithmic system?

Whether it’s for reassurance, a gut feeling that something’s off, or to manage an unknown, this is a common question.

There are four main options:

• The team that builds and/or runs the system
• Internal audit / risk assurance
• Your usual external audit or consulting firm
• A specialist algorithmic reviewer

As with many other questions, it depends. And you might select different options at different points, for varying purposes. But for that immediate review, you’ll naturally want to narrow down to one.

This article briefly explains one way to think about the answer.

Internal option 1: Your team

If your team includes a distinct model risk governance, independent validation or first line oversight function, this can be a good option. The considerations under internal option 2 will be more relevant in this case.

If you don’t have such a function in your team, and your people are involved in building or running the system, then this isn’t a good option on its own. Your team should be involved in a review. They know the data (and its quirks), the workarounds, the limitations, how the system is tested, and the real state of the process flows and architecture. They’re essential for explaining what happens and providing evidence. But they can’t independently decide whether the system is acceptable. Their role is to explain and support, not to sign off on their own work.

Internal option 2: Internal audit / risk assurance

The first real decision is whether the review should be led by another internal team, separate to yours, like internal audit or risk assurance.

These teams are usually not involved in day‑to‑day delivery. They understand your policies and governance, and are generally comfortable asking difficult questions. They know how to translate findings into language that your exec and board committees use.

An internal review, led by these teams, is a good choice if:

• The team has capacity to deliver, within the timeframes you need. In many situations, the teams already have a waiting list of priorities. If the topic is already getting executive attention, you’ll have a better chance.
• The team will look at governance and documents, but the main focus will be going deeply into how the system works and what the outcomes are. In other words, they won’t stop at the high level because it’s safer or more familiar.
• Extending on the second point, the team has the capability to adequately deliver the review; for example, they do something very similar, routinely, for other algorithmic systems.

If all those apply, an internal review can work. The internal team could also bring in the right outside help to fill any gaps in capacity or capability.

 If your internal audit team hasn't reviewed an algorithmic system before, look outside.

External option 1: Your usual audit or consulting firm

Most banks and insurers already use one or two external firms for a lot of external work. It’s natural to reach for them first, and this makes sense when:

• You want a broader piece of work that covers several topics, with your algorithmic system as one component of a much larger audit or review.
• You need a report that fits neatly into existing assurance cycles and templates.
• You want an independent view that feels familiar to your board and regulators, even if the methods are more general.
• Your regulator or supervisor expects a report from a large, recognised firm. For example, when you’re faced with an enforceable undertaking or a serious systemic problem.

These firms are typically good at structure, documentation, and producing reports that align with other exec and board committee reports.

However, there are trade-offs. These usually fall into three areas:

• Cost: These can be more expensive than other options.
• Capability: Many firms have broad capability but typically send junior staff to do the work.
• Independence: If the same firm also builds or advises on systems, they may end up reviewing their own work (in whole or in part).

If you can manage all of these, or they don’t apply, this is a good choice. If the work will be done by a junior team with little or no experience, you'll get a well-formatted report that misses the point. Go for the next option instead.

External option 2: Specialist reviewer

A reviewer that specialises in algorithmic systems will focus on how your system behaves in practice, not just whether the right documents exist, or that approvals are in place.

This option makes sense when:

• You have one or a small number of systems where accuracy and/or fairness really matter. For example: underwriting (pricing and eligibility), claims (including fraud), loan origination, transaction accounts, or third-party commissions.
• There are technical nuances to understand, including proxies, messy data, logic changes over time, and intent/code mismatches. It’s not just about whether governance looks tidy on paper, but extends to tracing flows end‑to‑end, reviewing rules and code in detail, and checking that customers in similar situations aren’t treated differently because of personal characteristics.
• Your other options struggle with the “black‑box”, or stick to the edges (policies, committee minutes, model inventory) and don’t go into the decision logic itself.

It’s not always the best option. The trade-offs can be similar: they may not be independent, they can be costly, and they might suffer from capability or capacity constraints. They also don’t typically know your environment well, and may not be able to get up to speed quickly.

A specialist review won’t replace your other assurance work, but it should give you clear answers to the hard questions about data, logic, and outcomes.

Disclaimer: The info in this article is not legal advice. It may not be relevant to your circumstances. It was written for specific contexts within banks and insurers, may not apply to other contexts, and may not be relevant to other types of organisations.