Algorithm Integrity: Third Party Assurance

One question that comes up often is “How do we obtain assurance about third party products or services?”

Depending on the nature of the relationship, and what you need assurance for, this can vary widely.

This article attempts to lay out the options, considerations, and key steps to take.

Nature of the relationship

What the third party does for you, and how.

For example, they may provide one or more of these:

1. Hosting services
2. Custom applications
3. Off-the-shelf applications you configure
4. Custom models or algorithms
5. Off-the-shelf models you install and configure
6. Off-the-shelf models or algorithms accessed via APIs
7. Developers that work under your direction.

Each of these relationships presents unique challenges for ensuring algorithm integrity.

Types of third-party assurance

The assurance methods available vary in their reliability and depth. Here are the main types, from most to least reliable/usable:

1. Independent third-party assurance reports (e.g., SOC2): these provide an independent, detailed assessment of the third party's controls and processes. *
2. Certifications (e.g., ISO27001, ISO42001): not as comprehensive as SOC2 reports, and typically not relied on for external audits. They indicate a level of compliance with recognised standards.
3. Internal audits: can provide insights, but lack the true independence of external assessments.
4. Self-assessments: offer a starting point for understanding the third party's approach to algorithm integrity.
5. Vendor questionnaires: useful for gathering basic information, these are the least reliable form of "assurance".

* We prefer third-party assurance reports, because they are more reliable, comprehensive, and useful than any of the other types. But, while SOC2 and similar reports exist for information security and other areas, there isn't currently a standardised, widely accepted independent assurance report specifically tailored to algorithm integrity. This is an emerging area, and as the importance of algorithm integrity grows, we expect to see the development of such specialised frameworks in the future.

Factors to consider

With the nature of the relationship and the options in mind, consider these questions:

• How important is the service/product?
• How risky is the use of the service/product?
• What are the associated risks? [e.g., potential impact on stakeholders (customers, employees), reputational risk, risk of financial loss, etc.]
• What are the security/privacy concerns?
• What, if any, are the historical issues with the service or product?
• What are your peers – others in your industry – doing?
• What do regulators expect you to do**?
• How complex is the service/product?
• How transparent (and explainable) is the algorithm?
• How often is the algorithm updated or changed?

These factors/questions are particularly relevant in banking and insurance, where algorithms often handle sensitive customer data, make (or are relied on to make) critical decisions, and are subject to strict regulatory oversight.

**In Australia, banks and insurance companies need to consider various laws. There are also prudential standards that relate to third parties, like CPS220, CPS230, CPS234, etc. CPS230 is quite specific – certain service providers must be classified as material service providers, unless there is a justification for exclusion.

• For banks, this includes third parties that provide credit assessment, funding and liquidity management, and mortgage brokerage services.
• For insurers, this includes third parties that provide underwriting, claims management, insurance brokerage and reinsurance.

Key steps to take

Note: this article is about obtaining assurance, but there are some prerequisites – without them, an assurance approach will be difficult to execute.

1. Policy: formally document your approach to third party due diligence, risk assessment, risk mitigation, ongoing monitoring and review
2. Catalogue: keep a record of all third-party arrangements, risks and risk management practices, including frequency of reassessment and the nature of ongoing reviews
3. Regulatory: check the specifics of relevant compliance expectations (some of them, like CPS230 in Australia, can be quite extensive)
4. Procurement: before entering into a third-party arrangement, make sure that you can obtain the assurance that you need. What you need will depend on the nature of the relationship and the “factors to consider”.
5. Monitor and review: get copies of the agreed reports, analyse them, and consider what steps you may need to take if there are critical problems highlighted.

Disclaimer: The information in this article does not constitute legal advice. It may not be relevant to your circumstances. It was written for specific algorithmic contexts within banks and insurance companies, may not apply to other contexts, and may not be relevant to other types of organisations.