Skip to content

Structuring the Review Objective: 10 Key Aspects of Algorithm Integrity

Picture of Yusuf Moolla
TL;DR
• This article proposes breaking down algorithm integrity considerations into 10 key aspects, including accuracy, fairness, security, and compliance.
• Regularly assess these aspects, as algorithms and their use change.
• Effective review scoping takes into account your risks, context and needs.

 

We previously explored the challenges of placing undue reliance on audits and reviews.

One potential solution that we outlined is a clear scope, particularly regarding the review objective.

In this article, we will focus on algorithm integrity as the broad review objective.

Podcast icon Listen to the audio (human) version of this article - Episode 4 of Algorithm Integrity Matters

While it’s easy to assert that an algorithm has integrity, confirming this assertion is a bit more complex. To help simplify this, this article breaks it down into a set of key areas to consider.

Mixing and matching the various aspects could help shape an ongoing review program that works for your specific context, objective and needs. 

[Note: the term "review" in this article includes "audits". This is explained here.]

Ten key aspects of algorithm integrity

These are not ranked, and do not appear in any particular order.

Relevance and importance will vary based on your context.

The goal here is to provide clarity on what 'algorithm integrity' can entail as you plan your review program.

It is important to note that this list:

  • is not exhaustive
  • was designed with banking and insurance algorithms in mind
  • focuses on "traditional" algorithms - e.g., traditional machine learning models (like linear regression or decision trees) and rules-based systems
  • is not tailored for emerging AI systems/models- e.g., generative AI, LLMs.
"Emerging AI systems" are often complex, opaque and present a unique set of challenges and considerations. While the items below may apply to them, other aspects may be more important due to the nature of these systems and the context in which they're used. Details will be different - e.g., determining the accuracy and robustness of an LLM is not the same as determining the accuracy and robustness of a "traditional" model.

With the context established, here are the 10 key aspects:

  1. Accuracy and robustness
  2. Alignment with objectives
  3. Fairness (incl. impact assessments)
  4. Transparency and explainability
  5. Security
  6. Privacy
  7. Governance, Accountability and Auditability
  8. Risk Management
  9. Ethics and Training
  10. Compliance.

The table below expands on each aspect, broadly.

# Aspect / Category Description
1 Accuracy and Robustness
  • Outputs / results are correct and align with expected or true values.
  • The algorithm performs consistently and dependably under various conditions, including edge cases and unexpected inputs.
  • Feedback loops and performance metrics to ensure ongoing accuracy, robustness and reliability.
  • Accuracy is used here as a blanket term that includes completeness and validity where relevant.
2 Alignment with Objectives
  • Serves its intended purpose.
  • Aligns with organisational goals.
3 Fairness (including impact assessments)
  • Biases are mitigated, especially as they relate to protected attributes. The design ensures equitable treatment.
  • Data is not used unless specifically needed, and does not introduce bias.
  • External data is not used unless specifically needed, and does not introduce bias (similar to the above point, important enough to repeat).
  • The potential impacts - on individuals, groups, and society - are assessed, using representative data.
4 Transparency and Explainability
  • Understandable and interpretable processes and decisions.
  • Incl. clarity about data sources, quality, and decision-making logic.
5 Security
  • Protect the algorithm, including design, data, and infrastructure from unauthorised access, manipulation, or exploitation.
  • Ensure that authorised access is provisioned (more in this article).
  • Incident response and recovery (e.g., fallback processes).
6 Privacy
  • Safeguard personal data.
  • Ensure compliance with privacy regulations.
  • Maintain data confidentiality throughout processing and storage.
7 Governance, Accountability and Auditability
  • Establish clear lines of responsibility for the algorithmic system and resultant decisions. This includes ethical oversight, data governance, oversight of algorithm development, deployment, and operation, etc.
  • Document the algorithmic system, including risks, controls, testing, processes, decisions and performance over time.
  • Enable independent verification and review through documentation and audit trails (systematic record-keeping).
8 Risk Management
  • Identify and manage algorithmic, ethical, third party (vendor) and other risks throughout the system lifecycle.
  • Monitor and adjust risk mitigation strategies (ongoing).
9 Ethics and Training
  • Establish ethical oversight and frameworks for decision-making.
  • Identify, assess, and manage ethical risks.
  • Document relevant codes of conduct, policies, and guidelines.
  • Conduct ethics training (including bias awareness).
10 Compliance
  • Adhere to relevant laws and regulations.
  • Adhere to contractual obligations.
  • (possibly) Consider industry standards.
  • Incl. maintaining documentation of compliance efforts and staying updated on regulatory changes.

Is this not too much for one review?

If this is the first review, or your overall objective is something other than certification or compliance.

And if it is not practical to cover all 10 in one go, you may be able to split it into bite-sized chunks.

Instead of including all ten aspects in every review, one approach is to separate the items into three sets:

  1. Areas to cover each time (i.e. critical aspects)
  2. Areas that could be rotated (i.e. covered in every alternate review - e.g., low risk, infrequent change)
  3. Areas that rely on other review (e.g., security and privacy may have broader, dedicated reviews). Note: reliance requires some work - you typically want to understand the specifics. Consider your unique risks in the area (e.g., what privacy means for your model), and cross-reference this to what has already been covered.

This approach allows for thoroughness, over time, without overwhelming your team.

The specific frequency of your reviews may vary. This can depend on a range of factors, e.g., model changes, etc.

Scoping is important, and it does not have to be difficult

This list should, hopefully, help with your scoping exercise.

Consider how these aspects relate to your context, objective, and needs. 

Importantly, write it all down, to ensure clarity.

It is good practice when commissioning a review. You won't regret spending the time and effort.

 

Disclaimer: the information in this article does not constitute legal advice. It may not be relevant to your circumstances. It is certainly not appropriate for high-risk use cases (as outlined, for example, in The Artificial Intelligence Act - Regulation (EU) 2024/1689, a.k.a. the EU AI Act). It was written for consideration in certain algorithmic contexts within banks and insurance companies, may not apply to other contexts, and may not be relevant to other types of organisations.

 

 

Weekly Articles in your Inbox or via LinkedIn   Fill in your details for weekly emails, or subscribe to the newsletter on LinkedIn.  

, ,