Algorithmic System Integrity: Explainability (Part 3) - Complicated Processes

In a previous article, we explored the concept of explainability, its importance, and four challenges. We then addressed the first challenge - complexity.

In this article, we explore some solutions to the second challenge: complicated processes.

Challenge Recap

Algorithmic systems often involve intricate workflows, combining multiple data sources – both internal and external - and transformations.

Consider how much work was involved in unpacking data flows for capital adequacy projects – Basel or Solvency – and the resulting spaghetti. These are both wide (many systems) and deep (multiple data elements/variables).

Some of these data flows cross over with credit scoring processes, pricing models and claims flows. So, the same complications apply. Data lineage was challenging enough with "simple" algorithms. And now the flows include more complex algorithms and directly impact customers.

This poses several problems:

• Trust: opaque or convoluted processes are not naturally trusted
• Compliance: Regulatory frameworks require clear explanations of how data flows, especially for decisions affecting customers
• Errors: Complicated processes make it difficult to identify where problems occur, increasing the risk of undetected issues.

Solutions

Banks and insurers can use several methods to address this problem.

Here are three sets of commonly used methods, which may look familiar if you've had to deal with capital adequacy projects or similar:

1. Picture it

Visual tools are tremendously helpful.

One such solution is a data flow diagram. It provides a visual representation of how data enters and moves through your systems, what models/algorithms are involved, and how these all work together.

Example: For an insurance claims process, a data flow diagram might show how data moves from initial claim submission through various validation steps, risk assessment algorithms, fraud triage, and then payment processing. The diagram can highlight where external data enters the flow and where automated decisions occur.

2. Document it

Documentation typically goes beyond what even detailed diagrams can capture, especially in explaining transformations, systems, and algorithms.

In documenting, consider how often the process changes in deciding what depth to go to. Too deep, in the context of frequent changes, can be difficult to maintain in the long term. Automated documentation tools can help with this.

Documentation commonly includes:

• data provenance tracking – origins and transformations
• explanations for each transformation step (in plain language, if appropriate)
• details about each system, model and algorithm involved.

Example: In banking, documentation for a credit scoring system might track how a customer's data is captured, normalised, combined with credit bureau information, and put through through the scoring model. This can help show how the fields that influence the final decision are processed at each step.

3. Simplify it

Traditional processes often contain redundant transformations and unnecessary complexity.

You may have data formatting or standardisation at various stages. Moving these to early in the process can make the flow simpler.

It is not uncommon to find transformations that reverse earlier steps as system flows evolve over time. You may be able to eliminate both, and get to the same result while making the whole process simpler.

Simplifying processes can also reduce operational costs, improve system performance and make maintenance easier.

Example: A bank's customer onboarding process might be simplified by checking for redundant identity verification steps, and reducing these. Rather than running separate validations for address, ID documents, and personal information across different systems, a unified verification service could handle all these checks while maintaining a clear, explainable process.

Next

The next article in this series will focus on the third challenge: privacy and confidentiality, exploring how to balance transparency with safeguarding sensitive data.

Disclaimer: The information in this article does not constitute legal advice. It may not be relevant to your circumstances. It was written for specific algorithmic contexts within banks and insurance companies, may not apply to other contexts, and may not be relevant to other types of organisations.