Core questions we need to be able to answer about our decisioning systems

There are (at least) 5 key sets of questions we should be able to answer quickly about our decisioning systems.

These can come up suddenly. For example, when an exec asks whether we're confident in the system.
The questions may not be posed exactly as below, but having these answers ready can help.

1. What are the manual post-model steps?

The model produces a score or a recommendation, and then something manual happens before a customer or prospective customer is affected. (Assuming it’s not fully automated, which would be a different kind of risk.)
What exactly are those steps? Can we easily depict the workflow, including any branching?

2. How much do our teams passively trust the model results?

For those manual steps, we need to know how much the humans involved rely on the score or recommendation.
Can we measure that? Do we have any QA, training, and other mitigation steps in place?

3. When did someone last look at outcomes, rather than performance?

Performance monitoring is usually about whether the model is working as intended, technically.
Whether the outcomes are reasonable, and for which customers, is a separate question.
Do our metrics reflect this, or do we measure it periodically?

4. If a customer asked why, could we answer?

Not only an explanation in model terms, but something we could directly say to a customer in a way that the customer would easily understand.

5. What has changed since it was built, or last refreshed?

The data feeds, the customer population, the product rules and others can change over time. If we don’t actively watch them, we may not notice until something breaks, or a customer complains.

There are plenty of others, but these deserve quick, consistent answers.

If it takes weeks to figure out, or we get different answers depending on who we ask, we have work to do.

Disclaimer: The info in this article is not legal advice. It may not be relevant to your circumstances. It was written for specific contexts within banks and insurers, may not apply to other contexts, and may not be relevant to other types of organisations.