TL;DR • AI and algorithm audit guidelines vary widely and may not be universally applicable. •...
Sometimes we need perfect, sometimes good enough - assess the risk
I wrote this last week and scheduled it to go out while I'm hopefully somewhere with sand between my toes.
We decided to visit an island in a country we’d never been to before. I spent hours over the last couple of weeks researching where to stay. Checking reviews, locations, activity options, and stretching the budget of course. All because my 16 year old (who agreed to join us) has very different needs to my wife and I.
But then I caught myself being a bit too thorough about details that might not matter once we get there. TBH, I didn't quite spot it like that, I just noticed that I was running out of time and had to focus on other priorities.
A risk based choice
Some things demand precision. Safety, cleanliness, the basics that could genuinely go wrong.
But food choices? Daily schedules? Sometimes we just need to go with something workable and figure it out later.
The key is understanding the stakes. When safety is at risk, we need to get close to perfect. When we're chasing the "best" Nasi Goreng, good enough beats getting lost in rabbit holes.
Algorithmic systems
This is sometimes how testing and tuning algorithms works in practice. We start with a set of parameters, then constantly tweak to make it more accurate.
If there is potential for bias or unfair treatment, we need more precision. For performance improvements or productivity gains, good enough might really be good enough, and perfectionism could just burn time and resources.
We need to know what needs precision and what is a bit less risky.
The decision point
Eventually we reach that moment where we need to dig deeper or accept what we have.
If we've assessed the real risks, and covered the non-negotiables, it's a reasonably easy choice.
Personally, by the time you read this, I'll know whether all that planning helped or we're winging it anyway.
Disclaimer: The information in this article does not constitute legal advice. It may not be relevant to your circumstances. It was written for specific algorithmic contexts within banks and insurance companies, may not apply to other contexts, and may not be relevant to other types of organisations.