Articles: algorithm integrity in FS | Risk Insights Blog

Do we trust systems that advise more than those that act?

Written by Yusuf Moolla | 02 Sep 2026

Lately we’ve become accustomed to thinking "LLM" when we hear “AI”. And who can blame us when that’s pretty much what we’re drowning in, from the media, frontier labs, colleagues, etc.

But this is a recent change. Only three or four years ago, AI had a broader meaning, when “machine learning” was still fashionable as a term.

When AI and LLM become the same thing, does that mean we trust AI more, or less?

Think about the driver assistance systems in newer cars. Lane keeping, radar cruise control, automatic emergency braking. None of this is self-driving, and it doesn’t make headlines. It does one job, and has become reliable enough that it's now standard on many models.

But, according to recent research by insurer IAG*, many drivers switch at least some of it off. IAG estimates that full fitment and consistent use could prevent around one in five road casualties.

I don’t often meet people that don’t use some sort of large language model, either for work or personal use. Does this mean that we trust the chatbots more than the narrower AI systems?

The same research found that drivers respond better to systems that warn them than to systems that take action. So maybe we’re more accepting of AI that advises, and tend to resist AI that acts?

Or perhaps it’s about the context. Parts of driver assistance are “AI-powered”, but not many automakers really market it that way. So we’re not deciding whether we trust the AI in it; we’re just annoyed at the beeping.

This is interesting for how we build and deploy our systems. Are we more accepting of a model that flags a claim for manual review? Do we tend to resist a system that can automatically approve a loan application on its own? Perhaps, and that's probably a good thing; there's a case for human review here. More importantly though, we need to know exactly which parts are deterministic rules and models, and which are stochastic (give a different answer each time).

*IAG Driving Safer Outcomes Report: https://www.iag.com.au/content/dam/corporate-iag/iag-aus/au/en/documents/adas-report-2026/iag-driving-safer-outcomes-report-august-2026.pdf

 

Disclaimer: The info in this article is not legal advice. It may not be relevant to your circumstances. It was written for specific contexts within banks and insurers, may not apply to other contexts, and may not be relevant to other types of organisations.