Curated Insight: Over reliance on GenAI can inhibit Critical Thinking

Intro to “Curated Insights”

I regularly come across interesting and relevant content.

So, in a planned departure from the normal articles, here is the first of a monthly curated insight series.

These will highlight the most interesting article or piece of research, relevant to algorithm integrity, that I’ve come across each month.

Regular articles are certainly not going away, they will still form the core of this blog/newsletter, but one in every four or five will be in this format.

The Impact of Generative AI on Critical Thinking

Generative AI (GenAI) is seeping into knowledge work, raising questions about its effects on cognitive skills.

This study by Lee et al. explores this issue, surveying 319 knowledge workers who use GenAI in their tasks.

The study reveals that while GenAI can enhance efficiency, it can also diminish critical thinking skills over time.

Why is this important for us?

IF Over-reliance on AI reduces our independent problem-solving / critical thinking skills.

AND Generative AI is largely/solely driven by existing content.

THEN We may miss novel solutions to the challenges we face.

OR Our ability to create new value for our customers may diminish.

This could mean, for example, that our critical thinking skills weaken, and:

• We aren’t able to find creative ways to stop new types of insurance claims fraud. AI could help spot the patterns, but we may still need human brain power to creatively find mitigating actions.
• We aren’t able to create new, unique products. While AI can assist in copying existing products, it’s human ingenuity that leads to truly groundbreaking innovation.
• Importantly, we may not be able to identify potential biases in new models or processes. AI may produce models that are discriminatory, in an attempt to "improve" model performance. We want to intervene and limit the potential for bias, but this requires us to critically analyse those models. Inhibiting our critical thinking skills will then potentially reduce our ability to do this.

An interesting research paper. What are your thoughts?

Full Reference

Hao-Ping (Hank) Lee, Advait Sarkar, Lev Tankelevitch, Ian Drosos, Sean Rintel, Richard Banks, and Nicholas Wilson. 2025. "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers." In CHI Conference on Human Factors in Computing Systems (CHI '25), April 26-May 01, 2025, Yokohama, Japan. ACM, New York, NY, USA.

https://www.microsoft.com/en-us/research/uploads/prod/2025/01/lee_2025_ai_critical_thinking_survey.pdf

Disclaimer: The information in this article does not constitute legal advice. It may not be relevant to your circumstances. It was written for specific algorithmic contexts within banks and insurance companies, may not apply to other contexts, and may not be relevant to other types of organisations.