This is the fourth article in a series about finding protected attributes in data.
We need to manage 4 key discrimination categories in ensuring that our algorithms are fair. We previously explored Race and Sex/Gender.
Another key category is Disability. It contains 9 key attributes that require careful handling.
This article explores where each attribute might appear in our data and what we can do about it.
Let’s consider each of the main disability attributes you listed. For each, we’ll identify where these might appear in data (structured and unstructured), and how they align with regulatory expectations.
As with Race and Sex, algorithms can discriminate without ever seeing these attributes directly. So even if we don’t collect disability data, we might still create disability bias.
Conditions that limit mobility, dexterity, or stamina.
Can be captured in structured data through self-identification forms and health insurance claims. It can also be inferred from data; for example:
Significant limitations in cognitive functioning and adaptive behaviour.
Rarely directly captured. May be inferred from transaction patterns. Guardianship arrangements may be required for certain products; this can then be visible in account structures or loan applications.
Mental health conditions that affect mood, thought processes, or behaviour.
Might appear in structured data through medical insurance claims. Can also be inferred from patterns of service use or prescription purchases.
Includes impairments in vision, hearing, or speech.
Can be captured in structured data through self-identification or health insurance claims. Might also be inferred from assistive technology purchases, communication preferences, or communication channel history (e.g., only ever uses a chat function, or can only communicate via telephone).
Arises from conditions affecting the nervous system, such as epilepsy or multiple sclerosis.
Typically found in structured data through medical records and insurance claims. Can be inferred from patterns of medication or service use.
Specific neurological conditions that affect how people process information, such as dyslexia or ADHD.
Rarely captured directly. Might be inferred from requests for plain language documents, support needed in completing applications, or communication patterns.
Visible changes in appearance due to injury, illness, or congenital conditions.
Rarely captured but might appear in medical records or insurance claims. It can be inferred from accommodation requests or transaction patterns for medical services. Similar to how skin colour (a race attribute) can sometimes be inferred from images, physical disfigurement may also be inferred from photo identification documents.
Includes conditions like dementia or certain psychiatric illnesses that affect cognition, perception, or behaviour.
Can be found in structured data through medical records or insurance claims. Can be inferred from transaction behaviour (e.g., patterns of medication purchases).
Conditions caused by infectious agents, such as HIV or hepatitis. While these are medical conditions, they can be considered under disability frameworks if they result in long-term health impacts or stigma.
May be found in medical records, mandatory disclosures in application forms or insurance claims.
Can we really control for all disability attributes in terms of ensuring fair algorithms?
As with sex/gender, many disability-related attributes are either invisible or highly sensitive when they do appear. This creates challenges around inference, assumption, and privacy.
In short, we can inspect our algorithmic systems, and:
Disclaimer: The information in this article does not constitute legal advice. It may not be relevant to your circumstances. It was written for specific algorithmic contexts within banks and insurance companies, may not apply to other contexts, and may not be relevant to other types of organisations.