Justice (AI ethics)
Matthieu Ferry ⇄ IAIn brief: Distributing the benefits and risks of AI systems fairly, preventing discrimination inherited from data, and preserving solidarity. In mental health, its test question: does the tool broaden access to care — or does it install a two-tier medicine, human for some, algorithmic for others?
Frame of reference
This principle inherits the distributive tradition (Rawlsian): justice as the fair allocation of goods among groups. This ontology of distribution leaves in shadow two questions that other traditions raise: who decides, and what is extracted. → See other perspectives
Why this concept matters
The central argument of AI proponents in mental health is a justice argument: democratize access to care in the face of the shortage of clinicians. This principle provides the tools to scrutinize this argument instead of accepting or rejecting it wholesale.
Floridi and Cowls (2019) give the principle a tripartite form: use AI to correct existing inequalities, prevent it from worsening those its data inherit, preserve solidarity in societies that algorithmic infrastructures transform.
For the clinician, justice is not an abstract concern for regulators: it plays out in your office, each time a tool validated on English-speaking students meets a patient who does not resemble them.
What the principle requires
1. Fairness of access — and of quality
Vilaza and McCashin (2021) identify the dilemma: chatbots can broaden access to care in medical deserts — or entrench a two-tier medicine. Justice requires either comparable quality, or an explicitly complementary positioning, never a substitutive one.
For the clinician:
The question is not only “who has access to the tool?” but “who has access only to the tool?“.
2. Discrimination inherited from data
Chatbots are massively trained on Western English-language data: unsuitable or stereotyped responses for linguistic and cultural minorities, cultural idioms interpreted as symptoms. Justice requires representative data, differential audits by sub-population, post-deployment correction.
For the clinician:
For your patients far from the majority profile (language, culture, neurodivergence), the “validated” tool may not be validated for them. Ask for disaggregated performance — its absence is already an answer.
3. Procedural justice: who participates?
Beyond distribution (who gets what), procedural justice interrogates the process: who designs, who defines the equity metrics, who evaluates the harms? Costanza-Chock (2020) radicalizes it: authentic justice requires co-design with marginalized communities, not the top-down application of principles by distant experts.
For the clinician:
Did users and clinicians take part in the design — upstream, not in beta testing? It is a quality criterion as discriminating as clinical validation.
4. The impossibility of a purely technical fairness
A fundamental mathematical result (Chouldechova, 2017): one cannot simultaneously satisfy all formal definitions of fairness once groups differ. Choosing a fairness metric is therefore a political trade-off, not a technical setting — and a system “certified fair” has simply chosen its definition.
For the clinician:
When a developer claims its system is “unbiased,” the right question is: according to which definition, chosen by whom, at the expense of which other?
Illustrative case
A regional health agency deploys a front-line chatbot in a territory where the wait to access a psychologist exceeds eight months. The argument is explicitly a justice argument: better than a waiting list.
Scrutiny through the principle: the extension of access is real — people with no resource at all obtain one. But three blind spots remain: the chatbot, trained in standard French, responds poorly to non-native speakers who are nevertheless overrepresented in the territory; no funding simultaneously reinforces the human offer, turning the provisional into the permanent; and the usage data, massively produced by the most precarious inhabitants, feeds a commercial product that will be sold elsewhere.
The deployment is neither wholesale just nor unjust: it corrects one access inequality while installing two others (differential quality, extraction). This is exactly the work the principle demands — scrutinize the three dimensions, not decide by slogan.
In practice for the clinician
- Disaggregate every claim of efficacy: validated on whom? Your patients outside the majority profile deserve the question every time.
- Spot creeping substitution: when a tool deployed as a complement becomes the only offer for precarious populations, two-tier medicine is installed — name it in the bodies where you sit.
- Value co-design: between two comparable tools, the one designed with users and clinicians offers better guarantees than the one designed for them.
- Do not leave fairness to the engineers: the choice of a fairness metric is a choice of values — clinicians are fully legitimate to take part in it.
What this concept does not say
Interpretive caveats:
- Justice is not a technical property of the system: the impossibility theorem shows it requires explicit political trade-offs
- The absence of a one-off bias is not enough: the structuring effect of infrastructures (which durably shape the care offer) exceeds local non-discrimination
- Downstream consultation is not co-design: having users validate what is already designed is not procedural justice
- Justice-washing exists: displaying parity indicators without touching structural conditions (extractive business model, closed governance) is the distributive equivalent of ethics-washing
- An acknowledged internal tension: extending access fast (distributive justice) and co-designing slowly (procedural justice) pull in opposite directions — the trade-off must be explicit
Other perspectives
Distributive justice thinks in terms of allocation: goods to distribute among groups. This ontology of distribution presupposes that what circulates is a good — and that the question stops at its distribution.
Capabilities: distributing is not enough
For the capabilities approach (Sen, Nussbaum), justice is measured not by the resources distributed but by the real capacities to convert them into functionings. A chatbot equally “accessible” to all does not create equal capabilities: digital literacy, language, trust in institutions and bandwidth condition its real use.
For the clinician: Your patient’s formal access to a tool says nothing about their effective capacity to derive a benefit from it — it is that capacity that must be evaluated.
Data colonialism: the injustice is in the extraction
For the critique of data colonialism (Couldry & Mejias), the primary injustice is not in the poor distribution of benefits but in the constitutive extractive relationship: users’ confidences — the most intimate raw material there is — become a commercial asset. Redistributing the benefits of an extraction does not make it just.
For the clinician: Beyond “who benefits from it?”, ask “what is extracted, from whom, and for the benefit of which model?”.
Ethics of care: the justice of the encounter
For Tronto, justice in a care context plays out not only in large allocations but in the concrete quality of each encounter: being heard in one’s language, in one’s cultural idiom, in one’s singularity. A perfect distribution of inadequate encounters remains an injustice — simply better distributed.
For the clinician: Equality of treatment is not uniformity of treatment — true of your consultations, true of tools. See the resource Ethics of Care.
These perspectives shift the question without cancelling it: fair distribution remains necessary — but it says neither what is worth distributing (capabilities), nor what its production extracts (colonialism), nor what happens in each encounter (care).
Further reading
The ↩ arrows link back to the passage of the resource that cites the reference.
The tripartite formulation: Floridi, L. & Cowls, J. (2019). A Unified Framework of Five Principles for AI in Society. Harvard Data Science Review, 1(1). DOI ↩
The access dilemma in mental health: Vilaza, G. N. & McCashin, D. (2021). Is the Automation of Digital Mental Health Ethical? Applying an Ethical Framework to Chatbots for Cognitive Behaviour Therapy. Frontiers in Digital Health, 3. DOI ↩
The radical procedural critique: Costanza-Chock, S. (2020). Design Justice: Community-Led Practices to Build the Worlds We Need. MIT Press. DOI ↩
The impossibility theorem: Chouldechova, A. (2017). Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data, 5(2), 153-163. DOI ↩
All concepts
Last updated: July 2026