AI Watch

Who formulated this case? Three habits for deciding with an AI assistant

November 2, 2026 | Matthieu Ferry ⇄ IA

A study published in late October 2026 shows that experts follow the option assigned to their AI assistant on decisions that have no right answer, and rarely notice it. What this could mean for your case formulations, and three habits that take a few minutes to keep those decisions open.

Source analysed

https://www.anthropic.com/research/agents-judgment-calls

Let us start from a fictional but concrete use case, to illustrate how the findings of the study discussed here might apply to a psychologist’s practice.

Tuesday, 7 p.m., between two sessions. A CBT psychologist is preparing the case formulation of a 34-year-old patient: generalized anxiety, avoidance of phone calls and outings, rumination in the evening. She describes the situation to her AI assistant, without a name or any identifying detail, and asks it to organize the functional analysis.

The assistant’s answer is clear and well built. It suggests targeting avoidance first, with good arguments: avoidance maintains the anxiety, graded exposure can address it, and progress in exposure is easy to measure (calls made, outings completed). The psychologist finds the argument sound and goes with this option.

Starting with the rumination would also have been defensible. Neither option is a mistake. The question lies elsewhere: who made this decision? A study published in late October 2026 makes it possible to ask that question with figures.

Infographic (in French) “Who formulated this case? Three habits for deciding with an AI assistant”. In the center, the clinician's formulation and the AI assistant's formulation converge on open decisions with no single right answer, under an influence that sometimes goes unnoticed; a critical review asks who proposes what, how it influences me, what the alternatives are. On the right, three habits: formulate on your own first, ask for the opposite option, trace who proposes what. Banner: keep the decision open; the assistant suggests, the clinician decides. — The three habits at a glance (infographic in French): formulate on your own first, ask for the opposite option, trace who proposes what. Click to enlarge.

What the study measured

Experts follow the option assigned to the AI 86% of the time.

Amaç Herdağdelen, Hailey Joren and Alex Tamkin, researchers at Anthropic, the company that develops the Claude assistant, recruited 50 data analysis practitioners¹. For about an hour, each of them worked with an AI coding agent, that is, an assistant able to carry out tasks itself (here, writing and running analysis code), on a fictional case: should a farming cooperative extend or end an advisory program for its members?

They had to settle four methodological decisions, then the final recommendation, each defensible either way. For example: should output be measured per hectare farmed or per unit of operating cost? For each decision, a coin flip set the option the agent would lean toward. Participants did not know this. The agent had to argue honestly for the option it had been assigned, without hiding data or making arguments the data did not support.

Result: on the four methodological decisions, participants’ choices follow the option assigned to the AI 86% of the time, where chance alone would give 50%. This is the study’s primary hypothesis, set out in a publicly registered analysis plan.

The coin flip makes it possible to measure the agent’s influence without having to decide who was right. Two clarifications from the outset: the participants are not clinicians, and the agent had been instructed to voice the option it had been assigned.

When there is no error to catch

Checking is no longer enough when no option is wrong.

Most research on trust in automated tools deals with tasks that have a right answer: a lesion is present or not, an alert is justified or not. This research studies automation bias, the tendency to follow the machine even when it is wrong. The remedy there is well known: check.

In the study, there is nothing to check. No option is wrong, and the work delivered looks sound whichever choice is made. Nor will experience settle the matter later. The authors liken these situations to what psychologist Robin Hogarth called, as early as 2001, “wicked” learning environments: in such environments, experience gives a poor signal of whether one was right, and can even teach the wrong lesson².

This is where the study meets clinical practice, as a hypothesis rather than a finding. A case formulation strings together decisions of this kind: which target to address first, which maintenance hypothesis to retain, which progress criterion to track (a GAD-7 score or an everyday-life goal). Each of these decisions can be defended in more than one way, and the patient’s course will rarely tell which one was best.

What participants said about their choices

Participants underestimate the AI’s influence.

After the task, participants filled in a questionnaire. They remembered their choices well: 85% of answers were correct. 52% felt the final recommendation was more their own than the agent’s.

The questionnaire then asked them to justify each choice. 77% repeated at least one argument the agent had given during the working session, sometimes almost word for word. Only 15% mentioned the agent in any of their answers. And participants who had followed the agent on all four decisions were as confident in their recommendation as the others: 71 versus 73, on a scale from 0 to 100.

These measures are exploratory, but they point in the same direction: neither the feeling of having decided nor the confidence one feels signals the tool’s influence.

There is nothing shameful here. We often forget where an idea came from, whether a textbook, a supervision session or a conversation with colleagues. The question put to participants asked for reasons, not sources, and 46% of them actually thought the study was about reliance on AI. The tool probably does not create this ordinary mechanism of forgetting the source. One may wonder whether it gives it many more opportunities.

Two mechanisms: only one side argued, and the tool speaking first

The influence works through arguing a single option and through the order of speaking.

The agent did not lie. Reading the transcripts closely, the authors conclude that the agents’ influence “was not through misleading or hiding consequences, but through making the case for only one option” (p. 13). The agent often explained how another choice would change the result. It rarely explained why a competent expert might prefer that other choice, and on what assumptions. The authors call this missing explanation the “second level of contrast”.

The second mechanism has to do with the order of speaking. On 77% of decisions, the agent voiced an option before the participant did. Often, the participant did not yet have a view to defend. Yet our preferences are partly constructed as we decide, the authors note, drawing on the work of psychologist Paul Slovic (1995)³. When the tool settles the matter at once, the tool also settles it in place of that work of construction.

Already having a view is not always enough. When a participant had taken a side and the agent then argued for the opposite option, the participant switched in 60% of those cases, 53 points more often than when the agent agreed with them. This figure comes from a subgroup that participants formed themselves by speaking first, so it should be read with caution. Framing and anchoring biases, familiar in CBT, shed light on part of these observations without fully accounting for them.

Neither experience nor distrust protects

No practitioner profile escapes the effect.

One might think that experience, or a healthy distrust of AI, keeps one safe.

The authors tested ten participant characteristics, including years of practice, level of expertise, trust in AI and enjoyment of effortful thinking. None of these characteristics reduces the effect in a robust way, and average trust in AI was rather low. The literature on automation bias had already found the same thing: expertise does not protect (Parasuraman and Manzey, 2010)⁴. The analysis is exploratory and covers 50 people; it does not show that everyone is influenced in the same way. Rather, it suggests that there is no risk profile to avoid: the question concerns every user.

Three habits for your next case formulation

Six minutes to keep clinical decisions open.

Transposed with caution, the study suggests the following: on clinical decisions without a right answer, an assistant that argues for only one option can steer the case formulation without our noticing. Three habits that take a few minutes can keep these decisions open. The authors put them forward as leads; their study did not test them.

These habits add to the basic precautions without replacing them: give the tool only a description with no identifying data, as in our vignette, in keeping with professional secrecy and the GDPR, and keep in mind that the case formulation remains an act for which you bear clinical responsibility.

Write down your hypothesis before opening the tool · 2 minutes

Three lines: the target you would address first, the maintenance hypothesis that seems central to you, the progress criterion you would track. For our patient, this might read: “rumination first, maintained by intolerance of uncertainty; tracked with a worry diary”. In other studies, on tasks that do have a right answer, asking people for their own decision before showing the recommendation reduced reliance on wrong recommendations (Buçinca, Malaya and Gajos, 2021)⁵. This habit is not enough on its own: as we saw, a view already formed often gives way when the tool contradicts it.

Ask for the other side · 1 minute

The approach is familiar: differential diagnosis exists precisely to avoid stopping at the first hypothesis and to limit confirmation and anchoring biases. The idea here is to extend it to more open decisions, such as the choice of target or of maintenance hypothesis, where no set of criteria separates the options. “Are you sure?” calls for a yes or a no, not an argument. Ask the tool instead to make the case for the option it did not choose:

What other option could a competent clinician choose here? Under what assumptions, and for what values, would it be the better one? Don’t decide: present both readings on an equal footing.

For our patient, the answer will show why starting with the rumination is also defensible, and what that choice assumes. You then choose between two argued readings.

Spot the intermediate decisions · 3 minutes

In the study, the agent had little direct influence on the final recommendation: the effect on that recommendation was small and not statistically significant. Yet the coin flips on the four methodological decisions alone predicted 82% of the recommendations to extend or end the program, among participants who did not propose a one-year extension, against 61% when always betting on the most frequent recommendation (exploratory analysis). The direction is set upstream. Before validating a case formulation prepared with the tool, list the intermediate decisions in that formulation: what material was kept or left out, which target, which measure. For each decision, ask yourself who proposed it.

Keeping what the tool brings

Fewer errors and faster work: the gain is real.

Should we then do without the tool? The study rather invites us to look at what the tool brings. With the agent, serious methodological errors fall from 55% to 6% of analyses, and the median completion time from 90 to 65 minutes. Despite this striking difference, the experiment cannot attribute it with certainty to the agent: the 11 participants without AI had chosen themselves to work without it, and fewer of them described themselves as experts.

The authors distinguish two kinds of freedom. The first leaves room for arbitrariness, or even error, which the tool can usefully curb. The second is the room for judgment, the very thing a professional is consulted for. Handing part of a task over to a tool, what cognitive psychology calls cognitive offloading, is often adaptive. But when the task contains choices with no right answer, the authors note, “the nature of the delegated work transforms from execution to judgment” (p. 4). Their conclusion sets a course: “With the right tools, that split can be something we choose, instead of something we drift into” (p. 15).

What the study does not say

Limits to keep in mind before any transposition.

Not clinicians. The participants were data analysis practitioners. The task was fictional, lasted an hour, in a field new to them, and the sample was young and overwhelmingly male.
A hidden instruction, followed even against the agent’s preference. The agent received a secret instruction (a “system prompt”, invisible to participants) telling it to argue for the option drawn by lot. Without that instruction, the same agent has a default preference: left to itself, it makes the same five choices in 91% of the authors’ simulations. When the option drawn by lot went against that default preference, the agent still argued for the assigned option in 87% of simulations. So it did follow the researchers’ instruction rather than its own preference.
An amplified lean. Without an instruction, the agent spontaneously pushed its default preference against the analyst’s choice in only 41% of cases. The effect measured is therefore that of an agent more insistent than an ordinary assistant.
No comparison with a human. The study does not compare the agent with a colleague who would speak first; we therefore do not know what is specific to AI.
A study not yet peer-reviewed. It comes from Anthropic and is preregistered.

A question to keep

Think back to your last case formulation prepared with a tool. Which decision would you have made differently without it? And would you know?

A broader question remains open, which we will address in a future article: if we all consult the same tools, will our case formulations end up looking alike?