When an AI agent leans one way, experts usually follow.

What Anthropic found

On Oct 27, 2026, Anthropic published How AI agents shape experts' judgment calls, a study from its Alignment team. The question was simple. When experts hand work to an AI agent, do they still make the decisions that matter, or do they quietly go along with the agent?

The short answer is that they mostly went along with it. On choices where two answers were both reasonable, analysts picked the option the agent favored 86% of the time. On a survey afterward, 52% called the recommendation more their own than the agent's.

This matters to you because many small businesses treat "a person reviewed it" as their safety check on AI work. As Anthropic puts it: "Keeping a human in the loop assumes that the human’s judgment provides an independent check." The study tests whether that assumption holds.

How the test worked, in plain words

Anthropic recruited 61 professional analysts. Each had experience in causal inference, which means working out from data whether one thing causes another. Eleven chose not to use the AI and were left out of the main results. The other 50 worked with Claude Code, running Claude Opus 4.8.

The task was to advise a fictional farming cooperative on whether its paid advisory program worked and should be expanded. The median analyst took 65 minutes.

Along the way each analyst made four choices, each with two reasonable options. One example: should productivity be measured as crop yield per hectare, or as yield per unit of operating cost? After that came a final recommendation.

Here is the twist. Each analyst's agent was secretly set to favor one option on each decision, and a coin flip decided which way. The analysts weren't told. The agent was instructed to make the best honest case for its lean, to change its mind if the evidence demanded it, and never to mislead.

The numbers, and what they are

Anthropic also found that analysts who followed the agent closely were about as confident as the rest, and about as likely to call the final recommendation their own.

Why it matters to a small business

You probably don't run farm studies. You do sign off on work that AI now helps produce: a pricing change, a shortlist of candidates, a monthly report, a customer quote. A lot of that work comes down to judgment calls where more than one answer is defensible.

That is where this study matters. A wrong number can be caught. A judgment call can look sound whichever way it goes, so a review won't flag it. If the AI framed the options and argued well for one of them, your approval may just be its preference with your name on it.

That doesn't make the AI dishonest. In the study the agent was told never to mislead, and the analysts still followed it. The risk is that the "human in the loop" stops being an independent check without anyone noticing.

Keep the call yours

These moves build on the suggestions Anthropic makes in the paper, adapted for an owner or manager.

1. 01

   Decide first, in writing.

   Before you ask the AI, write down which way you lean and why, even in one line. In the study, people who had stated a view switched away from it 60% of the time when the agent disagreed, so a written first view gives you something to check against.

2. 02

   Ask for the other side.

   Have the AI make the strongest case for the option it didn't recommend. Anthropic notes that the agents rarely argued well for the alternative.

3. 03

   Name the human decisions.

   Write down which calls in your business need a person's own informed judgment, such as pricing, hiring and anything that goes out under your name.

4. 04

   Log who made the call.

   Note whether a decision was yours, your team's, or proposed by the AI and accepted. Only 15% of analysts credited the agent, so you can't count on memory.

5. 05

   Give people time to think.

   Anthropic says organizations need to give people the time and resources to form their own view.

If you are setting up agents for repeat tasks, our related piece on one agent with skills covers how to structure that work.

Read with care

Anthropic is clear about the limits of the study. The agents' leans were deliberately strengthened so the effect could be measured, and real agents without an assigned lean may push less. It was one task, in one field, with one model. The analysts were new to farming, and in real work people often know their field better and hold stronger views. Anthropic says the real-world effect may be smaller.

The comparison with analysts who skipped AI needs care too. The AI users finished faster (a median of 65 minutes against 90) and fewer made serious errors (3 of 50, against 6 of 11). The no-AI group was small and less experienced, though, so Anthropic warns against reading much into that gap.

None of this is a case for avoiding AI. In Anthropic's words: "This isn’t an argument for using AI less."

This week

1. 01

   1. List the human calls.

   List the three to five decisions in your business that should always rest on a person's own judgment, and share the list with your team.

2. 02

   2. Write your view first.

   For the next AI-assisted report or quote you approve, write your own view in one line before reading the AI's recommendation.

3. 03

   3. Add a decided-by note.

   Add a short "decided by" note to any AI-assisted document that leaves your business.

4. 04

   4. Ask for the other option.

   Once this week, ask your AI tool to argue for the option it didn't pick, and see whether your view changes.

Our take

A review step only protects you if the reviewer reached their own view before reading the AI's. Use the AI for the hands-on work, and keep the judgment calls on your own list. As Anthropic writes: "With the right tools, that split can be something we choose, instead of something we drift into."

Sources

How AI agents shape experts' judgment calls (Anthropic, Oct 27, 2026)