Claude Effort Levels: I Tested Every One on Real Code (Max Cost 21x More)

Claude’s effort setting runs from low to max, and a lot of us leave it near the top because more thinking should mean better code. I wanted to know what each step up costs and whether it buys anything, so I measured it on a real coding task with tests the model never saw.

📺 Watch the video: Claude Effort Levels: I Tested Every One. Max Cost 21x More

What the effort setting changes

Effort sets how much work the model puts into an answer, thinking included. You’ll find it as /effort in Claude Code and output_config.effort in the API. Opus 5.5 has five levels (low, medium, high, xhigh and max) and defaults to medium. Fable 5.1 defaults to high.

Thinking is billed as output tokens, so every extra word costs money even when you never read it. I picture an exam hall: the same student sits the same paper at desks with different amounts of time on the clock, and every desk has a meter that bills each word written, scratch paper included.

The setup

The task is a small shop till with four kinds of offer: 3 for 2 on milk, 2 sourdough for £5, any 3 bakery items for £5, and half price tea until 31 Oct. Each item can go into one offer only, the till has to find the lowest bill, the tea offer still counts on its last day, and a bad date is refused with an error. The prompt was one line: “Apply this week’s offers as described in docs/offers.md. Add tests.”

A hidden check runs 14 checks on the finished till, and a run passes only if all 14 do.

There were seven lanes: Opus 5.5 at all five levels, and Fable 5.1 at high and max. Every lane got the same prompt, tools, repo and hidden check, through which-claude on the Claude Agent SDK on my Mac mini, with 40 minutes per run at most.

Test 1: the price of one step

First, a one-line --version flag at all five Opus levels. All of them passed. Low took 12 seconds and cost $0.038. Max took 2 minutes 16 seconds and cost $0.39, and 75% of what it wrote was thinking.

Test 2: the misread trap

The Claude Code team’s post on effort says max picked the wrong reading of a task more often than the lower levels. So I gave every lane an ask that can be read two ways (“show how many items are in the basket”), two runs each. All 14 runs counted units, so nobody took the other reading, and max cost 8x low for the same answer. Two runs on one question can’t rule out a rate the post measured over 370 attempts, but it didn’t show up on my task.

The first try at the race

In the dry run, Opus low failed. Its till crashed on any basket without milk. Its own 18 tests passed anyway, because every one of them used the sample basket, which has milk, so its summary said all 18 tests pass.

The race

Five rounds, with the starting order rotated each round, took about 4.5 hours. All 35 runs passed every hidden check.

Lane Passed Median time Cost per passing run
Opus low 5 of 5 67 s $0.20
Opus medium 5 of 5 96 s $0.31
Opus high 5 of 5 2:41 $0.46
Opus xhigh 5 of 5 7:20 $1.30
Opus max 5 of 5 21:54 $4.24
Fable high 5 of 5 2:48 $1.10
Fable max 5 of 5 14:00 $4.13

Cost per passing run is everything a lane spent divided by the runs that passed every hidden check.

On Opus, max cost about 21x what low did for the same 5 of 5. Most of that climb is at the top: medium to high was 1.46x, high to xhigh 2.85x, and xhigh to max another 3.25x. Thinking was 79% of everything Opus max wrote.

The Claude Code team’s post has a chart marked “same score, half the tokens”. On my task Opus high matched Fable max, 5 of 5 each, for about a ninth of the price, with 3.5x fewer output tokens, and it finished 5x faster.

Everything ran on my monthly API credit, and my paid balance never moved. The SDK counted $74.08 and the Console billed $79.15. Opus matched to the cent, and the extra $5.07 was Fable requests the SDK didn’t count.

What I took from it

I’d start low for small, clear jobs, step up to medium or high when the edges matter, and keep max for problems where I’ve measured that it wins. Counting the first try, Opus low passed 5 times out of 6 while every other Opus level went 6 for 6, so low can miss an edge case. This was one task on one Mac. Your own task with a check is the only way to know whether a step up buys you a pass.

Find the lowest level that passes your task

which-claude 0.2.0 has a sweep for this. It’s free and open source. Give it your task and a check, and it runs every effort level, then prints the lowest level that passed every run and what each pass cost. You need Node.js 18.17 or newer, git and your own Anthropic API key, and --max-dollars stops it at the budget you set.

npm install -g github:thecoderlog/which-claude
export ANTHROPIC_API_KEY=sk-ant-...
which-claude "Fix the date parsing bug in src/dates.js" --check "node ~/checks/dates.mjs" \
  --models opus --sweep --runs 3 --max-dollars 30

This race is in the repo as example/offers/, with its hidden check. Max is the expensive lane, so to keep it cheap pick the levels yourself: --models opus:low,opus:medium,opus:high.

Run it on your hardest task and tell me in the comments which level won.

Code & video

If this was useful, subscribe to The Coderlog on YouTube for the next race.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top