In my last effort video, Opus 5.5 on low effort passed my coding task five times out of five for 20 cents a run, and max passed it for 21 times the price. A lot of you said the task was too easy, and I agreed. So I made it harder one step at a time until low broke, and 90 runs later I know where it broke and what the fix costs.
📺 Watch the video: I Made the Task Harder Until Claude’s Low Effort Broke
The plan: a staircase
The idea was one task where low fails, then one where medium fails too, then high, then extra high. Max had to pass every step, or the step didn’t count. Every level got the same prompt, tools, repo and hidden check, through which-claude on the Claude Agent SDK on my Mac mini M4, Opus 5.5 only.
The task
It’s the small shop till from my last two videos. It reads a basket and prints a receipt. This time Claude had to write the offers from scratch, and I kept adding rules on top:
- returns that undo an offer
- fruit sold by weight, with a half penny that goes to the customer
- a rounding helper already in the code that the new code has to find and use
- a £2 voucher, where the cheapest bill sometimes means leaving an offer out
- a staff discount that comes off after everything else
Every step had a hidden check Claude never saw. On the hardest one that’s 42 baskets, each compared line by line with a reference till I wrote.
Test 1: the climb
Low didn’t break. Across ten versions of the task I had 89 finished runs, and 87 passed. Low passed 17 of 18. Its one miss was a VAT half penny. The other miss was a high run that rounded a coffee discount the wrong way (−£2.02, should be −£2.03) with its own tests green.
On the hardest written task, low finished in 1 min 52 s for $0.36. Max took 23 min 50 s and $4.88 to pass the same 42 checks.
Test 2: the order nobody wrote down
Then I went back to the code every run had already written and gave it something the spec never mentioned: big orders. A party, a café restock, 300 items in one basket, 200 rows. A real till has to print that while the customer waits, so I gave each receipt 2 seconds. Across the five tasks that never mentioned size, I had 11 runs at each level.
| Level | Runs that failed a big order |
|---|---|
| Low | 10 of 11 |
| Medium | 8 of 11 |
| High | 3 of 11 |
| Extra high | 0 of 11 |
| Max | 0 of 11 |
Most of low’s tills were still going when I stopped them at 10 seconds. Extra high and max printed the same receipts well inside the limit, most in under a tenth of a second.
The difference is in how they choose the offers. Low’s code tries every way of grouping the items. That’s fine for ten things in a basket, but every extra item multiplies the ways, so at 300 it never finishes. Max worked out the best saving for each product once and reused it.
Every one of those low tills was green everywhere else. Its own tests passed and so did every one of my hidden checks. It would have shipped looking tested.
Test 3: write it down
So I added one more page to the spec: up to 300 items in all, across up to 200 rows, receipt in under a second. Low passed all three runs, big orders included. Once the limit was on paper, low wrote the fast version too.
To compare prices I use cost per passing run: everything a level spent, divided by the runs that passed every check. On that task low came to $0.69 per passing run and max to $6.69, about ten times more for the same result.
Which level should you leave on?
| Cheapest level that held | Its cost per passing run | Max | |
|---|---|---|---|
| Size written in the spec | Low | $0.69 | $6.69 |
| Size not in the spec | High on one task, extra high on the other | $1.31 and $2.51 | $5.48 and $4.76 |
Max was never the cheapest level that held. Every time max passed, extra high passed too, and so did something cheaper.
All of it ran on promo API credit. The SDK counted $167.15, the Console bill agrees, and my paid balance never moved.
What I took from it
Low does what you write down. The higher levels pay for checking what you didn’t. So before you turn effort up, write the size down (how many items, how fast), add one test at that size, then start at low and step up only when a test fails.
I built these steps for one shop till, so they show where each level broke on these tasks only. Whether it holds on your code is the open question.
Find your own level with which-claude
which-claude 0.3.0 has a climb mode for that last step. It starts at low, moves up one effort level at a time, and stops at the first level that passes every run, so it only pays for max when every level below it failed:
which-claude "Fix the date parsing bug in src/dates.js" --check "node ~/checks/dates.mjs" \
--models opus --climb --runs 3 --max-dollars 30
The tasks from this video and their hidden checks are in example/staircase. Run it on your hardest task and tell me in the video’s comments which level held.
Code & video
- 📺 Full video: I Made the Task Harder Until Claude’s Low Effort Broke
- ⚡ The Short: youtube.com/shorts/6GRPfS_Vikk
- 💻 which-claude: github.com/thecoderlog/which-claude
- 📚 Docs: Effort, Pricing
- 🔁 Part 1: Claude Effort Levels: I Tested Every One on Real Code and its video
If this was useful, subscribe to The Coderlog on YouTube for the next test.