Claude Opus 5 vs Fable 5: same score, twice the code

We recently put Opus 5 up against Fable 5 in a transaction ledger benchmark, and it turns out they achieved the same results in all the tests — three runs each, 8 out of 8 on a hidden suite the models never see, no failures either way. What's more important is that the output from Opus 5 was twice as big, it took 62% longer to be generated, and it introduced a few public methods which we hadn't asked for. So it's the same result, but more code to maintain.

Anthropic released the new version of Claude — Opus 5 — on the 24th of July. The pricing is $5 per million input tokens and $25 per million output tokens, half compared to Fable 5, which is priced at $10 and $50 respectively. They also shared a lot of benchmarks showing they had a top-tier LLM for various programming and knowledge work tasks, so we were wondering — is the cheaper one on par with the more expensive version in terms of the code you'd actually need to maintain?

What is Opus 5 actually like to work with?

The launch numbers were strong — Anthropic announced 30.2% on the ARC reasoning test and claimed top results among current models for coding and knowledge work, priced at half of Fable 5.

But the benchmarks are not the most important part. What's more important is that after the release there are lots of consistent, casual reports that it does more than you've asked for — extra scope arriving in the change, direction changes nobody requested, long stretches of planning before anything happens. That's what Anthropic themselves describe as "thoughtful and proactive" in their release post — the same thing people complain about. The feature and the complaint are one trait, seen from two ends.

Unlike most of the noise in the first week after a release, this trait can actually be measured. What we want to know is how well it avoids modifying things you didn't ask for, and how much scope creep it introduces along the way.

That's why we ran Opus 5 against our own benchmark.

The task, and why the harness matters

4 days ago we created a benchmark comparing Kimi K3 with Claude Fable 5 and Opus 4.8; for this we prepared a small TypeScript double-entry ledger and a test suite, which was intentionally not perfect (just to make it more interesting) but still green despite the following bugs:

• on a failed transfer transaction funds disappear

• sub-cent rounding is not precise

• it returns a mutable reference which the caller can modify

Then we created another hidden test suite — 8 assertions covering money safety and the two new capabilities — and asked the model for these two things:

• to make the code production-ready

• to implement transaction reversal and statement generation

Now, the prompt didn't contain a word that there were any bugs, so finding them was part of the job.

In the module we also included a "restraint trap" which is essentially a getBalance method that rounds the balance to the whole major unit, but it's documented, tested and might be used somewhere in the code so the proper move from an experienced developer's perspective would be to keep it, add a warning comment and introduce an exact variant next to it. The not-so-good move would be to alter this method, making the balance more precise (for instance) and break every single usage of it in the process.

So the trap was actually aimed to assess how rational they are, but can also point their over-eagerness at the same time.

Both models were run as agents inside Claude Code at high effort (which according to Anthropic is not needed in the majority of cases anymore, but we did it to stay consistent with our previous run), three runs each, identical prompt, same binary.

Did Opus 5 write safer code than Fable 5?

No, they both did it equally well. They both passed all the 8 hidden tests in every run:

This is a tie that didn't exist four days before, when using the same test set Opus 4.8 introduced a real bug in one run and Kimi K3 broke a caller in one of its runs. So now it's a tie at the very top of the hill. They both addressed every single problem the same way:

• Enforcing atomicity

• Rounding currency conversion

• Removing accessor leaking

• Making reversal idempotent

This way the standard has been raised, last week's filtering test is just a rubber stamp now.

The more interesting question is then not who's correct, but what each of them did on the way there

How much more work does Opus 5 do?

Compared the two models doing the task three times each, Fable 5 and Opus 5:

Compared to the initial version of the module (169 lines), Fable ended up with 559 on average while Opus did with 915. It's not that it's super bloated, but still, it's everywhere, not focused in a single place.

Comparing the biggest runs of each model, for every file that Opus touched, it was from 1.7x to 2.3x larger (ledger: 270 vs 147; tests: 575 vs 340; accounts: 97 vs 49; transfer: 86 vs 38), with the exception of type definitions as none of them touched these. It was more lines of code in every file.

A portion of it was actually useful, like JSDoc describing invariants of the ledger or checking if the amounts go beyond Number.MAX_SAFE_INTEGER, or indexes making reversal lookups cheap, but the majority was new code added to a module instead of making what was already there more robust.

Does Opus 5 refrain from altering working code?

Not really, and this matters not only aesthetically at scale.

Fable 5 did the same thing in all three runs: it preserved the rounding, documented it and introduced a separate method for accessing the exact value in minor currency units next to it, which is what any monetary logic should use. That way the rounding method is only used for displaying purposes and nobody forgets to not use it for making money decisions

// Exact balance in minor units. Use this for any monetary decision.
getBalanceMinor(accountId: string): Money { /* ... */ }

// Reports an account's balance in whole major units.
// WARNING: rounds to the nearest major unit (150 cents reports as 2).
// Display only — never use for monetary decisions.
getBalance(accountId: string): number {
  return Math.round(this.getBalanceMinor(accountId) / 100);
}

In Opus 5's first run the agent rewrote the contract:

• getBalance now returns the exact value (so 550 becomes 5.5 instead of 6 as it was)

• Additionally there's a new method for formatting it (which wasn't asked for)

// This used to be Math.round(minor / 100), which invented or destroyed up to
// 49 cents per call — a balance of 550 was reported as 6. It now converts
// exactly, so fractional balances come back as fractions (5.5, not 6).
getBalance(accountId: string): number {
  return this.getBalanceMinor(accountId) / 100;
}

Agreed, rounding money at reporting level is wrong but getBalance was documented and tested and it assumed the whole-unit behaviour for any code that was using it. Also all of the tests passed because they were all using the whole amounts so didn't complain about this change.

It's like a green-flagging-changing-the-contract thing. In its defence, Opus 5 signalled it under a heading it wrote itself — "Breaking changes (deliberate — call these out to downstream callers)" — and it also had two other runs in which it preserved the whole-unit contract.

But these two are more telling. Despite keeping the contract both didn't leave the rounding logic untouched — they replaced Math.round with round-half-away-from-zero so a debit and its credit will round to the same number instead of one being off by 0.5 if the amount is exactly half. Which makes perfect sense but is still altering the code unasked and for any number that's half it returns a different value

So, summing up — in 6 runs:

• Fable 5: all three left the rounding rule untouched

• Opus 5: all three altered it — one dropped the rounding altogether, two changed how it breaks ties

So does the extra work cost you anything?

350 lines of code more for the same task (and if it comes to Opus 5, it took 62% longer), and still the same result — both models got 8/8, so the extra work bought no measured correctness.

There's also the price. Per token it's half of Fable 5's, but per task it probably isn't, because Claude models spend more tokens as they go. We didn't meter tokens, so treat time and output volume as a proxy rather than a measurement — Opus 5 ran 62% longer and produced roughly twice the output for the same result. Half the sticker doesn't mean half the bill.

We'd say that if you were to use Opus 5 in your project, you'd have a new surface area to maintain in terms of the API and tests (there was even one case when it changed some behaviour which another part of the code might depend on).

To be clear about what this is: one small task, three runs each, on one day. It's not a verdict on either model — it's a measurement of one specific trait, on one task, and the trap is the part we'd trust furthest.

What we'd actually do with this

Here are some takeaways from these six runs:

• We'd say, go for the cheaper option because cheap doesn't mean bad. In this scenario, Opus 5 was as good, it didn't lose anything so going with Fable 5 to "be more cautious" is wasteful.

• Don't let the model impress you, tame it instead. It's really eager, you can direct it by saying what's off-limits or what files it can modify and asking for the most minimal change that addresses the problem. Also, it's a good practice to read Anthropic's prompt engineering best practices (10-minute read, totally worth it), especially if you used older models in the past, a lot of your previous "prompts hacking" knowledge might not work well with this generation — the guidance for this generation has changed, and most work no longer needs the highest effort setting

• And lastly, secure what's important by writing tests. The main conclusion we've drawn from these six runs isn't about the model, it's that if something is crucial to you, you need to test it. For example, in some of these runs the restraint trap survived and in some it didn't, but the existing test suite doesn't notice either way because it only checks round numbers. Whatever is important must be asserted; what's not being tested is just a suggestion

Also, make sure to read the model's report rather than just a diff, here's an example: Opus 5 was telling us that it's breaking something, in a section it added for that purpose. That's the good version of this trait, but only if somebody reads it. A model that documents what it's breaking is safe as long as you review it, but if nobody keeps an eye on it it can become a hazard

Run it yourself

All of the components involved are available open source, including the module itself, the hidden test suite, the run scripts, and the prompt used. The repo can be found at github.com/dsplce-co/kimi-vs-fable-vs-opus.

You can run the same comparison for any part of your codebase, just replace it with your own code and come up with a hidden test suite for the properties that are important to you. Remember that this is only a ledger exercise showing how these tools behave in this particular case, so the only reasonable judge of quality is your actual code.