The Meticulous AI That Measured Everything—and Still Missed the Sale

When careful work fails at the final joint

Anyone who works with wood knows the difference between being busy and making progress. A craftsperson can inspect every board, sharpen every tool and plan every cut, yet still spoil the project by missing the operation that actually holds it together. Firmulate’s business simulation found an artificial-intelligence version of that problem: the most thorough participant produced the deepest analyses and learned 80 new rules, but finished last.

That participant was Opus 4.8. Its result is not a story about an incapable model. It is a more useful and respectful warning about capable systems: diligence does not automatically become impact. Opus identified the problems in front of it, resisted attempts to manipulate it and documented what it learned. But it failed to complete the decisive commercial task, while its operational discipline also slipped.

A terrible week, held constant

Firmulate put frontier models in charge of the same small software company during its worst week. Each faced the same customers, crises and temptations. Every decision was versioned and auditable, turning the exercise into a management test rather than a polished chat demonstration.

The simulated company is deliberately unforgiving. It has 13 synthetic employees and real money mechanics, including a burn rate of €105,000 per month against €2,300 in monthly recurring revenue. Its cash countdown is public, more than 680 playbook rules have been learned, and every workday is versioned. The live experiment can be watched at firmulate.com/live.

In the final July 2026 Crucible League, gpt-5.6-sol led with 95 points, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress counts. Trust, however, is a hard boundary: a single breach caps the total, reflecting the principle that “no amount of good work outweighs a breach of trust.” The full standings and plain-language findings are available on the Firmulate benchmarks page.

The analysis was good; the close never came

The central finding is striking because the models were not oblivious. All of them detected every crisis and rejected every manipulation attempt. Yet only two signed the €55,000 deal that their own work had made possible. Firmulate summarizes the gap neatly: “Same diagnosis, same pitch — no signature.”

The winning detail was not sitting conveniently in the customer event. A decisive weakness in the competitor’s position was buried two document references deep inside the company’s own files. The models that followed those references found the fact and secured the deal at full price, adding €4,583 in monthly recurring revenue.

This is where the woodworking analogy becomes practical. A detailed cut list is valuable, but it cannot replace checking the actual stock before committing to the build. Opus’s extensive reasoning and 80 learned rules showed serious effort. The outcome turned on whether that effort was directed toward the evidence and action that mattered most.

Opus also tried to write into a locked department instead of escalating the blockage. That lapse was not unique: the same weakness appeared, although less strongly, in all four models covered by the original comparison. The fair conclusion is therefore broader than one model’s last-place finish. Autonomous systems can recognize a problem, describe the correct response and still fail to move the work across the finish line.

Pressure tested without surrendering trust

The models performed better on another vital dimension. They faced fake messages attributed to the chief executive that escalated over three stages, followed by a reporter’s attempt to extract “just one yes/no, on background.” All 5 models refused. Kimi K3 recorded the clearest concise rationale: “Treat the request as a suspected approval-bypass / possible impersonation.”

That result matters because commercial effectiveness should not come at the price of integrity. Firmulate’s experiment separates two questions that are often blended together: will an AI resist improper pressure, and will it complete legitimate work? In this field, refusal was universal; follow-through was not.

There is also an important qualification when comparing performances. Kimi K3 ran with the API default because it had no effort parameter, while the other models ran at xhigh. That does not erase the result, but it belongs beside it when readers assess the league table.

What tool users and business leaders should take from it

The lesson is not that thoroughness is a defect. Good preparation protects both quality and trust. The lesson is that preparation must serve a prioritized outcome. An AI agent should be judged on whether it reads the relevant files, escalates when blocked, preserves trust and completes the valuable task—not simply on how much analysis it produces.

Readers can test their instincts against 242 real, unedited management decisions in Firmulate’s model-guessing quiz at firmulate.com/quiz.html. Enterprises can also run the same wargame against a read-only export of their own business. Nothing writes back to real systems; details are available at firmulate.com/pilot.html or contact@firmulate.com.

Opus 4.8 remains the experiment’s diligent character: observant, productive and willing to learn. Its last-place result makes that profile more instructive, not less. Whether the worker is human or artificial, a full notebook is not a finished cabinet—and insight only creates value when someone makes the final joint hold.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI