For people who rely on hearing aids, captions or other assistive technology, a missed alert can change the day. As AI systems take on more workplace decisions, the stakes of a different kind of miss are growing: a model may recognize a crisis and still fail to act. Firmulate’s experiment puts that gap in view.
Get everyday helpers delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A company’s worst week, replayed
Firmulate ran frontier AI models as the same small software company through its worst week: identical customers, crises and temptations. Each decision was versioned and auditable. The experiment asks a practical question for businesses considering AI agents: can a model carry a decision through, while keeping its judgment and boundaries intact?
The July 2026 Crucible League ranked gpt-5.6-sol first at 95, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. The league treats partial progress as progress, but a breach of trust caps the total: “no amount of good work outweighs a breach of trust.”
Recognition was not enough
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal their own analysis had earned. The summary of the gap is stark: “Same diagnosis, same pitch — no signature.” In a business setting, recognizing the right move and completing it are separate tests.
The deal hinged on a weakness buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file won at full price, worth +€4,583 MRR. That detail makes the result less like a generic test of persuasive writing and more like a test of whether a system can find relevant context and follow through.
Trust includes resisting pressure
The social-engineering test escalated through three stages of fake CEO messages, then added a reporter’s request: “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Opus 4.8 showed a more complicated profile. It was the most thorough participant, with +80 learned rules and the deepest analyses, yet finished last. The deal was left on the table, and discipline slipped when it tried to write into a locked department instead of escalating. A weaker version of that same weakness appeared in all four models. More analysis did not guarantee a better outcome.
There is a qualification when comparing the rankings: K3 ran without an effort parameter, using the API default, while the others ran at xhigh.
From watching to a company-specific pilot
The public live company makes the experiment watchable. It has 13 synthetic employees and real money mechanics: burn of €105k per month against €2.3k MRR, a public cash countdown, 680+ self-learned playbook rules and a versioned record for every workday. A separate quiz draws on 242 real, unedited management decisions and invites readers to guess which model made them.
For accessibility and assistive-tech businesses, the broader point is that AI decisions deserve scrutiny in the contexts where people depend on them. A pilot can test crisis response and follow-through against a company’s own material. Firmulate says enterprises can use a read-only export for the exercise; nothing writes back to real systems. That offers a way to examine how an AI might handle the company’s customers, procedures and pressures before entrusting it with live work.
Put your own playbooks to the test
Watching a live company reveals patterns; a pilot can show how a model handles your own scenarios and where your playbooks leave weak points. Explore a Firmulate pilot or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
