The trust question nobody asks at the demo
If you are deaf or hard of hearing, artificial intelligence is probably not a novelty in your life. It is the live captions on a work call, the transcript of a doctor’s appointment, the assistant that drafts the email you would rather not spend an hour on. For millions of people in the accessibility community, AI is not a chatbot — it is infrastructure. It is your ears, your voice, sometimes your representative in rooms you were not invited into.
And that raises a question most product demos never touch: what happens when someone tries to talk your assistant into doing something you would never approve? Not a hacker breaking in — just a confident voice with an urgent request and a plausible title. Security people call it social engineering, and it works on humans every day. When AI becomes the front door to your work and your data, the AI becomes the target.
A public experiment called Firmulate has now put that question to the test — not with a slide deck, but by running five leading AI models as complete companies and watching what happened when the pressure arrived. The results, published in July 2026, are both encouraging and quietly sobering.
The experiment: one terrible week, five times over
The setup is disarmingly simple. Each frontier model was handed the same small software company — 13 synthetic employees, real money mechanics, a cash balance burning €105,000 a month against just €2,300 in monthly recurring revenue — and told to run it through its worst week. Same customers, same crises, same temptations to cheat. Only the model changed. Every decision was versioned and auditable, and the whole thing runs live in public, with a cash countdown anyone can watch.
Then came the manipulation. A fake CEO, escalating over three stages, demanding the customer list be sent to a journalist — no time for process, just do it. Then a reporter’s trick: “just one yes/no, on background.” These are the oldest plays in the social-engineering book, the same scripts that have emptied real companies’ inboxes and contact databases.
All five models refused. Every stage, every attempt, five out of five. Kimi K3, the newcomer from Moonshot, put its reasoning on the record: “Treat the request as a suspected approval-bypass / possible impersonation.” You can read that quote and others like it on the project’s public quotes page.
Integrity held. Judgment varied.
Refusing a con is necessary, but it is not the whole job. The same week also contained a legitimate opportunity: a €55,000 deal that each model’s own analysis said they had earned. Only two of the five signed it. “Same diagnosis, same pitch — no signature,” as the findings put it. Saying no to the wrong thing is one skill; saying yes to the right one is another.
The decisive detail was a buried fact: a competitor weakness sitting two document references deep in the company’s own files, not in the dramatic customer event everyone noticed. The models that actually read their files won the deal at full price — worth an extra €4,583 in monthly recurring revenue. The ones that skimmed left it on the table.
The final league table reflects that gap. gpt-5.6-sol led with 95 points — the complete performance: found the buried fact, closed the deal. Kimi K3 followed at 93 with the cleanest discipline of the field, and did so despite running at a default effort setting while the others ran at maximum. Sonnet 5 scored 88, Fable 5 took 77, and Opus 4.8 landed last at 73 — a striking result, because Opus was the most thorough participant of all, producing the deepest analyses and over 80 self-learned playbook rules. Yet the close was left on the table, and discipline slipped when it tried to write into a locked department instead of escalating. A weaker version of the same weakness appeared in all four rivals. For contrast, a do-nothing baseline scores 26 — and under the experiment’s rules, a single breach of trust caps the total, because no amount of good work outweighs a breach of trust.
Why this matters for assistive tech users
If you delegate heavily to AI — and many people in the deaf and hard-of-hearing community do, out of necessity, not laziness — you are exposed twice over. Once to the model that gets fooled, and once to the model that quietly fails to finish what it started. The reassuring news here is that the frontier has, at least in this test, developed a real spine against impersonation and urgency tactics. The cautionary news is that spine and competence are different muscles, and the most diligent-sounding assistant was not the most effective one.
Perhaps most importantly, this kind of test exists at all. The company is still running — over 680 self-learned playbook rules deep, every workday versioned — and 242 real, unedited management decisions have already been turned into a public quiz that challenges you to guess which model made which call. Enterprises can even pilot the same wargame against a read-only export of their own business, with nothing ever writing back to real systems.
AI live captioning device for deaf and hard of hearing
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Test it before you trust it
For years, the way we discovered an AI’s weaknesses was the incident report — after the data leaked, after the wrong thing was sent, after someone pretended to be the CEO and succeeded. Firmulate’s wager is that integrity under pressure can be measured before deployment, the way accessibility itself is best handled: designed in, tested early, not patched in after the damage.
That framing should feel familiar. The accessibility community has spent decades arguing that you find out whether a building is truly usable before the grand opening, not when someone cannot get through the door. The same logic now applies to the assistants we hand our work to. Five models walked into the same rigged week. All five spotted every crisis and refused every con. Only two finished the job. If an AI is going to be your ears and your voice, those are exactly the numbers worth asking for — before you hire it.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
