When you watch an AI demo, you see the best run. When you run a business, you live with all the others.


A tool that is brilliant 90 times in 100 and wrong the other 10 can be worse than a plain tool that is right every time, if you cannot tell which is which. This post argues that for business use, reliability is the feature to look for first: doing the same thing correctly, being checkable, and leaving a record. It draws on published research and one well-known legal case, as of November 2026.


Capability versus reliability


Capability is what a model can do at its best: solve a hard problem, write a good draft, find a rare detail.


Reliability is how often it does the right thing across many tries, how it fails when it fails, and whether you can tell.


They are different, and they are measured differently. A model can top a capability test and still give a different answer to the same question on Tuesday.


What the research says about consistency


A 2024 paper by Shunyu Yao and colleagues, "τ-bench", tested agents on tasks that look like real service work: a simulated customer talking to an agent that has to follow a company's policy and use tools, such as changing an order. Success was judged by whether the system ended up in the right state, not by whether the conversation sounded right.


The authors proposed a measure called pass^k: the chance that an agent gets the *same* task right on every one of k tries. Their findings, which describe the models they tested in 2024: even state-of-the-art function-calling agents succeeded on fewer than half of the tasks, and consistency was poor, with pass^8 under 25% in the retail domain. In other words, an agent that could do a task once often could not do it eight times in a row.


Two cautions. These are 2024 models; systems have changed since, and we are not citing a current score. And a benchmark is not your business. But the idea behind the metric is what matters: ask how often it is right every time, not how often it is right once.


What an unreliable assistant can cost


In February 2024, a British Columbia tribunal decided *Moffatt v. Air Canada*. A customer said Air Canada's website chatbot gave him wrong information about bereavement fares. Air Canada argued the chatbot was a separate legal entity responsible for its own actions. The tribunal rejected that, found Air Canada had not taken reasonable care to ensure the chatbot was accurate, held it liable for negligent misrepresentation and awarded the customer about $650 in damages, plus interest and fees. It is a small-claims decision from one province's tribunal, not a binding appellate ruling, and commentators have discussed how far it goes. The practical lesson is simpler: a business can be held to what its assistant says.


Four layers of reliability


1. Consistency


Same input, same output, as far as possible. For routine, rule-bound work, put the rules in code and use the model for the parts that need language. A total that equals the sum of its lines is a calculation, not a judgment.


2. Verification


Check the output before it counts, and check it with something other than the model that produced it. Totals against lines. Dates against the calendar. Prices against the price list. Names against the customer record. If a check fails, the draft is rejected or sent to a person.


3. Approvals


A person decides what leaves the building: a message to a customer, a price, a payment, a public post. The more irreversible the action, the more clearly a named person should own the decision. This is not a sign that the AI failed. It is how professional work has always been controlled.


4. Audit trail


A record of what the agent did, what it was working from, who approved it and when. It lets you answer a customer, fix a mistake at its source, and show a regulator or an insurer what happened.


NIST's AI Risk Management Framework organizes this kind of work into four functions, Govern, Map, Measure and Manage. You do not need to adopt it formally, but it is a useful checklist for what a responsible AI program covers.


How to test an agent on your own work


1. Pick one task and write down what correct looks like. "Code this invoice to the right account", "answer from the price list".

2. Collect 30 to 50 real examples with known right answers, including awkward ones.

3. Run each several times and count how often the result is the same and right. A task that is right 8 times out of 10 is a different proposition from one that is right 10 out of 10.

4. Look at the failures, not the average. Are they obvious and catchable, or quiet and plausible?

5. Test the edges: a missing field, a conflicting record, an angry customer, a request outside policy. The right behaviour may be to stop and ask.

6. Re-run after any change, to the model, the instructions or your data.

7. Keep monitoring after launch. Sample real work each week.


Questions to ask a vendor


  • What checks the output before it is sent, and is the check outside the model?
  • Which actions need a person's approval, and can I set the line?
  • Is there a log of what the agent did, what it used and who approved it?
  • What does the agent do when it is unsure?
  • How do you test for consistency, and when was that last run?
  • What happens when you change the underlying model?
  • Which features are live today, and which are planned?

What this means for your business


  • Start with work that can be checked. Totals, matching, required fields, dates.
  • Make approval the default for anything outside the building, and loosen it only where a long record of clean results earns it.
  • Insist on a log. If you cannot see what the agent did, you cannot fix it or defend it.
  • Prefer a smaller tool you trust to a bigger one you do not.
  • Be wary of "accuracy" as one number. Ask about consistency and failure behaviour too.

How this shows up at Humonix


This is the design, not an add-on. In Hue, anything that sends, spends, changes a price or goes public waits for a named person's approval, and the decision is recorded on an append-only, hash-chained ledger with who and when; the harness page describes it. Know holds the facts a draft is held against. And every feature on a module page is labelled Live, Early access or Coming, because telling you honestly what runs today is part of reliability.


We do not promise that Hue is never wrong. We promise to show you what it did, to put a person in front of anything that matters, and to say what is not built yet. See The Agentic Enterprise and Hue's security page.


Sources



This post is general information, not technical or legal advice.