Two numbers dominate every conversation about AI agents right now. One says 95% of enterprise AI pilots deliver nothing. The other says over 40% of agentic projects will be cancelled by the end of 2027. Both are real, both come from serious sources, and both are routinely used to argue something they do not actually support.
What follows is what AI agents can do reliably today, what they cannot, the arithmetic that explains why long agent workflows break, and how to scope one so it does not become the project that gets cancelled. Written for someone running a small business, not an enterprise transformation office. Checked 20 August 2026.
First, what counts as an agent
An assistant answers when asked. An automation follows a fixed path you defined. An agent decides the path itself: it takes a goal, chooses steps, uses tools, and adapts when a step fails.
That distinction matters commercially because Gartner has named the practice of blurring it — agent washing, meaning rebranding existing chatbots, assistants and robotic process automation as agentic without the underlying capability. Gartner estimated that of the thousands of vendors claiming agentic AI, only around 130 were building the real thing.
The test to apply on a sales call is simple: show me it recovering from a step that failed. If the demo only ever runs the happy path, you are looking at automation with a new label. Which may be fine — automation is cheaper and more predictable — but you should not pay agent prices for it.
What AI agents do reliably in 2026
| Task shape | Why it works |
|---|---|
| Research and compile — gather sources, summarise, produce a briefing | Errors are visible to the reader and cost nothing to correct |
| Draft-and-route — write a reply, ticket or summary for a human to approve | A person sits between the agent and the consequence |
| Sort, tag and triage inbound messages or leads | Mistakes are cheap and individually reversible |
| Structured extraction from documents into a fixed format | Output is checkable against the source in seconds |
| Code changes inside a repository with tests | The test suite is an automatic verifier |
Look at the right-hand column. Every reliable use has the same property: a cheap way to detect a wrong answer. That is the actual selection criterion, and it predicts success far better than task complexity does.
What they still get wrong
The most useful public evidence is TheAgentCompany, a benchmark published at NeurIPS 2025 that drops agents into a simulated software company and asks them to do ordinary jobs — engineering, project management, finance, admin, HR — using a browser, a code environment and simulated colleagues. Even with partial credit awarded for partially completed work, the best-performing agents finished only around 30% of tasks. Administrative, data-science and finance tasks scored lowest, with several models completing none.
Meanwhile METR’s tracking of task length finds that the longest software task frontier agents complete half the time was roughly two hours’ worth of human work as of mid-2026, with that horizon doubling every several months. Read those two findings together and you get the honest picture: agents are increasingly capable at bounded tasks and still poor at open-ended office work that requires knowing what “done” means.
- Anything where the goal is implicit. “Handle this client” is not a task specification. Humans fill gaps from context; agents invent plausible ones.
- Irreversible actions. Sending, paying, deleting, publishing. Not because agents fail often, but because the failures cannot be undone.
- Tasks with no verifier. If checking the work costs as much as doing it, the agent has saved you nothing.
- Long chains without checkpoints. See the arithmetic below.
- Anything reached through a logged-in session on the open web. Agentic browsing carries documented prompt-injection risk, covered in our Perplexity review.
The arithmetic nobody puts in the pitch deck
This is the single most useful thing to understand before you buy anything agentic. Reliability multiplies across steps. An agent that is right 95% of the time at each individual step is not 95% reliable over a ten-step workflow.
| Per-step accuracy | 5 steps | 10 steps | 20 steps |
|---|---|---|---|
| 90% | 59% | 35% | 12% |
| 95% | 77% | 60% | 36% |
| 99% | 95% | 90% | 82% |
That is just multiplication, not a benchmark. But it explains almost every disappointing agent pilot: the demo was three steps and worked, the real workflow was fifteen and did not. It also explains why the fix is architectural rather than a better model — break long workflows into short ones with a human or a deterministic check between them, and the compounding stops.
Independent evaluation work has found something related and worse: agent performance measured across repeated runs is substantially lower than on a single run, and the surrounding scaffolding can move accuracy more than the choice of model does. Ask any vendor for results across repeated runs of the same task, not a single successful demo.
What those two headline numbers actually say
The 95%. This comes from MIT’s NANDA initiative, The GenAI Divide: State of AI in Business 2025, based on interviews with business leaders, an employee survey and analysis of around 300 deployments. The finding is that 95% delivered no measurable P&L impact. That is not the same as “failed.” A large part of it is that organisations never established a baseline before deploying, so there was nothing to measure against. The study has also been criticised for a small interview base, a narrow success definition and the absence of peer review. The direction is credible; the precision is not. Do not let anyone quote it at you as proof that AI does not work — and do not repeat it yourself for the same reason.
The 40%. Gartner’s forecast attributes cancellations to escalating costs, unclear business value and inadequate risk controls. Read that list again: none of the three is “the model was not good enough.” These are scoping and governance failures. The implication for a small business is encouraging rather than discouraging — the failure modes are ones you control.
How to scope an agent project that survives
- Measure the task first. How many times a week, how many minutes each, who does it now, what a mistake costs. If you cannot answer those four, you will not be able to tell whether the agent helped — which is precisely how projects end up in the 95%.
- Pick a task with a cheap verifier. Ideally the person receiving the output can spot a mistake in seconds.
- Keep the chain under five steps. Insert a human approval or a hard rule between chains rather than extending one.
- Give it read access before write access. Run it in draft-only mode for two weeks and count how often you would have needed to intervene.
- Set a kill switch and an owner. One named person who can turn it off, and knows what happens to the work in flight when they do.
- Compare against plain automation. If the path never varies, a rules-based workflow is cheaper, faster and does not hallucinate. Much of what is sold as agentic does not need to be — the honest comparison is in automating work with AI tools.
The costs that do not appear on the pricing page
- Token cost scales with attempts, not tasks. An agent that retries a failed step three times bills for four. Budget on worst-case runs, not the demo.
- Review time. If a human checks every output, you have moved work rather than removed it. That is fine — as long as reviewing is faster than doing, and you have measured that it is.
- Integration. The agent is rarely the hard part. Getting it clean access to your data usually is.
- Maintenance. Model updates change behaviour. A workflow that worked in March can quietly degrade in September, with no error message.
For most small businesses the sober conclusion is that a good assistant plus a few narrow automations beats an agent, and costs a fraction as much. We made that case in best AI tools for small business owners and in cutting business costs with AI, and the agent evidence has not changed it.
Where agents genuinely earn their place
Three situations justify the complexity for a small team.
Volume with variation. Hundreds of similar-but-not-identical inputs — support tickets, invoices, applications — where a fixed rule set keeps breaking on edge cases. Start with triage and drafting rather than resolution; see AI customer service without losing the human touch and building a support system without code.
Work you already outsource. If you pay someone hourly for something with a checkable output, the ROI comparison is concrete rather than speculative, and the substitution logic in AI tools that replace expensive software applies directly.
Bounded research. Competitive monitoring, supplier comparison, pulling structured facts from many documents. Wrong answers are visible and cheap.
Everything else can wait. The capability curve is moving fast enough that the tasks agents cannot do reliably today are a poor place to spend money — waiting six months costs you almost nothing and the tooling gets meaningfully better. If you want to start somewhere with near-zero risk, a no-code chatbot on a narrow scope is the usual first step: building an AI chatbot without code.
Frequently asked questions
What is the difference between an AI agent and a chatbot?
A chatbot responds to messages. An agent is given a goal and decides its own sequence of steps to reach it, using tools and adapting when something fails. Many products marketed as agents are chatbots or fixed automations with new labelling — Gartner calls this agent washing.
Can AI agents replace employees?
Not for whole roles. The best public benchmark of ordinary office work found leading agents completing roughly 30% of tasks even with partial credit, with administrative and finance work scoring lowest. Agents replace bounded tasks with checkable outputs, not jobs.
Why do so many agent projects fail?
Gartner attributes cancellations to escalating costs, unclear business value and inadequate risk controls — scoping and governance problems rather than model capability. The second common cause is arithmetic: long step chains multiply small per-step error rates into large failure rates.
Is it true that 95% of AI pilots fail?
That figure comes from MIT’s NANDA report and measures projects with no measurable profit-and-loss impact, which often reflects the absence of a pre-deployment baseline rather than technical failure. The study’s method has been criticised for a small sample and a narrow success definition. Treat it as directional, not precise.
What should my first AI agent do?
Something high-volume, low-stakes and easy to check — triaging incoming email, drafting first-pass replies for approval, or extracting structured data from documents. Run it in draft-only mode first and count how often you would have had to intervene.
Are AI agents safe to give access to my accounts?
Be careful. Agents acting inside logged-in sessions are exposed to prompt-injection attacks, where instructions hidden in a page or email are read as commands. Keep agents out of banking, payroll and client systems, and prefer read access with human approval before any irreversible action.
Sources
- Gartner — Over 40% of agentic AI projects will be cancelled by end of 2027
- TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks (NeurIPS 2025)
- Fortune — MIT NANDA report on generative AI pilots
- Forbes — Why 40% of agentic AI projects may be cancelled by 2027
Research findings and forecasts checked 20 August 2026. Benchmark results vary substantially with scaffolding and run count; treat any single published score as one data point rather than a capability guarantee.
