AI & Automation

The Pilot to Production Gap in AI Projects

The pilot worked and the rollout did not. The six failures that sit between a demo and a production system, and what has to exist before the second unit goes live.

Mauricio Esparza By ·Published ·3 min read
mithub.club
Short answer

AI pilots stall for operational reasons, not model reasons. The six that account for most of it: the pilot was a build rather than a configuration, the data was cleaner in the pilot than in reality, nothing was written back where humans work, there was no error route, nobody owned it after launch, and success was measured in activity rather than in a business result.

The pilot worked. That is exactly the problem.

A pilot runs under conditions that disappear the moment you scale: the person who built it is watching, the data was cleaned by hand, and the team involved knew it was coming. At unit number two, none of those things are true.

Here are the six failures that account for most stalled rollouts, and what has to exist before the second unit goes live.

1. The pilot was a build, not a configuration

The first location got a workflow. The second one needs different fields, a different script, a different routing rule and a different phone number, so somebody rebuilds it. Now there are two systems, and a fix applied to one silently does not apply to the other.

The test: write down what changes between unit one and unit two. If the list is values, it scales. If the list includes logic, it does not.

What to do: turn the pilot into a template with a configuration per unit. A location should be a config, not a development project.

2. The data was cleaner in the pilot than it is in reality

Somebody normalised the phone numbers for the pilot. Somebody removed the duplicates. Often they do not even remember doing it, because it felt like preparation rather than work.

Then the real list arrives with three variants of the same company, numbers that predate the records holding them and empty fields where the logic expects values.

What to do: enrich, verify and deduplicate before contact, and route bad contact information into its own path instead of spending a dial on it. A wrong number costs exactly what a right one costs.

3. Nothing was written back where humans work

The AI called, learned something and stored it in its own platform. The CRM shows nothing. So the rep calls back and asks questions the customer already answered, and the customer concludes nobody is paying attention.

What to do: whatever the system learns lands on the record a human already opens. Outcome, a two line summary, the corrected contact details, the next step. A log only the builder reads is worth nothing operationally.

4. There is no error route

The API times out at 2am. The model returns valid JSON with a wrong value. A webhook fires into a workflow that was renamed last week. Automation documentation treats error handling as a first class concern precisely because what has no error route falls over in silence.

What to do: every flow gets an error path and an alert with enough context to act on. And build small: start with the simplest thing that works and add autonomy only where it earns its cost.

5. Nobody owned it after launch

The builder moved to the next thing. Six months later the script no longer matches the offer, one integration has been failing quietly since a platform update, and nobody can say when it stopped working.

What to do: name an owner before launch, not after. Write the runbook while building, not at handover. And schedule a review that actually happens, because these systems decay in the direction of the business changing around them.

6. Success was measured in activity

Twenty thousand calls is not a result. Neither is a hundred thousand emails sent. Activity metrics are easy to produce and they let a project look healthy while producing nothing.

What to do: define the business result before launch and instrument the path to it. Money received, walked backwards through the deal, the handoff, the conversation and the list it came from. If you cannot trace it, you cannot decide what to keep.

The one thing to fix first

If you only address one of the six, make it the fifth: give it an owner. Most of the other failures are survivable when somebody is watching, and none of them are when nobody is.

The second thing is the definition of done. A green execution is not proof that anything worked. The proof is the record: open the lead, and the business result is either there or it is not. Teams that change that one definition find half their silent failures within a week.

If the blocker is inside your systems

That is a specific job, and it has a name. MitHub keeps open profiles of forward deployed engineers who have taken this work from pilot to production inside real companies, with each deployment written out: the problem, the approach, what was built and what it produced.

See the engineers. Reading the work and contacting them is free.

Frequently asked questions

Is the model the problem?

Almost never. Models fail in visible ways and get fixed. Deployments fail in invisible ways: a field that was never written, an error nobody was alerted to, a process that only one person understood. The failures that kill pilots are operational.

How do I know if my pilot can scale before I try?

Ask what changes for unit two. If the answer is a configuration file, a phone number and a script, it scales. If the answer includes rebuilding a workflow, it does not, and fixing that now is cheaper than discovering it across ten units.

What is the single best predictor of a rollout working?

Whether a person other than the builder can operate it. Not whether it works, whether it survives its author. That is testable in an afternoon: ask someone else to run it from the documentation.

How should success be measured?

By a business result visible in your system of record, not by execution counts. Open the record and the outcome is either there or it is not. Volume metrics tell you the machine ran, not that anything changed.

Sources

  1. Building effective agents · Anthropic (accessed 2026-09-22)
  2. Handle errors gracefully · n8n Docs (accessed 2026-09-22)
  3. Why evaluate AI workflows · n8n Docs (accessed 2026-09-22)
  4. The Future of Jobs Report 2025 · World Economic Forum (accessed 2026-09-22)
AI & AutomationImplementationOperations
Mauricio Esparza
Mauricio EsparzaGTM Systems Lead · Revenue Engineer · Founder of MitHub. Designs and runs revenue systems for multi-location businesses: AI voice campaigns, enrichment, CRM automation and attribution. Founded MitHub to teach the method in the open.

Part of AI & Automation on MitHub.

Keep going

AI & Automation

Human in the Loop: Where People Belong in AI Workflows

How to place humans in AI workflows on purpose: risk tiers, seven approval patterns, the rubber-stamp trap, and written criteria for removing a review gate.

Read · 9 min →mithub.club
n8n

n8n Error Handling for Production Workflows

How to make n8n workflows survive production: node retry settings, error workflows, deliberate failure, dead-letter records, timeout sweeps and real alerts.

Read · 8 min →mithub.club
AI & Automation

Process Mapping Before Automation: How to Map Work Before You Build

A four-pass method for mapping a business process before you automate it: the money line, swimlanes, numbers on every step, and finding the constraint.

Read · 8 min →mithub.club
AI & Automation

When to Hire a Forward Deployed Engineer

The buyer's view of the forward deployed engineer: the problems that need someone embedded in your systems, how the engagement is scoped, and how to tell a real one from a consultant with slides.

Read · 3 min →mithub.club
Revenue Engineering

Revenue Attribution Without the Hype

What revenue attribution can and cannot tell you: the models, a worked example across first touch, last touch and linear, the limits, and what to do instead.

Read · 7 min →mithub.club
AI & Automation

Systems Thinking for AI Automation: Stocks, Flows, Loops and Bottlenecks

Systems thinking in plain language for AI automation: stocks, flows, feedback loops and bottlenecks, plus a MitHub map to find what to automate first.

Read · 6 min →mithub.club