A deployment write up, anonymised. The company is a consumer finance network in the United States with branches. The name is withheld because publishing it requires the client's approval.
The situation
The AI calling operation was running across the network and there was one number to describe it: a company wide average, taken from the calling platform's own view.
Everybody looked at it in the weekly meeting. Nothing changed for weeks, and not because people were not trying.
Why an average does not work here
A branch manager reading a number that includes 27 other branches learns nothing about their own market, their own calling hours or their own team's response to transfers.
Worse, the average hides the thing that matters most in a network: the spread. Two branches at opposite ends of the distribution can produce a perfectly reasonable looking middle.
What was built
Calls, outcomes and transfers pulled from the calling platform's API into a report per branch, showing:
- Dials, connects and connect rate for that branch
- Live transfers, and how many were answered
- Outcomes by category
- All of it next to what was expected for that branch
The technical work was straightforward. The work that mattered was deciding what each branch should see and what to compare it against, because a report nobody can act on is easy to produce and worth nothing.
What it revealed
Three things, in order of surprise:
The spread was much wider than the average implied. Some branches were converting transfers at several times the rate of others.
Most of the causes were operational rather than about the AI. Calling windows set for the wrong local hours. Transfer lines nobody was answering during certain shifts.
Some branches had stopped answering transfers entirely, which no company wide number would ever have shown.
What changed in the meeting
The question went from "how is the AI doing" to "why is this branch at half the transfer rate of that one".
The second question has an answer. Usually the calling window, sometimes the transfer line, occasionally the script. All three are fixable in a week by someone who can see them.
The general rule
Report at the level where somebody can act.
A number that reaches a person who cannot change what it measures is decoration. This is true well beyond calling platforms, and it is the most common reporting failure in multi unit businesses of every kind.
The cadence that worked
Weekly for operations, which is fast enough to change a calling window and slow enough to be more than noise. Revenue attribution stayed quarterly, because it needs enough closed deals to be signal rather than variance.