Call Data Health Check

We took every call our AI agents made between 12 and 15 September, read the transcript, and checked whether the lead details the AI wrote afterwards actually match what was said on the call. This page explains what we found, in plain language.
Silent calls (detail) Connected calls (detail) Every field (detail)
Start here

What happened to 35,766 calls

Four days of dialling, funnelled down. Each step is a plain count — no adjustments.

35,766calls dialled
Our agents placed 35,766 calls across 83 agents and 51 institutions.
17,48949% picked up
Of those, 17,489 were answered and billed as connected. The rest were busy, unanswered or failed.
7,20441% of answered
On 7,204 answered calls the person never said a single word. The line opened, our agent talked, and nobody replied. These still count as connected.
10,28559% of answered
That leaves 10,285 real conversations where the person actually spoke. This is the only pool where lead details can genuinely be gathered.
The finding

Two things go wrong — and one thing goes right

We checked 138,245 individual field values against the rulebook each agent was given. Most were fine. Here is the split.

22,996
values we could confirm are correct

The AI gets the basics right far more often than not

When nobody speaks, the rulebook says to mark the lead "Not Sure", callback "No" and sentiment "1". The AI does exactly that on the large majority of silent calls. That part is working, and it is worth saying so before the criticism.

3,914
invented details on calls where nobody spoke

Problem 1 — Details appear out of thin air

On silent calls the AI correctly says it isn't sure about the lead — but then fills in other boxes anyway: which exam they're preparing for, which city they're in, how interested they are. Nobody said any of it. 2,003 of the 7,204 silent calls carry at least one detail like this.

2,444
wrong values on real conversations

Problem 2 — Real conversations get misread

Even where the person did speak, 2,049 of the 10,285 conversations — one in five — have at least one field that contradicts the rulebook. Usually the person said very little and the AI read far more into it than was there.

See it for yourself

Two real calls

These are genuine calls from 12–15 September. Nothing is edited except trimming the agent's longer speeches.

Parikshe8 secondsnobody repliedID 56ac367a
What was said on the call
Our agent: "Hi, I am Riya. Am I talking to Rahim?" (silence — the call ends)
What the AI recorded afterwards
Lead Qualification Status
Correct — the rulebook says use "Not Sure" when nobody engages.
Not Sure ✓
Call Sentiment (1–5)
Correct — "1" is the score for no engagement.
1 ✓
Exam Category
Invented. Rahim never spoke, so he never mentioned SSLC or anything else.
SSLC ✗
Parikshe13 secondslead said one wordID 4cac1a2d
What was said on the call
Our agent: "Hi, I am Riya. Am I talking to Rajat?" Rajat: "Yes." Our agent: "ನಮಸ್ಕಾರ! I'm calling from Parikshe, Karnataka…" (Rajat says nothing further)
What the AI recorded afterwards
Lead Qualification Status
Rajat only confirmed his name. The rulebook reserves "Qualified" for someone who answers questions or shows interest.
Qualified ✗
Call Sentiment (1–5)
A score of 2 means "participated a little". One word confirming identity is not participation.
2 ✗
Exam Category
Invented again — Rajat never mentioned any exam.
SSLC ✗

This is the pattern behind the complaints. A lead says one word to confirm who they are. The AI treats that as a conversation, marks them Qualified, scores the mood, and fills in a subject they never mentioned. A counsellor then rings someone who has shown no interest at all.

What it costs

What this means for the team

Two numbers matter more than the rest, because they are the ones that send a human to make a phone call.

1 in 18
leads marked Qualified never actually engaged — they gave a greeting and nothing more. That's 215 of 3,831 qualified leads going to counsellors on no real signal.
1 in 9
records saying the lead asked for a callback have no such request anywhere in the call. That's 112 of 1,021. The rulebook is explicit that our agent offering a callback does not count.
311
leads who did talk properly were marked "Not Sure" and likely dropped. This is the opposite error, and it quietly costs us real prospects.
Where it goes wrong

Which details are least reliable

Across all connected calls. "Checked" is how many times we could judge the field against the rulebook; the percentage is how often it was wrong.

What the field recordsWrongCheckedWrong %

The big-volume fields are mostly fine. It is the smaller, more specific questions — which exam, are they interested in admission — that the AI guesses at rather than leaves blank.

The deeper issue

"Qualified" means eight different things

This one is ours to fix, not the vendor's. Across 71 agents there are eight different written definitions of what makes a lead qualified — and some flatly contradict each other.

The common definition44 agents · 26,436 calls
A lead is Qualified if they engage at all — answer a question, share a detail, or agree to a callback.
A second definition12 agents · 2,409 calls
A lead is Qualified only on real interest in the offering. Agreeing to a callback explicitly does not count. The exact opposite of the common rule.
A third definition3 agents · 2,142 calls
A lead is Qualified only if they give a callback time. Nothing else qualifies them.
A fourth definition7 agents · 1,752 calls
One line, 78 characters: "user shows interest OR asks questions OR discusses admission OR shares details" — against 1,202 characters of detail in the common rule. Far too vague to apply consistently.

Why this matters. A "Qualified" lead from one agent is not the same thing as a "Qualified" lead from another. Comparing qualification rates between institutions right now is meaningless, and some of what looks like the AI getting it wrong is really our own settings disagreeing.

Next

What we would fix first

  1. Stop filling in details when nobody spoke. If there is no reply in the transcript, every field beyond the three standard ones should be left blank. This alone removes 3,914 wrong values.
  2. Raise the bar for "Qualified". Confirming your own name is not engagement. This is 215 bad handoffs to counsellors over four days.
  3. Fix the Exam Category question on the Parikshe agent. It returns "SSLC" by default rather than leaving it blank — one mis-written question causing over 1,000 wrong values.
  4. Agree one definition of "Qualified" and apply it everywhere, or accept that the number cannot be compared across institutions.
  5. Look at the 311 leads we marked "Not Sure" who actually talked. Those are prospects we paid to reach and then ignored.