4,353
records classified
0 failures
97.9%
quotes verbatim
41,338 quotes checked
0.8%
not verbatim
paraphrase or absent
0
attribution gate violations
a Kelvara quote can never decide a use case
Quote fidelity
Every quote is checked against the exact excerpt the model was shown. Four verdicts, not two, because the risk in this kind of pass is a fluent paraphrase rather than an invention. A paraphrase is not a substring, so it would otherwise land in the same bucket as a fabrication and could later be waved through as close enough.
| verdict | share | what it means |
|---|
| EXACT | 97.9% | the wording appears in the excerpt, allowing for the recorder splitting one utterance across lines |
| ELIDED | 1.3% | two or more verbatim spans joined by an ellipsis: nothing invented, but not one continuous utterance |
| NEAR | 0.4% | not exact, but close to a real line. Reported separately and never counted as a match |
| ABSENT | 0.4% | not close to anything shown |
The ELIDED bucket was tested before it was used, on four crafted records: a quote spanning a rep interjection, two verbatim spans joined by an ellipsis, the same with the second span invented, and a plain paraphrase. The invented span was refused as ABSENT. The ellipsis path cannot launder a fabrication.
The attribution gate
A Kelvara rep can corroborate a customer's use case but never establish it, so the use case must be UNCLEAR wherever the evidence quote is a rep's. The first stage broke that rule on 30 of 2,659 records (1.1%), every one of them by taking a rep's summarising question as evidence. Those records were re-asked under a stricter prompt and 28 of 30 resolved; 12 kept a classification with a genuine customer quote, so forcing all 30 to UNCLEAR would have destroyed 40% of them. Both later populations ran under the strict prompt and produced zero violations.
What each population bought
The pass ran in parts, kept apart because they answer different questions and a single blended rate would hide both.
| population | calls | security-relevant | what it was for |
|---|
| STAGE1 | 2,659 | 52.1% | screened candidates that touch a CRM-confirmed security opportunity, plus every candidate above the conversation floor |
| RESIDUAL_YES | 1,395 | 23.4% | screened yes calls with no CRM security opportunity: the cross-sell population by definition |
| OPPFILL | 229 | 12.9% | calls on CRM security opportunities the first stage could not resolve; the only calls left that could change the win analysis |
| MAYBE_SAMPLE_ONLY | 70 | 12.4% | a random sample of the maybe tier, held out of every rate so it stays a measurement of the tier |
Segment is derived, not read
No governed segment field was populated in the CRM, so every segment figure is derived from the employee range on the account record: each value is parsed to an employee band and classified on its midpoint. A missing or unparseable value is UNKNOWN and is never guessed from deal size, account name or industry.
| segment | rule |
|---|
| Enterprise | midpoint 1,000 employees or more |
| Mid-Market | midpoint 100 to 999 |
| SMB | midpoint under 100 |
| UNKNOWN | missing or unparseable; never guessed |
The cost of where the Mid-Market and SMB line sits was measured rather than asserted: classifying on the band floor instead of its midpoint moves 0.1% of export rows and none of the 280 won security opportunities.
Where this pass differs from the earlier Aegisync integration report
Both numbers, every time, rather than quietly superseding the earlier one. Agreement between two independently gated populations is a stronger result than either figure alone; divergence is a result too.
| measure | Aegisync report | this pass | reading |
|---|
| share of stated volumes above 1 TB a day | 2.2% / 2.4% | 2.2% | agreement |
| certified Aegisync integration claims | 318 | 313 | agreement |
| blanket compliance claims | 18 | 14 | agreement, from an independently stated definition |
| benchmark claims stating an actual figure | 37 | 8 | divergence: most screen hits state no number |
| FedRAMP authorization claims | 1 | 0 | divergence, in the good direction |
| screened qualifying calls | 3,359 | 4,054 | +20.7% on identical calls; a strict superset of the earlier screen |
| the maybe tier | none classified | 13.6% | a CRM-linked maybe is 2.4 times better than a random one |
What was deliberately not run
The residual maybe population, 2,231 calls, was measured and not run: 12.4% yield, over 150,000 prompt characters per usable record, and zero added opportunity coverage, because every CRM-linked candidate was already in the first stage by construction. It is the clearest stopping point in the dataset.
What the CRM could not answer
Why the calls are the evidence
The CRM's loss reason, competitor, pain point and volume fields were empty on every export row, and three further fields held a single constant value on every row, so a fill rate check passes them while they carry no information. Whether the empty fields are unused or simply not readable by the export user was flagged for a quick check, because if loss reasons are populated and hidden, win and loss analysis becomes a query rather than a model run.
The one competitive field with data held record IDs, and most of them did not resolve to a name in the export. Every competitive and loss finding in this report therefore comes from what was said on calls, not from what was typed into the CRM.