Why it matters
Your callers tell you how a call is going in two ways at once. The words carry the promise, the disclosure, the objection, and a reviewer can quote them. The delivery carries hesitation, strain, a raised voice, two people talking over each other. Call1 measures both on the same chunk and stores them as separate results, so anyone reading a verdict can see which signal produced it.
Most of the work downstream wants one number. The review queue has to rank, a contact driver has to trend, an alert has to fire or stay quiet. A versioned fusion policy produces that single label and confidence for every chunk, and the text result, the tone result, and the distance between them stay on the record. Polite wording over an escalating voice is a filterable field and a reason a call can reach a reviewer.
Telephone audio limits what tone can carry. At 8 kHz, with compression, crosstalk, background noise, clipping, and hold music in the mix, the dependable contribution from the voice is arousal and escalation; the read on how the interaction went comes mostly from the words. When a chunk fails the audio-quality gate, the tone signal is marked unavailable with a reason code and the text result stands alone.
How it works
- 01
Cut the call into speaker-aligned chunks
A chunk is a few seconds of one speaker with a stable start and end time, a speaker role, a channel, and a word range into the redacted transcript. Every model that runs afterward gets the same chunk boundaries, so two results can always be compared on the same span.
- 02
Run the text model
The text path scores the redacted transcript for the chunk and stores its label distribution, selected label, confidence, context window, model identifier, and revision. It completes whether or not the audio path does.
- 03
Gate the audio, then run the tone model
Speech fraction, signal-to-noise estimate, clipping, overlap, sample rate, and codec decide whether the chunk can support tonal inference. Eligible chunks get their own label distribution and confidence. The rest are marked unavailable with a reason code: silence, hold music, static, too short.
- 04
Fuse the two signals
A separately versioned fusion policy combines whatever is available into one label and confidence. Agreement between the paths raises confidence. Disagreement is written to its own field, and the result carries a status of consensus, ambiguous, or unavailable.
- 05
Attach events, dynamics, and lineage
Configured acoustic events sit beside the affect results: silence, hold music, DTMF, crosstalk, interruption, static, transfer, disconnect. So do the turn facts a reviewer can act on, including who interrupted whom, response latency, and dead air. Every field carries the chunk, the model revision, and the fusion policy version that produced it.
Technical shape
Chunk record
Call ID, chunk ID, ordinal, start and end time, speaker role, channel, language, redacted text, word range, transcription confidence, and speaker-attribution confidence. This is how an aggregate number in Insights opens the exact second of the call it came from.
Text run
Model ID and revision, raw outputs, canonical valence and arousal, label probabilities, raw and calibrated confidence, an out-of-distribution score, and evidence word ranges. Raw and canonical outputs are both kept, so models stay comparable and a single model's behavior stays auditable.
Tone run and its quality gate
The same run fields, plus the quality record that admitted the chunk: speech fraction, signal-to-noise estimate, clipping ratio, loudness, sample rate, codec, overlap, and the reason codes behind a rejection. A rejected chunk records why the tone signal was not used.
Fused result
Fusion policy version, contributing run IDs, fused valence, arousal and frustration, emotion probabilities, consensus strength, text/tone disagreement, model disagreement, status, and reason codes.
Embeddings
A semantic-text embedding and an acoustic-affect embedding per chunk, stored separately with their own model IDs and revisions. Kept apart, they support a later re-fusion under a new policy, and vectors from different models never share a similarity index. Speaker-recognition embeddings are not persisted.
Data boundary and privacy
Sentiment explains what your callers experienced. It never contributes to an agent's score.
Tonal analysis reads the most sensitive artifact you hold: the voice itself. That inference runs on your hardware, inside your boundary. No audio, embedding, or label goes to a hosted speech-emotion API.
Embeddings, label distributions, and fused results live in your database under the same retention and export policy as transcripts and scores. We never pool them into a vendor corpus.
If you allow model improvement from adjudicated calls, the evaluation and any training run in place. An adapter fitted to your examples stays on your appliance unless you approve a specific sanitization and export path.
Performance
Cost is bounded per chunk
Models run once per eligible speaker chunk, and the quality gate spends nothing on audio that cannot support a tonal result. The two retained vectors come to about a kilobyte per chunk, roughly 100 KB for a call, before index overhead.
Text and tone run independently
The two paths execute in parallel and fail apart. A slow or unavailable tone model delays the tonal contribution; the transcript, text sentiment, and your rubric checks still complete.
A challenger runs without changing what you see
A second text or tone model scores the same chunks and writes its own runs in shadow. Promotion takes a versioned evaluation sliced by campaign, speaker role, language, and audio-quality band, and it creates new runs and a new consensus version. Results already recorded are never rewritten.
What it doesn't do
- Sentiment never contributes to an agent's score. It is available for contact drivers, voice-of-customer work, and routing calls to a reviewer, and it is not selectable as a scored criterion anywhere in the product. EU AI Act Article 5(1)(f) has prohibited inferring emotions in the workplace from biometric data since 2 February 2025.
- The output describes what is observable in the recording: negative wording, elevated vocal arousal, repeated interruption. It makes no claim about anyone's internal state and carries no clinical or psychological meaning.
- A fused label never replaces the text result, tone result, quality status, or disagreement stored beside it.
- Poor audio disables the tonal contribution for those chunks and leaves the rest of the call transcribed, scored, and reviewable.
- Recomputing tone requires the source audio, so it works only while your archive still holds the recording. Once your retention policy expires it, the tonal runs already recorded stand and a new tone model cannot backfill them.
What to validate before deployment
- 01
Which languages, call types, and speaker roles must the tone path cover on your traffic?
- 02
What audio-quality thresholds should mark tone unavailable, and how should that read to your reviewers?
- 03
Should a text/tone disagreement raise queue priority, force a review, or stay a filter?
- 04
Which audio events do you want detected, and which parts of the taxonomy should stay off?
- 05
Which model licenses will your legal team accept for commercial on-prem use?
- 06
What labeled sample from your own calls validates the text model, the tone model, and the fusion policy separately?