What is voice AI latency?

Why latency defines interview quality

Human conversation tolerates brief pauses — thinking time feels natural. But consistent multi-second gaps after every answer feel broken, like a bad international phone call or a chatbot refreshing. Candidates interpret long latency as the system not listening, losing trust in the process before scoring even begins.

Interview engines compound latency across the stack. Voice activity detection must decide the user stopped talking. Automatic speech recognition must return text. A reasoning model must choose the next question or acknowledgment. Text-to-speech or pre-rendered audio must play. Each hop adds milliseconds; serial pipelines multiply them.

Turn-taking makes latency harder than simple Q&A bots. The system cannot start generating a full response until it knows the user finished — but waiting too long to detect end-of-turn also adds delay. Engineering teams tune end-of-turn thresholds, partial transcript streaming, and speculative prefetch to hit conversational targets appropriate for live hiring.

Components of end-to-end latency

Latency vs accuracy trade-offs

Pushing latency down by cutting ASR wait time risks transcribing incomplete sentences — producing wrong follow-ups. Aggressive end-of-turn triggers cause the AI to talk over candidates still speaking, which is worse than a slightly slow response in hiring contexts where candidates need space to think.

Streaming partial transcripts to the reasoning layer can shave perceived latency if the model begins drafting while the candidate is still finishing — but only if the final transcript rarely reverses meaning. Accent-heavy speech with higher word error rates makes premature inference dangerous.

Product teams sometimes publish engineering targets for sub-second components while acknowledging real-world mobile networks add variance. Buyers should evaluate demos on the same device class and connectivity their candidates use — a office Wi-Fi demo misleads for warehouse applicants on budget Android phones.

Measuring latency in production

• P50 / P95 end-of-turn-to-first-audio-byte — median and tail matter; tails ruin interviews for unlucky candidates.
• Segment by network type, device OS, and language — aggregate averages hide bad regional experience.
• Correlate latency spikes with session abandonment and negative candidate feedback.
• Separate 'thinking filler' audio ('Hmm, one moment…') from true silent gaps — fillers manage perception but should not mask systemic slowness.

Frequently asked

Related