Blog

How fast should an AI voice agent respond on a call? In about one second or less

By Sharrk Team · November 12, 2026

An AI voice agent should start answering within about one second of the caller finishing a sentence, and the best calls feel closer to the quick back-and-forth of a normal phone chat. Sharrk's AI voice agents are built for fast, natural turn-taking, with controls for responsiveness and interruptions, so a caller with a burst pipe or a new estimate request talks to something that sounds like an attentive person, not a slow machine.

That one-second figure isn't arbitrary. It comes from how people actually talk, what research says about pauses, and how voice AI platforms measure delay. Here's what a business owner needs to know about voice agent speed, why it affects booked jobs, and how to check it on your own line.

What is AI voice agent latency?

Latency is the pause between the moment a caller stops talking and the moment the agent starts talking back. Voice AI platforms usually call this end-to-end latency. Retell AI's documentation, for example, defines it as "the total time from when the user stops speaking until the AI agent begins responding," and breaks it into parts you can measure separately.

Every reply goes through the same short chain:

StepWhat happensWhat slows it down
Hearing the callerSpeech is turned into text as the caller talksBackground noise, a choppy connection
Knowing they're doneThe agent decides the caller has finished their turnWaiting too long, or jumping in too early
ThinkingThe language model writes the first words of the replyVery long prompts, slow models, lookups
SpeakingText-to-speech produces the first soundHeavier voices, network distance
ToolsChecking a calendar or booking a slotSlow outside systems

The total is what the caller feels. A fast model paired with a slow booking lookup still sounds slow at the moment it matters most.

How fast do people respond to each other on the phone?

Very fast. A study published in PNAS in 2009 looked at conversations in 10 languages, from small indigenous communities to major world languages, and found the same pattern everywhere: people avoid talking over each other and keep the silence between turns to a minimum. The average gap differed by language, but only within about 250 milliseconds of the overall average.

A 2015 review in Frontiers in Psychology put the typical gap between turns at around 200 milliseconds. That's faster than a person can plan a reply from scratch (the same review notes that producing even a single new word takes around 600 milliseconds or more), which means listeners start planning their answer before the other person finishes.

So when a caller hears a pause much longer than that, they notice.

Why does a slow AI voice agent cost you jobs?

Pauses carry meaning. The same 2015 review notes that gaps of 700 milliseconds or more tend to come before a "dispreferred" answer: a no, a problem, a hesitation. Callers read a long pause as uncertainty, even when the agent is about to say "Yes, we can be there this afternoon."

In practice, slow replies cause three problems for a service business:

  • Talking over each other. The caller assumes the line went quiet and starts again, right as the agent begins. Now both are talking, and the agent has to recover.
  • Lost trust. A caller deciding between you and the next company on the list hears hesitation as "this place isn't sure."
  • Hang-ups. Someone calling about an emergency has little patience. If the conversation feels like a bad connection, they call the next number.

Phone networks have long been held to the same standard. The ITU's G.114 recommendation, the international guideline for one-way delay on voice calls, says delays under 150 milliseconds give "essentially transparent" interactivity and recommends staying under 400 milliseconds for general network planning. An AI agent's thinking time sits on top of that network delay, which is why every part of the chain needs to be quick.

What is a good response time for an AI voice agent?

Aim for about one second or less from the end of the caller's sentence to the start of the agent's reply, with most turns faster than that. Here's how the research and platform guidance line up:

Response timeWhat it means
About 200 millisecondsThe typical gap between turns in human conversation
700 milliseconds or moreA pause long enough that, between people, it tends to signal a no or a problem
Under 1.5 secondsRetell AI's guidance for an agent's estimated latency
2.5 secondsThe slowest 1% of turns in Retell's example call: the kind of outlier to hunt down

Retell AI's guidance for builders is to keep estimated latency under 1.5 seconds. Its own example call shows why averages aren't enough: a median of 800 milliseconds, but a worst 1% of turns at 2,500 milliseconds. One slow turn at the wrong moment, like when the caller asks "Can you come today?", can undo a good call. Look at the slowest turns, not just the typical one.

What makes an AI voice agent slow?

Most delays come from a few fixable causes:

  1. Oversized instructions. Retell notes that prompts beyond roughly 8,000 tokens get noticeably slower. A tight, well-organized prompt beats a giant one.
  2. Slow lookups with no filler. Checking a calendar or a field service system can take a second or more. Good agents say something natural ("Let me check what we have open") while the lookup runs, instead of going silent. Retell recommends that for any tool that takes over a second.
  3. The wrong balance of speed and accuracy. Faster transcription and lighter models reply sooner; heavier ones can be more accurate. The right mix depends on the call.
  4. Turn-taking set too cautious. If the agent waits too long to be sure the caller is done, every turn gets slower. Set it too eager and it interrupts. This is a setting worth tuning.
  5. Noisy calls. Job sites, trucks and windy yards make it harder to hear when the caller has stopped.

How do you test an AI voice agent's speed?

You don't need special tools. Before you put an agent on your main line:

  1. Call it yourself from a cell phone, ideally somewhere noisy, like your truck.
  2. Ask real questions your customers ask: "Do you service my area?", "How soon can someone come out?", "Can I book Thursday morning?"
  3. Interrupt it mid-sentence. A good agent stops and listens.
  4. Ask it to book. The booking step is where slow lookups show up.
  5. Listen to the recording and read the transcript. Note any turn where you started talking again because the pause felt too long.

Run the same test after any big change to the prompt, voice or booking setup. For more on keeping callers from waiting, see our guide to how long is too long to keep a caller on hold.

How does Sharrk keep voice calls fast and natural?

Sharrk's AI voice agents answer every call 24/7, qualify the caller, book the job onto Sharrk's calendar, Calendly, Cal.com or ServiceTitan, and live-transfer to your team when a caller asks for a person. The speech controls that shape how quick and natural a call feels are built in: responsiveness, interruption sensitivity, backchannel (the small "mm-hm" and "got it" sounds people make while listening), noise and background-speech removal, and a fast transcription mode for lower latency. You choose the voice and the AI model, across OpenAI, Anthropic and Google.

Mako, Sharrk's AI assistant, builds and tunes all of this from plain English. Tell Mako "callers say the agent is slow to answer" or "it keeps cutting people off," and it proposes the specific changes, from tightening the prompt to adjusting responsiveness and interruption settings, for you to approve with one click. Mako can also run a test call to your phone, read the transcript afterward and suggest fixes, so tuning doesn't mean digging through settings yourself.

Every call is logged with its recording and transcript, so you can hear exactly how the agent handled the pauses that matter. If you're also weighing what calls cost to run, our breakdown of AI voice agent cost per minute covers it, and Sharrk's pricing starts with a free plan.

You can build and test a voice agent with Mako on Sharrk Free, with no credit card, or book a demo to hear one answer a call for a business like yours.

If your agent cuts callers off or stops talking for no reason, see our guide on whether an AI voice agent can handle interruptions and the settings that fix it.

Related on Sharrk: AI receptionist · AI answering service · Warm transfer vs. cold transfer

Sources

  • Stivers, T., et al., "Universals and cultural variation in turn-taking in conversation," Proceedings of the National Academy of Sciences 106(26), 10587-10592 (2009): https://doi.org/10.1073/pnas.0903616106; abstract at https://forms.mpi.nl/node/50939 (checked November 12, 2026)
  • Levinson, S. C., and Torreira, F., "Timing in turn-taking and its implications for processing models of language," Frontiers in Psychology 6, 731 (2015): https://www.frontiersin.org/articles/10.3389/fpsyg.2015.00731/full (checked November 12, 2026)
  • ITU-T Recommendation G.114, "One-way transmission time" (approved May 7, 2003): https://www.itu.int/rec/T-REC-G.114-200305-I/en (checked November 12, 2026)
  • Retell AI documentation, "Check actual latency": https://docs.retellai.com/reliability/check-actual-latency (checked November 12, 2026)
  • Retell AI, "Understanding voice AI latency and its impact on call experience": https://www.retellai.com/learn/understanding-voice-ai-latency-and-its-impact-on-call-experience (checked November 12, 2026)

Frequently asked questions

How fast should an AI voice agent respond?

Within about one second of the caller finishing a sentence, and faster on most turns. People leave gaps of around 200 milliseconds between turns in normal conversation, so long pauses stand out.

What is end-to-end latency in a voice agent?

It's the time from when the caller stops speaking until the agent starts responding. It adds up transcription, deciding the caller is done, the AI model writing a reply, text-to-speech and any lookups like checking a calendar.

Why do callers hang up on slow AI agents?

Because long pauses sound like hesitation or a bad connection. Between people, pauses of 700 milliseconds or more tend to come before a no or a problem, so callers lose confidence and some call the next business.

What slows down an AI receptionist?

Oversized prompts, slow calendar or booking lookups with no filler phrase, heavy transcription or models, overly cautious turn-taking settings and noisy calls are the usual causes. Most are fixable with settings and a tighter prompt.

How can I test my AI voice agent's speed?

Call it from your own cell phone, ask real customer questions, interrupt it and ask it to book a job. Then listen to the recording and note any turn where the pause made you start talking again.

Can Sharrk's voice agents be tuned for faster replies?

Yes. Sharrk voice agents have controls for responsiveness, interruption sensitivity, backchannel, noise removal and a fast transcription mode, and Mako can propose these changes from a plain-English request like "it's slow to answer."

Start free, no credit card · See pricing · Book a demo