How Do You Secure an AI Voice Agent Against Abuse and Fraud?

When AI voice agent security comes up, it usually means privacy: where the recordings are stored, who can read the transcripts, whether the vendor is compliant. Those are fair questions, and every vendor has a page for them.

The questions that get asked far less, and that I think matter more to a small business, are the uncomfortable ones. What if someone calls a hundred times a day to burn through my minutes? What if a script keeps my agent talking for hours? What if the junk ends up in my customer records and my team spends Monday chasing people who do not exist? What if a caller simply talks my agent into giving something away?

A voice agent is not like a website. A website visit costs you almost nothing. A voice agent call costs real money every second it runs, it is connected to your calendar and records, and it speaks in your company's name. That makes it a better target than most owners expect. This article is the map: the six ways a voice agent gets abused, a real incident with the numbers, and the layers that contain each one. Four deeper articles follow it.

Here's what matters most:

• A voice agent has a meter running. Every minute of every call, including a worthless one, is billed, so the cheapest attack is simply to keep the agent talking.
• A public address that starts a call is not protected just because nobody has published it. In one documented 2026 attack on a voice demo, a script made 921 calls in about four hours and cost nearly $1,000.
• Most damage comes from missing limits, not clever hacking: no cap on call length, no cap on calls per day, no cap on spend, no country restrictions.
• The second most common problem is junk: spam calls, test calls and bad data that land in your CRM and trigger follow-ups to people who never asked.
• The agent must never be the only thing standing between a caller and an action. Limits, approvals and records have to live outside the AI.

Six ways a voice agent gets abused

I find it useful to separate abuse by what the attacker wants, because the defence for each is different.

1. Running up the bill. The attacker wants your money or your service to stop. Every call costs minutes of speech recognition, a language model and a synthetic voice, so a flood of long calls is a bill. OWASP, the group that publishes the standard list of AI application risks, names this directly. Its 2025 entry on unbounded consumption describes "denial of wallet": attackers start a large volume of operations to exploit the cost-per-use pricing of cloud AI services and impose unsustainable costs on the provider. The OWASP fix is the dull one: rate limits, per-user quotas and timeouts.

2. Traffic pumping and toll fraud. The attacker wants to be paid by the phone network, and you are the one paying. The scheme is called international revenue share fraud, or IRSF. The criminal controls a set of expensive phone numbers, makes your system call them, and collects a share of the connection fees. I cover how this works, with the full numbers from a real case, in what IRSF is and how it can run up an AI voice agent bill.

3. Flooding the line. The attacker wants your phones busy. This is not new: the FBI has warned for years about telephony denial of service attacks, where a flood of calls stops legitimate callers getting through, and has said healthcare organisations have been a target. A voice agent handles many calls at once, but not unlimited ones, and when it hits its limit real customers get a busy signal.

4. Junk in your records. Nobody is attacking you. Spam callers, bored teenagers, test calls, robocalls, wrong numbers and other machines all talk to your agent, and your agent dutifully creates a lead for each of them. I cover this in how to stop fake leads reaching your CRM from a voice agent.

5. Talking the agent into things. The attacker wants a discount, a refund, someone else's information or access they should not have. This is social engineering, and an AI that is trained to be helpful is helpful to the wrong person too. I cover it in can callers trick an AI voice agent into giving things away.

6. Impersonation. The attacker wants to be someone else. Pindrop, a company that sells voice authentication, reported in 2025 that deepfake fraud attempts against contact centres rose more than 1,300 percent in 2024. It is a vendor figure and I would treat it as a direction, not a measurement. But the direction is clear: assume a voice proves nothing, whether the voice belongs to a person, a clone or a script.

Look at the list again and notice what is missing: movie hacking. Almost none of this needs the attacker to break into anything. They use the front door you built, in the way you did not expect.

A real case: $982 in four hours

The best evidence I have found of how this goes wrong is a post published by Twilio, the phone infrastructure company, in October 2026. A Twilio developer advocate, Rishab Kumar, built a voice AI demo and left it on a public address. He describes what happened on August 15, 2026, with figures from his own logs.

The demo had a web page where a visitor entered a phone number and the AI called them. The request that created each call had no login. The GitHub repository was public, and the demo had a public domain. The author never promoted it and could not tell how the attacker found it.

Over about four hours that morning, his server received 1,008 requests and created 921 calls. Of those, 920 went to Israeli numbers in the same range, covering 156 different destinations. A set of 104 numbers were each called seven to ten times and accounted for 851 calls. About 98 percent of the calls connected. Together the calls ran for almost 30 hours of connected time inside a four-hour window. The bill came to $971.50 for phone and voice-agent charges plus $10.58 for the language model: $982.08 in total.

Three details from that post are worth remembering.

First, the attacker was not interested in the demo. The assistant's logs show 8,363 prompts but only 541 distinct lines, mostly repeated conversation-extension prompts, and some calls played unrelated recorded audio. One recorded line was "Thanks for calling our numbers. We wish you a happy trafficking." The goal was minutes to numbers the attacker controls, not conversation.

Second, the usual protection did not apply. Twilio signs its callbacks so an app can check they are genuine, and the author notes that checking those signatures would not have stopped this attack, because his own app made the call request. The real gap was that anyone could ask it to.

Third, the author's one-line lesson, which is the best summary of this whole subject: "a URL is an address, not an access control."

His fixes are the ordinary ones. He capped each call at 120 seconds, set a daily limit per phone number, and noted that this alone would still allow hundreds of calls across 156 numbers, so a global ceiling is needed as well. Remember that thought. It comes back below.

Why a voice agent is a better target than a website

A contact form that gets spammed costs you annoyance. A voice agent that gets abused costs you money and, depending on what it is connected to, data. Four things make it more attractive than a typical web page.

It is metered in real time. Phone and voice AI usage is billed per minute, so an attacker's cost to waste your money is close to zero while yours is not. Retell, a voice agent platform, lists a standard limit of 20 simultaneous calls on its pay-as-you-go plan and offers a "burst" option that lets calls go over that limit. Its documentation says burst calls carry an added $0.10 per minute surcharge on the entire call. Read that twice. The setting that stops real customers hitting a busy signal during a rush is the same setting that makes an attack more expensive. I would not enable it on anything public without a spend cap next to it.

It looks legitimate. Bots filling a form look like bots. A script holding a voice conversation, with a human-sounding agent, looks like a successful call in your reports, and as the Twilio case shows, 98 percent of the fraudulent calls completed.

It is connected to things. A voice agent that books appointments writes to your calendar. One that qualifies leads writes to your CRM. One that can transfer calls can dial other numbers. Each connection is something an attacker can use.

It speaks for you. What your agent says is, in practice, what your company said. A Canadian tribunal held Air Canada responsible for wrong fare information its chatbot gave a customer, and rejected the argument that the chatbot was a separate entity.

The layers that contain it

No single control stops all six. What works is layers, and the layers have a simple order: before the call, at the door, during the call, behind the call and around the call.

Before the call: decide what the agent can do at all. The cheapest defence is a short list of abilities. If your agent only answers inbound calls and books appointments, it has no reason to dial out, transfer to arbitrary numbers or send messages. Every ability you leave off is an attack that cannot happen. OWASP's guidance on excessive agency says the same: minimise the tools an agent can use and limit what each one can do.

At the door: who can start a call. For a phone number, that means blocking countries you do not serve, and blocking known high-risk number ranges. Twilio publishes tools for this and says it updates its high-risk list several times a month without disclosing the ranges. For a web demo or a button on your site, it means the request that starts a call must come from your own pages, be limited per visitor, and be limited across all visitors. The Twilio author's point applies: a per-visitor limit alone does not stop an attacker who uses many.

During the call: limits on every call. A maximum length, a silence timeout so a call that nobody is speaking on ends, and a cap on simultaneous calls. These three limits alone would have contained most of the damage in the Twilio incident.

Behind the call: what the agent can touch. The agent should write new records to a holding area, not your live CRM. It should hold the minimum permissions: read a calendar, create a booking, nothing else. Anything with money or consequences, such as a refund or a discount outside a written limit, should go to a person. The voice-agent testing firm Hamming puts the principle in one line: "The model should never be the only control between a caller and a side effect."

Around the call: spend limits and switches. A daily cost ceiling that cuts the line off, alerts that tell a person when spend or call counts jump, an off switch you can reach from your phone, and logs that show who called, when, how long and what happened. Twilio's author recommends connecting alerts to automatic actions, such as turning off outbound calling when a threshold is crossed. I agree. An alert that arrives at 2 a.m. on a Saturday is a notification. A switch that trips by itself is a defence.

Notice that all of these live outside the AI. You do not make a voice agent safe by writing "do not be abused" into its instructions. You make it safe the way you make any system that handles money safe: limits, permissions, approvals and records.

What I would check on any voice agent this week

If you run a voice agent, or are about to buy one, these are the questions I would ask. Most vendors cannot answer more than half.

1. What is the longest a single call can last, and where is that enforced?
2. What happens after twenty seconds of silence?
3. How many calls can run at the same time, and is there a "burst" setting that raises it? What does burst cost?
4. Which countries can the agent call or be called from? Can I switch the rest off?
5. Can the agent dial any number it is told to, including through a call transfer?
6. What stops one person or one script from starting a thousand calls in a day?
7. Is there a daily spend ceiling that actually stops calls, or only an email alert?
8. Does the agent write straight into my CRM or calendar, or into a holding area that I review?
9. What is the most the agent can promise, refund or discount without a person?
10. Can I turn everything off from my phone in under a minute?

Then do something no demo will do for you: try to abuse your own agent. Call it twenty times in a row. Ask it for things it should refuse. Read it a fake account number. Read the call log the next morning and see whether you would have noticed.

For the specifics of one real system, see our guide to AI voice agents for business, the human in the loop guide for approvals and handoffs, and why AI agents work in demos and fail in production for reliability. If you want to hear an agent first, the demo below is one of ours.

When we build AI voice agents, these limits are part of the design from the first day, not a patch added after the first bad invoice.

FAQ

What are the main security risks of an AI voice agent?

Six come up repeatedly: runaway cost from long or numerous calls, toll fraud that makes the agent call expensive numbers, flooding that blocks real callers, junk data entering your records, social engineering of the agent into giving things away, and impersonation of a customer or employee. Most are prevented by limits and permissions outside the AI, not by AI settings.

How do you stop someone running up my AI voice agent bill?

Put a cap on every call's length, a silence timeout, a cap on calls per visitor and in total per day, a country allowlist, and a daily spend ceiling that cuts off calls automatically. OWASP's guidance on unbounded consumption lists rate limits, user quotas and timeouts as the core controls. Also switch off any burst setting that lets calls exceed your normal concurrency limit unless you need it.

Can hackers abuse an AI voice agent?

Yes, though usually without hacking. In a documented 2026 case, a script used an unprotected call-creation page on a voice demo to place 921 calls in about four hours, mostly to numbers in one Israeli range, costing roughly $982. The attacker used the feature as designed, at a scale nobody expected.

Do I need to worry about spam calls reaching my AI receptionist?

Yes. Spam and robocalls use up minutes and can create junk leads. Some owners of AI answering services report spam calls consuming their monthly allowance. Choose a service that filters known spam, lets you block numbers, and tells you what share of calls were junk.

Is a public voice demo on my website safe?

Only if it has its own limits. A public demo should run under its own account with its own spend ceiling, accept requests only from your pages, limit calls per visitor and across all visitors, cap each call's length, and have an off switch. A web page that anyone can click to start a metered call is an open door unless you limit it.