Your callers switch language mid-sentence. Your AI should switch script with them.
Premium थोड़ा ज़्यादा है but the cashless network बहुत बड़ा है।
Shown: Hindi–English. The same problem appears in Spanglish, Taglish, Arabic–French, Singlish and anywhere two languages share a conversation.
We fine-tune small language models on real code-mixed sales calls, then run them inside your own infrastructure. Each language keeps its own script. Nothing leaves your network.
Three reasons voice AI pilots stall in regulated industries
Legal will not sign the API
Customer calls carry PII, health disclosures and financial detail. Routing that to a third-party model means a vendor DPA, a subprocessor review and a security team that has to get comfortable with prompts leaving your perimeter. Under India's DPDP Act and RBI localisation expectations, that review often simply ends in no. Most pilots die here, not in the demo.
The latency is audible
A voice turn has a few hundred milliseconds before the caller hears the pause and talks over the agent. Speech-to-text, retrieval, generation, compliance check and speech-out all have to fit inside it. A round trip to a frontier API spends most of that budget on the network.
Nobody can explain what it said
When a regulator or your E&O carrier asks why the system made a specific statement to a specific customer, “we called an API” is not an answer. You need the model, the version, the inputs and a reproducible output.
Small models, in your cloud, on your language
A frontier model is doing enormously more work than a sales call requires. Most of what happens on these calls is bounded: classify the objection, retrieve the right fact, phrase the response in language your compliance team already approved, insert the required disclosure, log everything.
That work fits in a model small enough to run on a single GPU inside your own VPC — which changes the latency, the economics and the compliance conversation at the same time.
Fine-tuned on your conversations
Trained on how your agents actually handle your products, in the languages they actually mix — not a generic assistant prompted to pretend.
Deployed inside your boundary
Your cloud account, your data centre, or air-gapped. Audio and transcripts never transit a third party.
Auditable by construction
Pinned model versions, logged inputs and outputs, reproducible generation. “Why did it say that” has an answer a year later.
Honest about the stack
We fine-tune and serve the language and compliance layer. Speech recognition and speech synthesis are separate choices, and we tell you plainly which we host and which we do not.
Frontier models are weakest wherever two languages share a sentence
A code-mixed sales call is not a translation problem. It is a single utterance that switches language mid-clause, often uses two scripts, and carries product terms that must stay in one language even inside a sentence written in the other.
Frontier models have seen relatively little authentic code-mixed sales speech, and almost none of it in your product domain. A small model fine-tuned on ten thousand real calls can plausibly beat them at this narrow task — which is a stronger claim than “nearly as good”.
There is also a cost asymmetry nobody prices in. Non-Latin scripts tokenise far less efficiently than Latin in standard tokenisers, so per-token API pricing is materially worse for Hindi, Arabic or Thai than the headline rate suggests. Self-hosted inference has no per-token cost at all.
What we claim, and what we have not measured yet
- Script fidelity, code-switch handling and named-entity preservation are testable properties, and we test them per model.
- Non-Latin scripts consume more tokens than Latin for equivalent meaning, so per-token pricing penalises them. We are measuring the exact multiplier on real transcripts.
- We have not yet published benchmark results. The methodology is public; the numbers land when the first dataset is complete.
- Speech recognition on code-mixed audio is the weakest link in the whole pipeline, not the language model. We measure it first.
Code-mixing is not an Indian problem. It is most languages, most places.
Hindi & English — India
Two scripts in one sentence, English product terms inside Devanagari clauses. Where we measure first, because it is where we can reach real call archives.
Spanish & English — United States
Tens of millions of speakers, heavy insurance and healthcare call volume, and carriers still routing those calls to English-only systems.
Tagalog & English — Philippines
The country's outsourcing industry is voice work, and code-mixing is the default register on the floor.
Arabic & French — Maghreb
Two scripts, two directions of writing, and frequently a third language in the room.
Malay, Chinese & English — Singapore, Malaysia
Three-way mixing inside single utterances, with heavy financial services call volume.
Swahili & English — East Africa
Fast-growing insurance and lending markets served almost entirely by voice.
Every one of these has the same three properties: thin frontier training data on authentic code-mixed speech, a real script or orthography problem, and tokenisation that penalises the non-English half. We prove the method on one language pair and carry it to the rest — which is why our published benchmark names its pair explicitly.
Industries
- InsuranceObjection handling · compliance phrasing · rate lookup · call records
- Real estateLead qualification · price and financing objections · logged follow-up
- Healthcare In developmentPatient access · scheduling · intake · benefits verification
- Your industry is not on this listThe models are specific. The process is not.
We found this problem by running into it
We already operate a voice AI sales platform. EchoLeads handles lead qualification and follow-up calls for real estate and other high-volume sales teams, and it works — prospects have asked to speak to the agent again, believing it was a person.
Running it in production taught us where the current architecture breaks. Frontier models handle a clean English call well and a code-mixed call inconsistently — putting one script where the other belongs, mangling product names. Per-token pricing charges more for Hindi than for the same meaning in English. And every regulated buyer we spoke to stopped at the same question: where does the audio go?
So we built the layer underneath. EchoLeads is where we found the problem, where our training data comes from with customer consent, and where we test the fix.
Lohit Kapoor, Ph.D. — Co-founder
Founder of EchoLeads, where autonomous voice agents run sales conversations at production volume. Ph.D. from Thapar University, with published research on latency across geographically distributed cloud systems — the same constraint that decides whether a voice agent sounds like a conversation. Based in Hyderabad.
Atita Arora — Co-founder
In information retrieval since 2008. Search relevance consultant at OpenSource Connections, and AI and information retrieval roles across Qdrant, Lucidworks and Voyager Search. Contributor to Apache Solr, OpenNLP, Quepid and Chorus, and co-leader of Women in Search. Her focus is evaluation — defining and measuring what “good” means in a retrieval system, honestly.
We are looking for one insurance partner
We are building the first public benchmark of fine-tuned small models against frontier APIs on real code-mixed insurance sales calls — accuracy, script fidelity, latency, cost per call and compliance failure rate. We will publish the method and the results, including where the small model loses.
Our evaluation method is published in full before any results exist, so you can judge it independently of the numbers it produces. Training data comes from consented EchoLeads customer conversations, with personal information removed before any label review.
Questions go straight to the founders: founders@vaakya.ai
Common questions
What does Vaakya do?
Vaakya fine-tunes small language models for voice conversations in regulated industries and deploys them inside the customer's own infrastructure. The models handle bounded call tasks — objection handling, compliance-safe phrasing, information lookup and structured call records — in code-mixed languages such as Hindi–English, producing output in the correct script for each language run.
Why use a small self-hosted model instead of a frontier API?
Three reasons. Compliance: customer audio and transcripts never leave the customer's network, which is often the only architecture a regulated buyer can approve. Latency: a voice turn allows a few hundred milliseconds in total, and a network round trip to a hosted API consumes most of it. Cost: self-hosted inference has no per-token charge, which matters disproportionately for Indic scripts that tokenise inefficiently.
What is code-mixing, and why is it hard for AI models?
Code-mixing is switching between languages inside a single sentence, which is how a large share of sales conversations are conducted in India, the Philippines, the Gulf, Southeast Asia and bilingual communities across the US and Europe. It is hard for models because speech recognition is usually trained monolingual, because output must use the correct script for each language run, and because product terms often have to stay in English even inside a Hindi sentence. Most models trained predominantly on monolingual text handle it inconsistently.
Do you publish benchmarks?
The evaluation methodology is published now. Results are not, because the first dataset is still being assembled with a design partner. We report measurements only where we have taken them and show a dash where we have not, and we publish cases where our models lose to a frontier API. See methodology and benchmarks.
Which industries do you serve?
Insurance is the lead vertical. Real estate is live. Healthcare patient access is in development. For other industries we run the same process on the customer's own call archive — see other industries.
Where is Vaakya based?
Vaakya is a Delaware C-corp with an Indian operating subsidiary. Engineering and data work happen in India. We serve carriers and brokerages in India and, through the same models, global insurers running multilingual call centres anywhere.