High-Accuracy Recognition
on Real Indian Speech

Evaluation period: 20–21 October 2026. All speech-recognition results in this benchmark are for Naad v2. Competitor model names follow the evaluation record. Results represent this dated snapshot; current model specifications are available through the reference links.

The evaluation record describes up to 500 clips per language and dataset, selected with a fixed seed. Comparisons use the common set of clips completed by every system. Naad trains on Vaani and IndicVoices training data, with evaluation speakers described as held out. This is an in-domain evaluation. Success-only WER measures transcription accuracy; completion rates need a separate reliability assessment.

WER = (substitutions + deletions + insertions) / reference words × 100. The headline averages the 12 published Vaani WER values shared by all three systems, with equal weight per language and Hindi-proxy rows excluded. Calculations use the rounded source values. This equal-language mean differs from pooling all reference words into a single WER.

The recorded scoring process removes case and punctuation differences, normalises nukta and digit forms, and uses the Whisper English normaliser. On Vaani, each system receives its better score against native-script and mixed-script references. IndicVoices uses native-script references. Manipuri and Santali outputs are transliterated into the reference script for every system. These conventions define the comparisons shown here.

The page focuses on Vaani spontaneous speech. Complete downloads also include IndicVoices, Svarah and FLEURS; relative performance varies across these datasets, so the Vaani results should be read within their stated scope. Bodhan uses a Hindi setting for Chhattisgarhi, Magahi and Marwari; these rows are labelled as proxies and excluded from the headline. Missing entries describe this evaluation, not current product availability.

The recorded setup uses the same 25 prompts per language for each tested system. Generated audio is transcribed by Bodhan ASR and compared with the intended text using character error rate (CER). This measures recogniser-based intelligibility and includes the recogniser’s own errors and dialect effects. Human voice-quality and preference studies measure a different dimension. The Marathi panel is an example; the explorer includes all synthesis language rows.

UTMOS estimates naturalness on a 1–5 scale. It is an automated predictor, not a human mean opinion score. The evaluation record describes an English-trained model; scores on Indian-language speech should be treated as an automated diagnostic, not a validated native-listener preference ranking.

Requests were sequential from India, according to the evaluation record. Naad ran on the same machine; competitors were called over the internet. First-audio timing measures request to first audio byte. Sarvam REST timing measures the complete clip and is shown separately; its streaming endpoint was not evaluated. Provider summaries are medians of available language-level timings, not pooled request p50s. Different network paths and language sets prevent a like-for-like speed ranking. These measurements are not full conversation latency or a hosted latency guarantee.

The TTS record contains 27 Naad language rows, including beta rows for Odia, Kashmiri and Manipuri. The by-language timing chart shows 25 languages; Bodo and Odia remain in the complete downloads and the 27-language median of 454 ms. Gemini timing summaries use Hindi and Gujarati; the partial Punjabi result is excluded. Counts describe tested rows, not total product support. Telephone-band (8 kHz) speech, diarisation and production call latency were not evaluated.

Published by Oogam from its internal evaluation record; this is not an independent third-party audit. Downloads contain aggregate measurements and the recorded methodology. Per-run inputs and outputs, exact model configurations, split manifests, scoring code, failure counts and confidence intervals are not included in this publication. The results describe the listed tests and do not establish statistically significant superiority. Reference links document the products, datasets and metrics used.