How Autrans measures word error rate language by language: the public test recordings, how the interval is built, and what the figures cannot tell you.
What a word error rate is, which recordings we test on, how sure we are, and what the numbers cannot tell you about your own audio.
Test Autrans on your own recording
The short version:
Take a reference written by a person and a transcript written by Autrans. Count the words that were swapped, missed or added, and divide by the number of words in the reference. Lower is better.
The rate can pass 100 when a transcript adds many words, which is why we do not turn it into an "accuracy" percentage. A sentence worked through step by step is on what word error rate means.
Before scoring, both sides are treated the same way: one composed form of each letter, lowercase, no punctuation, tidy spacing. A short list of Nigerian spelling variants, such as wetin and wettin, counts as one word, so a correct word is not penalised for a spelling choice.
We score against public recordings that come with a transcript written by people, so nobody has to take our word for the reference. FLEURS, from Google, covers 102 languages with the same sentences in each. NCHLT, from South Africa's SADiLaR, covers four of ours. BibleTTS Asante Twi covers Twi. For FLEURS we score 250 recordings from the start of the test file, or all of them when there are fewer.
| Language | Test set | Recordings | Statement |
|---|---|---|---|
| Afrikaans | FLEURS | 250 | Word error rate 21.1% on read speech, FLEURS test set, measured Sep 2026. |
| Amharic | FLEURS | 250 | Word error rate 24.3% on read speech, FLEURS test set, measured Sep 2026. |
| Arabic | FLEURS (Egyptian Arabic) | 221 | Word error rate 13.7% on read speech, FLEURS test set, measured Sep 2026. |
| French | FLEURS (Parisian French) | 222 | Word error rate 5.3% on read speech, FLEURS test set, measured Sep 2026. |
| Hausa | FLEURS | 250 | Word error rate 31.8% on read speech, FLEURS test set, measured Sep 2026. |
| Igbo | FLEURS | 250 | Word error rate 77.4% on read speech, FLEURS test set, measured Sep 2026. |
| isiXhosa | FLEURS | 250 | Word error rate 52.2% on read speech, FLEURS test set, measured Sep 2026. |
| isiZulu | FLEURS | 250 | Word error rate 39.6% on read speech, FLEURS test set, measured Sep 2026. |
| Lingala | FLEURS | 250 | Word error rate 37.5% on read speech, FLEURS test set, measured Sep 2026. |
| Oromo | FLEURS | 41 | Word error rate 79.7% on read speech, FLEURS test set, measured Sep 2026. |
| Portuguese | FLEURS (Brazilian Portuguese) | 215 | Word error rate 3.5% on read speech, FLEURS test set, measured Sep 2026. |
| Sesotho | NCHLT | 250 | Word error rate 69.4% on read speech, NCHLT test set, measured Sep 2026. |
| Setswana | NCHLT | 250 | Word error rate 62.2% on read speech, NCHLT test set, measured Sep 2026. |
| Shona | FLEURS | 250 | Word error rate 47.3% on read speech, FLEURS test set, measured Sep 2026. |
| Somali | FLEURS | 250 | Word error rate 49.5% on read speech, FLEURS test set, measured Sep 2026. |
| Spanish | FLEURS (Latin-American Spanish) | 199 | Word error rate 2.2% on read speech, FLEURS test set, measured Sep 2026. |
| Swahili | FLEURS | 250 | Word error rate 14.0% on read speech, FLEURS test set, measured Sep 2026. |
| Tshivenḓa | NCHLT | 252 | Word error rate 74.0% on read speech, NCHLT test set, measured Sep 2026. |
| Twi | BibleTTS (one narrator) | 250 | Not yet measured on a public test set. |
| Wolof | FLEURS | 250 | Word error rate 58.0% on read speech, FLEURS test set, measured Sep 2026. |
| Xitsonga | NCHLT | 250 | Word error rate 71.1% on read speech, NCHLT test set, measured Sep 2026. |
| Yoruba | FLEURS | 250 | Word error rate 68.3% on read speech, FLEURS test set, measured Sep 2026. |
| English | None | None | Not yet measured on a public test set. |
| Nigerian Pidgin | None | None | Not yet measured on a public test set. |
The largest open collection of Yoruba, Hausa and Igbo speech we know of is licensed for non-commercial use only, so we do not score against it.
Each figure comes with a 95 per cent interval, built by resampling the test recordings many times. We resample whole sentences or whole speakers, because recordings from the same sentence or speaker are not independent. In FLEURS several people read the same sentence, so Igbo's 250 recordings are 88 different sentences. Where no grouping works, the interval is too narrow to trust. Oromo has 41 recordings and Twi is one narrator, so both carry a warning beside the figure.
Some languages lose a lot of the score to marks on letters, not to misheard words. For those we also show the same recordings with accents, tone and vowel marks removed from both sides, but only when that moves the result by three points or more. Yoruba and Igbo are scored after the engine puts tone marks back. Hausa is not restored, because in testing it made transcripts worse.
Apart from English and Nigerian Pidgin, which have no figure yet, a language opens only once it has a measured figure, and one we tested and found not good enough stays closed. Every figure on this site is read from one table, copied from the written record of the test run. An automated check fails our build if the two disagree. Figures were measured in September 2026.
The share of words that differ from a human-written transcript, so lower is better. See what word error rate means.
No. Accuracy is often quoted as 100 minus the error rate, but that breaks when a transcript adds many words, because the rate can pass 100. We report the error rate and do not convert it.
It depends on the language and the recording. Find your language in the table, read the statement, then test your own audio.
Send Autrans into your next meeting, use it in the room, or upload a recording. Your first 30 minutes are free, given once, and no credit card is needed to start.
Last checked 21 October 2026. Spotted a mistake? Write to hello@autrans.online.