We trained for Latvian–Russian phone calls. Our detector caught 82% of Russian-speaking callers.

Our detector identified 82% of Russian-speaking callers, compared with 67% for the tested off-the-shelf Whisper build. That is a gain of about 15 percentage points on the same replay, at the same observed wrongful-switch rate on Latvian-speaking callers.

Both outcomes matter on a bilingual line. The system needs to recognise a caller speaking Russian without switching a caller who is speaking Latvian incorrectly. In this replay, both builds wrongly switched about one Latvian-speaking caller in 70.

These are recorded-call results from one line and one recording window. Live field performance and results on another customer’s line require their own measurements.

Our model classifies individual speech fragments. Our detector makes the language-switch decision for a caller. Fragment results describe the model; caller results describe whether a Russian-speaking caller would have been switched during a replay of the recorded call.

We evaluated recorded Latvian and Russian phone audio, with native speakers labelling the correct language. A separate synthetic-audio evaluation tested how the model rankings changed. Processing times were measured on the same MacBook Pro with an M5 Max chip. The AiLab models used a different serving setup and are excluded from the latency comparison.

At fragment level, we report the share of Russian speech fragments identified and the share of Latvian fragments wrongly flagged. Decision rules were selected under a strict error constraint on separate calls from those used to report the results. Each model’s observed error rate on the unseen calls is printed in the scorecard; those rates are not identical.

At caller level, we replay recorded calls and score whether a Russian-speaking caller would have been switched at any point. We also report how many Latvian-speaking callers would have been switched incorrectly. The 82% and 67% results use the same switch rule and have the same observed wrongful-switch rate. A wrongly flagged fragment and an incorrectly switched caller are different units.

Part one showed the gap between a language-support claim and a real phone call. Whisper small labelled 60% of the tested Latvian speech fragments as English and only 2.8% as Latvian. The dedicated language-ID models also struggled with the same audio.

A clean synthetic evaluation changed which model appeared best. VoxLingua107 identified 93% of Russian speech fragments on synthetic audio and 20% on real phone audio under the same evaluation constraint. The general models’ synthetic results overstated their real-audio performance by 2× to 4.6×.

We trained our own Latvian–Russian detection model on real telephony audio, at the bandwidth and codec a phone line delivers, with native speakers labelling the correct language. We then measured it on real phone audio too.

On the tested Latvian phone speech, our model labels roughly nine fragments in ten as Latvian. Whisper small labelled 2.8% as Latvian in part one. When our model gets a Latvian fragment wrong, it labels it Russian, the other language in the task.

Separately, the median score our model assigns to Latvian on Latvian speech is 0.90, compared with 0.0006 for Whisper small. The plot shows model scores. The label and detection results measure correctness; a score of 0.90 is not itself a measured 90% success rate.

The general multilingual models shown assign very low scores to Latvian on this audio. The AiLab Latvian transcription models behave differently: they assign high Latvian scores to almost everything on the phone audio, Russian speech included. They are discussed separately below.

Our model and Whisper small both identify 37% of Russian speech fragments at the reported evaluation point. Their observed false-positive rates on unseen Latvian fragments are 3.2% and 3.0%, respectively. The rounded recall result ties; it does not establish identical performance at every operating point.

Whisper medium and large-v3 identify more Russian fragments, at 51% and 52%. Their processing times on the test machine exceed the live-turn requirement of the setup we evaluated. The scorecard separates recall, errors and latency so the tradeoff is visible.

Our model processes a fragment in about 21 milliseconds, compared with about 0.7 seconds for Whisper small and 5.3 seconds for Whisper large-v3 on the same test machine. These are inference times per fragment, not the total time a caller waits for an answer.

Whisper small keeps up on a typical fragment in this setup, but its worst measured fragment took 29.5 seconds. Our model’s lower processing time gave us room to make the decision per caller. We measured that outcome separately in the recorded-call replay.

The matched comparison is 82% versus 67%, with both builds at about one incorrect switch per 70 Latvian-speaking callers. Our model alone, judged fragment by fragment, identifies 57% of Russian-speaking callers at a lower wrongful-switch rate, about one Latvian-speaking caller in 140.

The 57% bar has a different observed error rate, so it is not an equal-error comparison with the other two bars. All three results are shown below with their error rates. The 82% result belongs to our detector deciding per caller.

The University of Latvia’s Institute of Mathematics and Computer Science, AiLab, publishes openly licensed Latvian transcription models: AiLab-IMCS-UL/whisper-large-v3-lv-late-cv17 and AiLab-IMCS-UL/whisper-large-v3-lv-phono. They recognise accented Latvian as Latvian, a useful capability. They were built to transcribe Latvian; bilingual language detection is a separate task.

We evaluated them as bilingual detectors because a team building for this market may consider using them that way. On synthetic audio, lv-late-cv17 identifies about 40% of Russian speech fragments; on real phone audio, about 1.3%. For lv-phono, the results are about 23% and 1.0%. The synthetic results overstate real-audio detection by about 31× and 23×, respectively.

On the real phone audio, both models label almost everything Latvian, including Russian speech. This is a result for the detection task we evaluated, not a verdict on their Latvian transcription quality. The synthetic-versus-real difference shows why the evaluation channel matters; this comparison does not isolate which audio characteristic caused the drop.

A claim that a model is tuned for a language or a phone channel should come with a comparison on the calls the system will serve. These are the questions we would bring to that discussion.

The caller results cover one line and one recording window. They are replay measurements, not live outcomes or a forecast for another customer’s line. Decision rules were set on separate calls from those used to report the results.

The external caller-level comparison is one configuration of one Whisper tier under our switch rule. Larger Whisper tiers were not run per caller because their measured processing time exceeded our live-turn requirement in this setup. Whisper large-v3 remains the most accurate model we measured per fragment.

Nothing here claims a better speech model in general, superiority over every possible configuration, or a statistically significant caller-level difference. Neither article publishes results for commercial cloud APIs. Our model and detector are described through measured outcomes.