Building a Production Greek-English Speech Recognizer

Building a Production Greek-English Speech Recognizer. By Christos Petrocheilos and five others, from the Sophea AI Lab at Kiefer. Published on the eleventh of September. No code repository is linked to it.

Sophea is a production bilingual Greek-English automatic speech recognition system, and the report describes the multi-month engineering program that built it. Its central finding is a negative one. The system was evaluated against nine production gates, covering word error rate in Greek and in English, language identification, and hallucinations on non-speech audio. Across twenty- three training iterations and two model architectures, no training-data composition passed all nine gates at the same time. Nine gates had to hold at once, and the search for a single data mix that satisfied them all is the thread running through the whole report.

The tension behind that result is concrete. Meeting the Greek target for noisy environments required about one thousand five hundred steps of dense domain exposure. Preserving English language identification tolerated only about two hundred fifty steps. A rebalanced mix stretched that to about one thousand two hundred fifty steps, but the rebalancing reduced Greek accuracy. Put side by side, the Greek requirement needed roughly six times more exposure than English language identification could survive.

Several pieces of engineering moved the numbers along the way. A six-stage data pipeline calibrated its audio-quality filter against in-domain anchors, and that calibration cut the share of scored Greek audio being discarded from ninety-eight point seven percent to ten point six percent. Separately, a pre- registered ablation isolated a hallucination defect to a single training-data package.

What did clear every gate was a combination of models. A three-model ROVER ensemble raised gate coverage from between four and seven of nine, for the individual models, to all nine. It also reduced word error rate on overlapping speech from fifty-three point three five percent to thirty-seven point eight seven percent, a twenty-nine percent relative improvement.

A different construction, a learned per-clip arbiter over two models, is listed as a preview entry on the public Open ASR Leaderboard. It shows four point two six percent average word error rate across eight public English test sets, and reaches twenty-five point eight eight percent on live Greek noisy-environment traffic.

The report is candid about its own limits and missteps. It documents five cases in which a measurement tool produced a plausible but incorrect result, and seven substantial approaches that were evaluated but not shipped. No model weights and no training data are released. What is published is methodology and quantitative results only. The finding concerns one language pair, Greek and English, measured against one set of production gates, with two model architectures tried.

The numbers describe production targets pulling against each other inside a single model's training. More Greek domain exposure helped the Greek target and hurt English language identification, and no data mix across twenty-three iterations resolved the pull. Meeting all nine gates together took several models rather than one. The pipeline fix, the ablation and the ensemble each answered a specific failure. None of them made a single training mix sufficient.