Two people, ten dollars, second in the world: a first-person account of placing second of 39 systems on a public Arabic speech-recognition leaderboard, ahead of OpenAI, Google, Meta, Nvidia, Mistral and Microsoft, from a simple home lab, with no training run and less than ten dollars of GPU time
Beginner ~14 min read
On 20 October 2026 a maintainer of the Open Universal Arabic ASR Leaderboard closed our GitHub issue with one line: the results had been added.[1] I opened the board. Our entry, Sygma ASR Arabic, sat in second place out of 39 speech-recognition systems, with an average word error rate of 23.01 per cent.[2] Below it were models from OpenAI, Google, Meta, Nvidia, Mistral, Alibaba's Qwen team and Microsoft. Above it was exactly one system, Jais-ASR, 0.19 points ahead.
There were two of us: my son Basel, a student at the University of Petra, and me, working from a simple home lab. No grant, no cluster, no team. The GPU time for the whole study, every experiment that worked and every one that did not, cost less than ten dollars on a rented card.[3] From the first run to the official listing took under two weeks.
Here is the sentence that matters most, and I want it in the second paragraph, not the last: we did not train a model. We did not collect a thousand hours of audio, and we did not fine-tune anything that ended up in the result. We took an open model that Cohere had released in July under an Apache licence, left every weight exactly as it was, and changed only how the model chooses among the sentences it already considers.[4] That change was worth more than a point of error rate, and a point was the distance between fourth and second.
This article is about why that was possible, what went wrong on the way, and what it means for anyone who has been told that AI is now a game for companies with billions. It is told in the first person because I was there, and in the plural where my son was too.
Ten years ago this story could not have happened. A competitive speech recogniser needed thousands of hours of transcribed audio, a team to clean it, and weeks on hardware that a university department might own and an individual would not. Three things have changed, and each one is a lever that a single person can pull.
Open weights. The base of our system is Cohere Transcribe Arabic, a two-billion-parameter model released on 7 July 2026 with its weights on Hugging Face under Apache-2.0.[4][5] It is not alone. OpenAI's Whisper has been open since 2022.[6] Meta's omnilingual models, Alibaba's Qwen3-ASR, Mistral's Voxtral and Nvidia's Conformer and Parakeet families are all downloadable. The expensive part of the work, the part that needs the data and the cluster, is finished and given away. What is left is everything a company does after training, and that part is cheap.
Public benchmarks with public rules. The leaderboard we entered is run by the ELM Research Center and described in an Interspeech 2025 paper.[7] It evaluates every model on six test sets covering Saudi television, crowd-sourced read speech from 25 dialects, YouTube audio clean and noisy, broadcast news and eight Maghrebi and Levantine dialects.[8] The evaluation code is on GitHub. You submit by opening an issue. The maintainers run your system themselves and publish the number. Nobody asks where you work. The same arena, the same referee, for a two-person entry and for Google.
Compute by the hour. We rented an RTX A6000 with 48 gigabytes of memory for 61 cents an hour.[3] Running all six test sets through the base model takes about 25 minutes; our eight-candidate decoding takes about 80. The starting credit on the account was $9.34, and it covered the study.
None of these levers is a secret. The gate in the illustration above is open and the key is on the grass. The question is why more people do not walk through, and I think the answer is that they believe the only move worth making is the expensive one: train something bigger. The rest of this article is about a different move.
A speech recogniser of this kind works in two stages. An encoder listens to the audio and turns it into a long sequence of numbers. A decoder then writes the transcript one token at a time, and at every step it is choosing among possibilities. The standard shortcut is greedy decoding: always take the single most probable next token and never look back. The better-known alternative is beam search: keep the eight most probable partial sentences alive, extend each, and return the best complete one at the end.
Beam search has a known flaw. The score of a sentence is the sum of the log-probabilities of its tokens, and every token adds a negative number, so longer sentences score worse simply for being longer. Given a six-word sentence and its last three words, the model will often prefer the three words even when it heard all six. On our test sets that preference showed up as deletions: plain beam search made 36 per cent more deletions than greedy decoding and was, on average, slightly worse.[9]
The fix is almost embarrassingly small. Add a reward of one point per word to each candidate's score before choosing. That is it. The long sentence the model had already found now wins. In one real clip from our tuning data, the reference had six words, the top candidate had three, and the error rate on that clip went from 50 per cent to zero because the full sentence was second on the list the whole time.[9] Across the six benchmarks the reward removed about 12,000 deletions and added about 3,500 insertions, and the average error rate fell from 24.16 to 23.05 per cent in our pipeline.
We added a second, smaller piece: a filter that throws away a candidate when it is obviously broken, for example one word repeated until the output is half that word, or a run of underscores. The filter needs no reference transcript and changed the average by a hundredth of a point, but it mattered for a different reason that I will come to.
Our method does not improve hearing. It improves selection.
I want to be clear about the size of this idea. Length bias in beam search has been known for years; length normalisation is the textbook fix and we tested it (it helps, but less). Nobody will win a best-paper award for a per-word bonus. What we did was notice that a model which a well-funded lab had just released still shipped with the simplest decoding, measure exactly how much that cost on the public benchmark, and fix it with the public evaluation code before anyone else did. That is the kind of work an individual is better placed to do than a company, and the next section is why.
The decoding idea was not the plan. The plan, in August, was the expensive move: fine-tune the Cohere model on Jordanian speech so that it would understand my own dialect better. We built a consent form and a recording web app for students, downloaded the 82-gigabyte MASC corpus, and filtered out 34 hours of Jordanian YouTube audio with subtitles.
Three runs later we had learned three things. The first run made the model worse on Jordanian test clips, from 18.14 to 22.40 per cent. The third run, after fixes, reached 17.88, which against 18.14 is noise. The bug behind the bad runs is worth recording because it will bite someone else: the encoder contains BatchNorm layers, and freezing the encoder's weights does not freeze BatchNorm's running statistics. We trained with the model in training mode, the statistics drifted, and a checkpoint that scored 11.9 in memory scored 17.6 after reloading, because the saved decoder no longer matched the original encoder.[10] The second lesson was the data: in a sample we checked, a third of the subtitle labels disagreed with the audio. The third was the rules: MASC is one of the leaderboard's test corpora, so a model trained on it would not have been accepted anyway.[8]
We tried two more expensive-looking ideas and both gave nothing. A 4-gram language model trained on 23 million Najdi sentences chose a weight of zero when tuned: the eight candidates differ mostly by spelling and punctuation, which a word-level language model cannot judge. Rescoring with a small neural language model, Qwen3-1.7B, moved one benchmark up and one down and netted zero.[10]
Then there was the claim we withdrew. Early on, our system appeared to beat the then-leader, Audar, on Saudi speech by a wide margin. It did not. Audar's raw output contained about forty clips where the decoder had fallen into a loop, repeating a phrase until it hit the length limit, and those forty clips cost it 7.6 points on that test set. When we gave Audar's output the same breakdown filter we gave ours, its average dropped from 26.37 to 24.05 and it was clearly better than us on clean Saudi speech.[9] We corrected the experiment log, rewrote the comparison, and reported both numbers. That is what the filter was really for: not to win, but to compare fairly.
Smaller things went wrong too. A language-model build ran out of memory at 62 gigabytes. A tuning script weighted an eight-clip set equal to a six-hundred-clip set. A file pattern silently skipped two files. A GPU sat idle for an hour waiting for a marker that would never appear. I list them because the finished result looks inevitable in a leaderboard row, and it was not.
The official table, as read on 3 November 2026, is below for the top four systems and the base model we built on. Lower is better; the metric is word error rate in per cent. The highlighted row is ours.[2]
| System | SADA | Common Voice | MASC clean | MASC noisy | MGB-2 | Casablanca | Average |
|---|---|---|---|---|---|---|---|
| 1. IFM Jais-ASR | 30.52 | 8.59 | 16.14 | 25.47 | 9.60 | 46.61 | 22.82 |
| 2. Sygma ASR Arabic (ours) | 33.63 | 5.24 | 15.92 | 25.10 | 13.48 | 44.67 | 23.01 |
| 3. Audar-ASR-V1-Turbo | 28.92 | 8.09 | 16.73 | 27.19 | 11.08 | 47.02 | 23.17 |
| 4. Cohere Transcribe Arabic (base) | 36.83 | 5.27 | 16.35 | 25.65 | 15.33 | 45.68 | 24.19 |
Official leaderboard values, Open Universal Arabic ASR Leaderboard, read 3 November 2026. Bold marks the lowest error on each test set among all 39 systems. The Cohere row is the unchanged base model with its default decoding.
Three honest readings of that table. First, our entry has the lowest error of all 39 systems on two of the six sets, Common Voice and Casablanca, and the lowest character error rate on Casablanca as well. On MASC clean and noisy it is second, behind an earlier Cohere model that is much weaker on the other four sets. Second, it is not the best on Saudi speech: Audar beats us there by almost five points, and Jais-ASR beats us on broadcast news by almost four. Our method sharpened the selection; it did not change what the model can hear. Third, the gap to first place is 0.19 points, and Jais-ASR was added to the board on 18 October, nine days after we submitted and the day before our row appeared. Had the board been updated on the day we submitted, we would have been first for those nine days. Rankings on a live board are weather, not climate.
The second figure is the one I would show a sceptic. It is the base model, unchanged, with its default decoding and with ours, on each of the six test sets, in our own pipeline where both runs are identical except for the decoding. The gain is on every set. It is largest on Saudi television speech and broadcast news, where the deleted words were.
Cohere has excellent engineers. The length bias is in every textbook. Why did a two-person entry get to fix it on the public board?
Incentives. A lab is rewarded for shipping models. Its launch post says "most accurate open-source Arabic model" and moves on to the next training run.[4] A decoding patch that gains a point does not make a launch post. For us, a point was the whole project.
Attention is the scarce resource, not compute. The one thing we had more of than any company was the willingness to stare at 45,845 test transcripts and ask why the model dropped words. The question cost nothing. It took a father and son a weekend. No one at a lab is assigned to it, because the tooling to run the public benchmark end to end is already a day's work and the expected reward is a footnote.
The arena is public and rewards exactly this. The leaderboard's own guidance says it welcomes entries that bring an architecture not already on the board and that avoid training on the test corpora.[8] That is a rule written for people who cannot buy their way up with in-domain data. We tuned one number on 608 clips from outside every benchmark and said so. The maintainers checked, ran our code, and published the result in eleven days.
Open weights move the frontier to the edges. Once the model is public, the lab's advantage ends at the weights file. Everything after it, decoding, filtering, chunking, evaluation, honest reporting, is engineering that one careful person can do as well as a hundred. I would go further: a hundred people in a company will do it worse, because none of them owns the whole pipeline the way we did.
Strip the story of its numbers and what is left is two things that cost nothing to rent and cannot be rented: understanding of how the machine actually works, and the creativity to use that understanding in a direction nobody was looking.
Understanding came first, and it was not the glamorous kind. We had to know that a decoder scores a sentence by adding negative numbers, one per token, and that this arithmetic alone punishes length. We had to know that beam search keeps eight candidates and throws seven away, so the right answer could be sitting in the list unused. We had to know why a frozen encoder can still change under training, because BatchNorm keeps statistics that are not weights. None of that is advanced. All of it is the kind of thing a student learns in a course and forgets by the exam, because it never seems to matter. Here it was the whole game: every one of our findings, positive and negative, follows from a textbook fact taken seriously.
Creativity was the second half, and it looked less like inspiration than like permission. The permission to treat a model released by a well-funded lab as unfinished. The permission to ask a naive question, "why is the model dropping words?", and to answer it by reading the candidates instead of training something. The permission to spend the budget on looking rather than on compute. The reward itself, one point per word, is almost a joke of an idea. The creative act was not inventing it; it was noticing that the expensive fixes everyone reaches for, more data, a language model, a bigger model, were the wrong tools for a selection problem, and that the right tool was one line long.
The two depend on each other. Without the understanding, the creative move is a guess, and we would not have known why beam search alone made things worse or why the language models could not help. Without the creativity, the understanding stays in the lecture notes. A company has both in abundance, spread across a hundred people. What it rarely has is both in the same head at the same time, pointed at a problem small enough to be solved on a weekend. That is the one advantage a person in a home lab has, and it is the only one that matters.
Compute can be rented by the hour. Understanding cannot, and creativity has no price list.
What follows is not advice I read somewhere. It is the list of the things we actually did, in the order that would have saved us time.
Total cost of the eight steps: under ten dollars of GPU rental, two laptops in a home lab, and about two weeks in which the father was also teaching and the son was also studying.
It does not mean a bedroom can produce a foundation model. The two-billion-parameter encoder that does the listening was trained by Cohere on data and hardware we will never see, and our ten dollars sits on top of that investment. The right picture is not David and Goliath. It is a mechanic who tuned a car the factory shipped, and beat the factory's own driver on a public track.
It does not mean the rank will hold. Jais-ASR appeared the day before our row did and took first place. The leaderboard's maintainers have already said they are cleaning the test sets and adding a standard length limit for every model, which will shift several rows.[1] If you need your number to be permanent, do not put it on a live board.
It does not mean decoding tricks are enough. On Saudi speech the ceiling of our method is visible: a model trained on more Saudi data beats us by five points, and no amount of re-ranking closes that. Selection cannot invent what the encoder did not hear.
And it does not mean it was easy. Most of the bench was failure. The thing that worked was small, and it only looked obvious afterwards.
For most of the history of this field, the answer to "can one person compete?" was no, because the scarce resources were data and compute and both were owned. In 2026 the scarce resource is different. It is the willingness to take a public model, a public benchmark and a rented GPU, look carefully at where the model fails, change one thing, and report the result with the failures attached. Companies are structurally bad at that last part. Individuals can be very good at it.
Second of 39 is not the point of the story, and by the time you read this the number may be different. The point is the row above ours and the rows below. Above: one team that trained its own model. Below: the best speech systems from the richest companies on earth, every one of them beaten on a public Arabic benchmark by a decoding change that costs a page of code. The gate is open. The key is on the grass.
Enjoyed this article?Follow the page for new AI articles and daily space news.
Follow on FacebookAll links accessed 3 November 2026. Leaderboard values are as displayed on that date; the board is updated continuously. Illustrations were generated with Google Gemini for this article.
