You have probably had this happen: a machine translation reads perfectly smoothly, yet something feels off, and when you check the original you find a negation has quietly moved. Smooth, and wrong — that is the failure mode peculiar to machine translation today.
Sixty years ago it failed differently. Output was stiff and wrong, and you could see at a glance where. What happened in between is rarely told as one story: people spent thirty years hand-writing grammar rules and lost to people who ignored grammar and counted instead — and those counters were then driven for fifteen years by a scoring script that understands nothing.
Start with how small the machine was. On 7 January 1954, Georgetown University and IBM gave a public demonstration in New York: sixty-odd Russian sentences translated automatically into English, on a vocabulary of 250 words and six grammar rules. Coverage was enormous, and the general expectation was that the job would be finished within three to five years.
For the next thirty years the real difficulty was not vocabulary. Every new rule had to fight the rules already written — you add one saying that Russian пол (pol) means "floor" rather than "sex" in this context, and it fires in twenty other sentences that were already correct. The longer the table, the more expensive each edit.
In 1960 Yehoshua Bar-Hillel, the first person in the world employed full-time on machine translation, pushed this to the wall. He offered a short passage: Little John was looking for his toy box. Finally he found it. The box was in the pen. English "pen" has two senses, a writing implement and an enclosure a small child plays in. Choosing between them is not a matter of grammar or of dictionaries; it depends on your knowing that a writing pen is smaller than a toy box and a playpen is larger. His conclusion was blunt: fully automatic high-quality translation would require the machine to hold not a dictionary but an encyclopedia — which he considered utterly chimerical.
There is a more everyday difficulty too: two languages need not line up at the level of the sentence. English says She swam across the river, with manner of motion (swimming) inside the verb and direction handed to the small word "across". The habitual Spanish is Cruzó el río nadando, literally "she crossed the river swimming" — direction moves into the verb and manner is squeezed into a participle. Leonard Talmy sorted languages into these two packaging strategies. It is a tendency rather than a law, and Spanish permits the English pattern in some contexts — which for a rule writer is worse news than a law: you must hand-write the rewriting rule and also hand-write when it does not apply.
In 1966 a United States review committee published the report now known as ALPAC. Its conclusion was not that the approach was wrong but that it was not worth the money: machine output still had to be reworked by a person from scratch, which cost no less than hiring a translator outright. In the report's own words, there was no useful machine translation and no near-term prospect of any. Funding stopped for close to twenty years.
Around 1990 a group at IBM's Watson Research Center changed the question. Instead of asking what an English sentence means, they asked which French sentence is most likely to be its translation. Meaning is stepped around, and what remains can be counted.
Counted from what? From text already paired up. The proceedings of the Canadian Parliament must by law be published in English and French at once, sentence by sentence, and there is a great deal of it: the version later put to use holds 3,603,971 sentence pairs, close to a hundred million words on each side. Nobody assembled it for research. It is what a country's bilingual policy leaves lying around.
What alignment looks like is visible in one thoroughly ordinary pair. English I do not speak French, French Je ne parle pas français. Three words map cleanly; the trouble is the other two. English has one "not" answering to two French words, "ne" and "pas", with the verb sitting between them, and English "do" answers to nothing at all. A rule writer must handle each case explicitly. A counting system writes nothing — after a few hundred thousand sentences containing "not", the fact that "ne" and "pas" co-occur implausibly often is simply there to be read off. To book-keep the one-to-many case, the IBM models gave every word a number called its fertility: not a concept borrowed from linguistics, but an accounting device invented so the counting could continue.
The shift was later compressed into a widely repeated quip: every time I fire a linguist, the system's recognition rate goes up. It came from Frederick Jelinek on the speech-recognition side; he disputed the wording himself, and decades of digging have never fixed the exact version. What the line obscures is that the knowledge was not removed — what was removed was certainty written down by hand. A statistical system still has to align, still has to segment; those judgements simply moved from "someone decided it should be this way" to "this occurred so many times in the data". Nor did it win quickly: through the 1990s and 2000s rule-based systems such as Systran stayed competitive on some language pairs.
In 2002 another group at IBM proposed a scoring method called BLEU, and its method is almost comically blunt: put a machine translation beside several human reference translations and count how many word sequences overlap — single words, pairs, triples — with a penalty to stop a system scoring well by keeping its output short. It understands nothing at all. It also runs in a second.
The consequence mattered more than the metric. Evaluating quality had meant days of human reading; afterwards you could change a parameter and know in half a minute whether it helped. How fast a field moves depends less on its supply of ideas than on how long verifying one takes, and this script cut that wait from days to seconds, so for fifteen years everyone climbed the same hill.
The cost surfaced quickly. In 2006 a group examined whether the score could be trusted: a BLEU gain does not mean real improvement, and real improvement does not guarantee a BLEU gain. Their telling comparison was that the rule-based Systran scored low on BLEU while holding its own on human ratings of adequacy and fluency — it chose different words that meant the same thing, and the score only recognises literal overlap. Something else has been confirmed repeatedly: the better the translation, the looser the correlation with human judgement. The metric is good at spotting bad output and poor at ranking good output.
Neural networks arrived from 2014, and in September 2016 Google replaced its production phrase-based system outright — at the time about eighteen million Chinese-to-English translations a day. What improved is real: sentences read smoothly, and long-distance ordering problems, such as a German subordinate clause driving its verb to the end, largely eased.
Three things did not improve. First, low resource: the method eats sentence pairs, and the overwhelming majority of the world's roughly seven thousand languages have almost none, so a 2022 model covering two hundred languages counted as a milestone. Second, hallucination: in 2018 people typed the word dog a dozen times over, asked for Maori into English, and received an apocalyptic prophecy — much of the available parallel text for such languages is scripture, and when the input is noise the model is still trained to emit something that sounds like a person, so it invents a passage. Third, the gap between two grammars: Turkish has one third-person pronoun, o, with no gender, but English forces a choice, and systems long rendered o bir doktor as "he is a doctor" and o bir hemşire as "she is a nurse" — the source never carried that information, the target demands the slot be filled, and the system can only guess the corpus majority. A rule-based system breaks into visible gibberish; a neural one breaks into fluent falsehood. That is where the smooth-and-wrong feeling from the opening comes from.
So why did hand-written grammars lose? Not because grammar is wrong. A rule can state that something is permissible; it cannot state that something is probable — a rule table has no column for "probably", and probability is exactly what every step of translation needs. Bar-Hillel's sentence is handled correctly most of the time today, not because the machine acquired an encyclopedia but because it has seen enough contexts in which boxes and enclosures occur together. Whether that counts as understanding depends on where you set the bar: if you want world knowledge that can be queried, questioned and corrected, it has none; if you want the same output on this sentence as from someone who understands, it delivers.
Bar-Hillel's argument does not depend on computing power: these judgements require world knowledge, and world knowledge cannot be enumerated as rules, so the purely rule-based route would most likely have stalled in the same place. One counterexample is worth remembering, though — where the domain is narrow and the sentence patterns fixed, as in weather bulletins, rule-based systems were usable at the time, and the Canadian English-French weather translation system ran for many years. The condition is that the world in question be small enough to finish writing down.
A safer reading is that meaning is still doing the work, only hidden inside someone else's behaviour. Those three and a half million sentence pairs were translated by people, and every pair has already passed through an act of understanding. The statistical system consumes the results of other people's understanding — which is also why it degrades immediately in domains nobody has translated.
No. Abundant parallel text usually means there was an administrative, religious or commercial demand for translation behind that language, which says nothing about its structure or expressive power. The Canadian parliamentary record exists as a by-product of a bilingual statute. Reading "machines translate it well" as "this language is somehow better" mistakes institutional history for a linguistic fact.