The Actual State of Machine Translation

I’ve seen a lot of commentary recently on the state of AI translation, and there’s a perspective sorely missing: anyone actually familiar with the industry! I used to work in it, and I still have a lot of friends who are full-time professional translators. I also still maintain a sizable amount of open-source tooling for human translation and subtitling.

Today, I read an ICML 2026 keynote by Arvind Narayanan (AI professor at Princeton), which included the following:

Translation is a more extreme example [than the radiology one]¹, which kind of blows my mind. AI was almost at human parity nearly a decade ago. Yet the employment of human translators has remained more or less stable, and is projected to remain stable over the next decade. There are many reasons for that, but one reason is that there’s really no ceiling to the amount of things you can translate, and the number of different languages you can translate them into.

This statement is wildly wrong on multiple levels. For starters, it is unquestionably false that AI was almost at human parity a decade ago. I don’t know if Arvind knows multiple languages, and I’m not familiar with the benchmarking situation for long-form translation², but he has certainly never worked in the localization industry.

The language pair I’m most familiar with is Japanese to English. For that pair, machine translation (MTL) was far, far worse than the median human translator; it wasn’t even close. As a consequence, the continued employment of human translators was a no-brainer – there wasn’t a viable alternative! You’d either get mistakes in practically every line, or you involved a human.

There was some use of “machine-assisted translation” on the low end in the industry, but it was mostly just an excuse to cut wages. Actual translators on those projects more or less ignored the machine-translated portion, or spent more time trying to clean it up than it would take to do it from scratch. It wasn’t a meaningful productivity improvement, so it’s not surprising that it had a minimal impact on the labor market. Confident claims about future employment based on this period are nonsensical, as capabilities have dramatically changed.³

The second thing he mentions is that there’s no ceiling to the amount of things you can translate, and many languages to translate into. While I’m skeptical of this claim in some respects⁴, let’s take it for granted: even so, there’s no reason to involve a human if the work can be done at higher quality for a hundredth the price. And unfortunately for translators, that is increasingly the situation we are in.

As it stands, LLMs are not particularly great at writing prose from scratch that is pleasant to read, at least to me. The Claudisms are inescapable, the figurative language is florid but nonsensical, the environments are bizarrely inconsistent, and so on. Direct translation work, however, seems to be an exception to this, likely due to the constraints imposed by the task. The models were previously not smart enough to translate complex prose accurately, but this has changed quite recently, at least for Japanese to English. As it stands today, I feel confident in saying that Astra with a proper harness is already in the top 10% of professional human Japanese to English translators at anything other than long-form text, and long-form text coherence for something this constrained is mostly just an engineering problem. As a consequence, I’m deeply skeptical of any claim that employment of human translators will remain robust over the next five years. Japanese to English isn’t a particularly favorable language pair, so I’d expect every other major language to be of comparable quality or reach it shortly.

Let’s go through some specific examples. I’m going to discuss a contemporary Japanese novel, 黄色い家 (localized as Sisters in Yellow).

Sisters in Yellow was translated by a pair of university professors, one of whom is a native Japanese speaker and both of whom have a Ph.D. in Japanese literature. It is a fairly competent translation by human standards. If you’re curious about the particulars, I’ve included an auxiliary post here written by a translator friend of mine. Below, I quote the conclusion in full:

Taken as a whole, the Sisters in Yellow translation is a fairly competent human translation that makes relatively few major errors that we would expect to see from an inexperienced language learner or writer. Reading this translation will come reasonably close to reading the original work in another language.

However, even with two translators (and one Japanese native translator with a B.A. in English literature and a Ph.D. in Japanese literature, at that), there were about fifty grievous errors, a range of five hundred to one thousand more minor errors, and three entire paragraphs cut. That leaves us with an inaccuracy rate between 4.3% and 8.6% -- taken harshly, one may say that around 1 in 12 sentences have some manner of problem or omission. That may come across like a staggering amount for someone who simply wants to read a translation and would be bothered by a dozen problems, let alone a thousand, but surprisingly this still puts the work near the top in terms of translation quality. A truly inexperienced translator duo would have put out work with 1,000 major errors rather than minor, and it is very common for people to read and love translations like those.

Machine translation powered by cutting edge LLM models, however, gives us something entirely different. It, well, obviously it translated the omitted passages so that’s just an immediate plus, but it avoided every single one of the grievous errors and produced accurate translations. Although one could probably quibble about the exact style of the prose, the focus of this analysis is accuracy; style is more subjective, and a human editor/proofreader (aka not a translator) is more than equipped to handle any prose oddities or what have you. More could be said on that subject but it would distract from the point.

The point is that a fine human translation put together by two lauded experts was fairly rough beneath the surface; I know I certainly would not want to read a book where every couple of paragraphs I am faced with an inversion of meaning or a just straight-up inaccurate translation. MTL, meanwhile, with its robotic devotion to recreating the original meaning to the best of its increasingly inhumane abilities, nailed every line that the human translators fumbled. Its accuracy rate clears 99% with fewer than 1% of lines containing errors.

Modern LLM MTL fumbles in places humans are less liable to. MTL by and large won’t misread names, numbers, or anything else that enters its maw as pure data, in contrast to humans just having slips of the mind and such, but instead it tends to fumble actually translating puns instead of just bringing the literal words over. There’s a pun/reference that goes like まさかまさかの坂こえて, with the common language wordplay being that the word for “no way” (masaka) shares a sound with the word for “hill”, so it goes masaka masaka no saka blah blah, and the MTL just gave “crossing over the slope of no-way, no-way,” which is readable but doesn’t preserve the wordplay.

All in all, even the best human translators tend to slip up and fumble lines over the course of a sizable project like translating a novel, such that even very qualified and skilled ones end up letting errors creep into 4.3% or more of lines. MTL, meanwhile, is at 1% error rate and that is lowering with each new release. It may be that humans have the edge in a very specific subset of moments like translating puns or localizing jokes, but personally speaking I find myself increasingly trusting LLM output more than human output in a vacuum.

A novel translated by a consistent pair of translators represents a best-case scenario. For most translations, especially in other mediums such as games, the interests of time, cost, and leak prevention mean the work is split up between many translators in far worse conditions. Often, translators (especially at agencies) are given text to translate without the context necessary to do a good job or maintain consistency between each other, and are never meaningfully checked for accuracy. In games, for example, this could mean receiving a spreadsheet of NPC lines without access to or info about the game itself; for television, a script without the video (or with a 240p greyscale watermarked video), etc. This obviously puts the humans at a disadvantage. With AI translation, the work can be fed fairly last-minute into a translation system, complete with available context, and be done promptly and, presumably, with no chance of a leak. There are still some issues with consistency over long-form text, but I believe this to mostly be a tooling issue that no competent engineer has made a serious effort to solve, although even using a generic agentic harness without much further prompting seems to work pretty well.

This, I suspect, means translation agencies will go the way of the dodo, especially as open-source tooling proliferates. Tooling will likely be the blocker on a lot of this – the more text-forward and shorter the translated snippet, the faster I expect it to fall completely. Today, almost every aspect of the localization workflow is human-led. With the strength of modern models, even workflows that retain humans in the loop will do so mainly for consistency checks and image editing rather than the translation itself. So novels will likely be merely proofread by a monolingual human (if at all), video subtitling is not quite a solved problem but is mostly a matter of tooling given how good transcription and coding + vision models have gotten, manga is mostly automatable but may require human image editing for a bit longer, and games are mostly automatable (including the technical work) with playtesters/QC and potentially human image editors. There is dramatically less need for human translators themselves, and image editors and quality control are a lot cheaper, so there is less incentive to automate them out as quickly.

I also expect consumer translation of works to be far, far, far more common than it is today. Fan translation used to be a frankly insane task; you’d spend months or years translating a work for no compensation, purely for love of the subject. But nowadays, with almost no instructions, any third party can get a serviceable result. Additionally, more apps (e-readers, browsers, video players) will build in translation capabilities last-mile, even if they continue to be subpar due to time constraints on generation. When consumer translation is this easy and demand for it present, publishers will be forced to launch with a larger initial catalog of languages, creating even more coordination challenges and expenses, creating even more pressure to utilize AI themselves, and at a faster rate than most diffusion processes go.

There’s probably still work for academic translators, particularly skilled translators, and for prestige projects or comedy/pun-heavy works, though I’m unsure how much longer all those categories will last. But make no mistake: translators whose job primarily consists of receiving a contextless spreadsheet and filling it in will not be around much longer. We’ve finally reached the point where the warnings are true. The best options for continued relevance are leaning more heavily towards “localization” as a holistic process/overseeing the AI/handling challenging lines, editing, pivoting to auxiliary skills like typesetting, leaning more into bilingual marketing more generally, etc. There is also some room for continued work, at least for a while, due to things like model guardrails. In Sisters in Yellow, the model refused to work on the final chapter because it dealt with descriptions of credit card fraud (yes, really), which obviously is not a problem for a human but seems weird, albeit a straightforward fix for a lab.

I expect that most of the value of translation’s commodification will be assumed by adjacent parties in the value chain – the rightsholders themselves and international distributors/marketers. Additionally, the wealth of new available options is a clear consumer benefit. But wrapping back around, telling translators that their jobs will be safe is wildly irresponsible. They are the clear losers in this rebalancing, and while the technology wasn’t there until recently, it’s now only a matter of time. It fills me with great sadness to see multiple industries I’ve been a part of going the way of Mel.

---