Image AIs now spell almost flawlessly, but tiny dots and strokes still trip them up

Two years ago, text in AI images came out as alphabet soup. Today GPT Image and Nano Banana get short Japanese words right almost every time. We explain the techniques behind that leap and where the last errors hide. We also show why the popular leaderboards barely notice those errors, and why nobody is properly measuring German yet.

Analysis Nov 21, 2026 9 min read Original analysis by our editors
Illustrative image, AI-generated

What this is about

For a long time there was a rule of thumb: ask an AI for an image with text in it, and you get letters that only look like letters from a distance. That rule no longer holds. In a test by the font company AIFont, GPT Image 2.5 rendered 243 of 244 short Japanese words correctly, and Nano Banana 2.1 managed 239. A Japanese blogger on note.com had Nano Banana 2.1, GPT-Image-2.6 and GPT-Image-2 generate a total of 126 images with Japanese text. He found exactly one character error: an extra period in a price (“1.200 Yen” instead of “1200 Yen”).

This week, three vendors launched new models with text rendering as a selling point: Kling Image 4.0 (“up to 2000 characters”), Ideogram 4.6 in 4K, and Microsoft's MAI Image 2.7. This article makes two arguments. First, the remaining errors almost always sit in tiny marks that carry meaning. Second, the common benchmarks are bad at catching exactly those errors, and for German, almost nobody measures them at all.

How image AIs learned to write

Four developments made the leap possible.

1. Text encoders that see letters. The text encoder turns your prompt into numbers the image model can work with. Older encoders like T5 split words into tokens, or word fragments. Common words become a single building block, so the model never sees the letters inside them. A 2022 Google paper measured this. Around 87 percent of common English words consisted of a single T5 token. When asked to spell those words, T5 scored only 14 to 66 percent depending on model size. The byte-level ByT5 reached 95 to 98 percent. The same showed up in images: character-aware encoders were ahead by more than 30 percentage points on rare words. Glyph-ByT5 builds on this. It's a ByT5 encoder additionally trained to link text with its visual glyphs. According to the authors, it lifted SDXL's text accuracy “from below 20 percent to nearly 90 percent.”

2. Large language models as encoders. Today, entire language or vision-language models often take on this role. In Qwen-Image, according to the technical report, the 7-billion-parameter Qwen2.5-VL serves as the encoder in front of a 20-billion-parameter image model. Baidu's ERNIE-Image uses Ministral-3 with 3 billion parameters. Kling describes a “unity of understanding and generation” for Image 4.0 but gives no details.

3. Synthetic text images. The Qwen team generates training images programmatically in three stages: text on plain backgrounds, text on paper or wooden signs within scenes, and text in templates such as slides and app interfaces. According to the report, that's “about 5 percent” of the data. Training follows a curriculum, moving from single words to full paragraphs. That way, rare Chinese characters get deliberate coverage.

4. Reinforcement learning with OCR as the reward. In reinforcement learning, the model learns through rewards, a bit like a dog with treats. OCR is automatic text recognition. A program reads the text out of the generated image, and the closer it is to the requested text, the bigger the reward. In the Flow-GRPO paper, the reward measures how many characters you'd have to change to make the text correct. That raised SD3.5-M's accuracy from 59 to 92 percent. For the X-Omni model, two OCR programs, GOT-OCR2.0 and PaddleOCR, score the images together.

Our take: this fourth ingredient also explains the weaknesses. A model only gets as good as its grader looks closely.

Where the errors are today

The AIFont test, published on dev.to, is the most thorough independent comparison we found. Only the first image counted each time, and the prompt explicitly demanded “not adding or changing a single letter.”

Model Japanese (244 styles) Arabic (native-speaker verdict)
GPT Image 2.5 243/244 53/57
Nano Banana 2.1 239/244 55/57
Grok Imagine 2.0 226/244 42/57
Seedream 5.0 Pro 214/243 34/54
Qwen-Image 3.0 Pro 154/244 8/59

The errors follow a pattern:

  • Japanese dakuten. These are two small strokes that turn は (ha) into ば (ba). Grok Imagine wrote ばくはつ (explosion) as ばくはづ. Among the 141 incorrect Japanese images, AIFont counted 16 dakuten errors.
  • Arabic dots. Many Arabic letters share the same basic shape and differ only by their dots. With Qwen-Image, فخر (pride) became فجر (dawn): a different, real word that no spell checker will catch. For Grok, 6 of 15 errors were dot errors; for Seedream, 7 of 20.
  • Text from the prompt. Qwen-Image often wrote words from the style description into the image, such as ころんと next to the requested いちご (strawberry). That affected 26 of its 90 incorrect Japanese images. The note.com test found this error too. Given a terse prompt for a YouTube thumbnail, all three models added titles, slogans and made-up prices between 80 and 200 yen to all nine images. Precise instructions like “no other text” made it go away. A test of Kling Image 4.0 on eachlabs.ai saw the same thing: the model filled the edges of a poster “with its own fine print.”

Long texts reveal another limit. A tester on dev.to had Kling Image 4.0 generate a long product page. The top half was flawless. In the bottom half, most of the numbers in the size chart were missing, and one line sat on top of a mirrored copy of itself. Even Ideogram, which markets itself on clean typography, showed garbled banners (“FIESTA E LALA ALDEA”) in an official 4K sample image.

For Thai, there are no reliable measurements. A single hands-on test by Marketing Oops, with just one run per prompt, rated the Thai text from both Seedream models as illegible. In Thai, vowels and tone marks sit above, below and beside the base character. A Thai OCR shows how hard even reading such characters is: version 2.0.0 missed 128 of 159 tone marks on an old book page. That says nothing about image generators, but it shows how unreliable an automated grader can be with characters like these.

The problem with the graders

That brings us to the second argument. The best-known text benchmarks test only two languages and let an AI reader be the judge:

Benchmark Languages Who reads the text?
LongText-Bench (160 prompts) English, Chinese Qwen2.5-VL-7B
OneIG-Bench (200 text prompts) English, Chinese Qwen2.5-VL-7B (“e.g.”)
UltraText Bench (432 prompts) English, Chinese Q-Judger scoring model
CVTG-2K English OCR metrics

In OneIG-Bench, symbols and double spaces are stripped from the text that was read before it's compared. On top of that, the benchmarks are close to saturated. ERNIE-Image scores 0.973 out of 1 on LongText-Bench, and 0.980 on the English part.

AIFont's Arabic results show how often such graders get it wrong. First, an AI judge and an author who isn't a native Arabic speaker gave their verdicts. Then a native speaker reviewed every image blind, without model names and without knowing the earlier verdict. He disagreed on 23 of 291 images and resolved 34 of 41 cases that had previously been left open.

Do the math and 23 of 291 is just under 8 percent. So the grader was wrong more often than the best models make mistakes in Arabic (GPT Image 7 percent, Nano Banana 3.5 percent). No native speaker has double-checked the Japanese results yet. The author himself expects some verdicts to change.

Researchers know about the problem too. The paper on the MULTITEXTEDIT benchmark explicitly names missing diacritics as errors that coarse text comparisons overlook. Diacritics are extra marks like dots, accents or umlaut dots. That benchmark's own AI judge agreed only moderately with native speakers (kappa 0.76). And there's another issue with training rewards: a metric like the number of characters to change scores ば instead of は the same as any other wrong letter. A four-character word with one wrong dakuten still gets 75 percent of the points. Our take: if even the grader overlooks a missing stroke, the model never learns to take it seriously.

And German?

Ä, ö, ü and ß are letters with exactly these kinds of meaning-bearing dots and special forms. The evidence is scattered:

  • AIFont's seven-language comparison includes German. But it lumps German together with English, Spanish and Portuguese into a single “Latin” row. All models scored above 90 percent there, GPT Image 235 of 236. Which German words were tested, and whether they included umlauts, isn't stated.
  • A March 2025 test by ai-imagelab.de with Gemini 2.0 Flash found that umlauts worked, and that “the AI only fails at the ‘ß’.”
  • In September 2026, the Austrian site ki-kompass.at wrote about Ideogram that with umlauts and long compound words, “errors do occur regularly.”
  • Kling lists seven writing languages for Image 4.0. German isn't one of them.

On Reddit, Hacker News and in Hugging Face discussions, we found no systematic reports on umlauts in Qwen-Image, Z-Image or Flux.

We have not yet run our own test for this article, so it contains no measurements of our own. To make such a test comparable, we propose the following protocol, which anyone can replicate:

  • 20 terms in four groups: single umlauts (“Grüße aus Köln”, “Bäckerei Müller”), ß and capital ẞ (“Straße”, “GROẞE PAUSE”), long compound words (“Straßenbahnhaltestelle”, “Rindfleischetikettierung”), and traps where a missing dot produces a different real word (“schön”/“schon”, “Bären”/“Baren”).
  • Three runs per model, with no cherry-picking afterward, each with the addition “kein anderer Text” (no other text).
  • Humans check the results and zoom in, rather than letting an OCR judge. Count ss instead of ß, ue instead of ü, and any extra inserted text as errors too.

The bigger picture

Text in AI images is largely solved for English, Chinese and, apparently, Japanese when it comes to short words. GPT Image and Nano Banana are clearly ahead in independent tests. Open models like Qwen-Image lag far behind in Arabic. The remaining errors are treacherous because they often produce other real words, and those are exactly the errors automated graders are most likely to miss.

Our verdict: when vendors advertise “perfect text,” they usually mean English and Chinese, measured by an AI reader. So for German posters, menus or labels, a simple rule applies for now: read every word yourself, pay special attention to umlaut dots and ß, and put “no other text” in your prompt.

#image generation#typography#benchmarks#ocr#multilingual

Ask the AI assistant

Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.

Connecting …

The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.

Comments

What do you think? Already tried it? Write it here, no sign-up needed.

  1. Loading comments …

Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.