Benchmark winner gets cut: How South Korea picks its national AI champion

What's going on

South Korea wants its own AI foundation models so it depends less on American and Chinese systems. Since 2025, the science ministry, MSIT, has been pitting companies against each other in its "Sovereign AI Foundation Model" program (abbreviated in Korean as 독파모, "Dokpamo"). Teams that survive a round get GPUs, data funding and the label "K-AI company." Teams that lose are out.

In August 2026, the axe fell on the very team that performed best on the most important independent leaderboard: Motif Technologies with its model Motif 3. LG AI Research, SK Telecom and Upstage advanced. That raises a fundamental question: does a government competition like this measure performance, or mostly the impression a model leaves on its judges? We re-ran the published scoring scheme.

How the competition works

In Round 1, in January 2026, Naver Cloud and NC AI were eliminated. Naver actually ranked among the top four on points but failed the independence check because, according to Korean media, its model relied on vision encoders and weights from Alibaba's Qwen family. In February, the ministry picked Motif as the fourth team in a supplementary selection. That left Motif only February through July for development.

According to a ministry briefing, Round 2 was scored out of 100 points:

• Benchmarks (40 points): 25 points from the Artificial Analysis Intelligence Index (AAII). That's a leaderboard from the private US company Artificial Analysis that combines nine hard tests covering agents, coding, knowledge and scientific reasoning. Another 15 points came from a non-public test by the agency NIA, covering Korean language and safety, among other things.
• Expert evaluation (35 points): Ten specialists rated strategy, track record and benefit to the domestic AI ecosystem.
• User evaluation (25 points): 49 expert AI users, such as startup founders, awarded up to 15 points. 185 randomly selected citizens awarded up to 10 points.

According to the ministry, the weighting and test selection had been agreed with the teams in advance.

Running the numbers: How 16 index points became 4

Only after Motif's appeal did the ministry publish the score breakdown on August 27. According to AI Times, it looked like this:

The totals add up. The AAII points correspond almost exactly to the index score divided by four. Nate News gives Motif an unrounded score of 47.4, and 47.4 divided by four is 11.9. Since no model comes anywhere near 100 index points, this heavily compresses the gaps. A 16-point index lead over LG turned into 4.1 scoring points. That's exactly what Motif criticized: the raw gaps weren't adequately reflected in the conversion.

What would have happened under different assumptions? Our recalculation using the published numbers:

• Without the user evaluation: Motif 51.7, Upstage 51.8, SKT 51.5, LG 50.1. LG, not Motif, would have come last.
• Without the citizen evaluation: Motif stays last with 60.1 points. In that sense, Vice Minister Ryu Je-myung is right that the citizen scores "did not influence the decision." The deciding factor was the 15 points from the expert AI users.
• AAII scaled relative to the best (top model gets 25 points): Motif would land at 78.9 and LG at 77.6. LG would have been eliminated in that case, too.
• Using the actual formula: To overtake LG, Motif would have needed just over 60 index points. That's nearly as much as the world's best model at the time, Claude Opus 5, at 63 points.

Our take: none of these methods is objectively correct. But the choice of conversion decides who wins and who loses. In effect, the scheme all but neutralized benchmark strength, leaving a small user panel to tip the balance.

The ministry justified the elimination on usability grounds. Motif had "demonstrated world-class capability at the technical level," it said, but scored lower on usability and application. There was also a suspicion: US data vendor AfterQuery described itself on LinkedIn as the "sole data partner" for Motif 3. According to iNews24, the company advertises that it specifically fixes benchmark weaknesses. The industry calls this "benchmaxxing": training for the test rather than for real capabilities. Motif CEO Lim Jeong-hwan denies it, saying the collaboration served agent performance. The ministry said it found no evidence of overfitting in its AAII analysis and that the allegation didn't factor into the scoring. According to ZDNet Korea, the funding agency NIPA rejected the appeal on September 10 without any substantive explanation. Motif is sitting out Round 3.

Who hands out the scores: Dispute over judges and blind evaluation

Of all things, the decisive expert-user panel is the one under fire. At the parliamentary audit in October, lawmaker Kim Jang-gyeom (People Power Party) presented figures: 33 of the 49 expert users had already scored Round 1 – that's 67.3 percent. Rotation had been ensured for the expert panel, but not for the user panel. According to Data Economy, conflicts of interest were ruled out mainly through signed declarations. Ryu acknowledged the overlap and pointed out that the rules had been agreed with the companies.

Our take, unproven: two-thirds of the judges already knew the models from LG, SKT and Upstage from Round 1, but not Motif's. Whether that affected the scores can't be read from the data. It hardly inspires confidence in their impartiality, though.

For Round 3, according to ZDNet Korea on November 3, the ministry and teams are discussing double-blind evaluation, where judges know neither the company nor the model name. One industry representative says the direction is clear. It hasn't been decided yet, and the ministry notes that a model can still be identified through targeted questions. The three teams have until the end of January 2027 to work on their models. Scores are due in mid-February. After that, under the current plan, two teams will remain.

Vendor claims vs. leaderboard: K-EXAONE 2.5

LG has so far been last on the AAII, but it was the overall winner in Round 1 and has now raised its game. On November 9, LG AI Research opened free API access to K-EXAONE 2.5 via docs.exaone.ai. The API is OpenAI-compatible and handles text only, in Korean and English. The model is a post-trained K-EXAONE 2.0, a mixture-of-experts model with 750 billion parameters. With this architecture, only part of the model does the computation for each token. The 2.0 weights are available on Hugging Face under the Apache 2.0 license. Whether 2.5 will also be released as a download is unclear.

LG claims a performance gain of more than 70 percent across 24 internal metrics in eight weeks. According to Unite.AI, all the published individual scores are vendor claims:

By our math, the relative gains are +26, +53, +341 and +24 percent. LG doesn't explain how it arrives at "70 percent." But an average of relative gains is highly sensitive to outliers like AutomationBench. LG also chose DeepSeek's smaller Flash variant for comparison.

An independent assessment is still missing. And the yardstick has moved in the meantime: on October 27, Artificial Analysis launched version 5 of its index, with more private tests. According to Trending Topics, open models lost up to 16 points in the process; DeepSeek V4.1 Flash, for example, dropped from 45 to 32. On the current model page, K-EXAONE 2.0 sits at 8 points (median for comparable models: 23). Motif 3 isn't listed at all right now because it lacks a public API. The August scores aren't comparable with these numbers. So a government competition that ties 25 percent of its scoring to a private US company's leaderboard is measuring against a moving target.

Another rough signal is Hugging Face downloads, as of November 9 and not clearly labeled: according to the model pages, K-EXAONE 2.0 is at around 14,000 over the past month, Solar Open 2 and A.X K2 at around 7,000 to 8,000 each, and Motif 3 at around 1,600. So the benchmark winner is the least downloaded. That's consistent with the judges' verdict, but proves nothing.

What Europe can learn

Seoul is restructuring in parallel. According to Digital Today, the 2027 draft budget earmarks 5.2 trillion won for a new, investment-based frontier AI program. Of that, 3.85 trillion is planned for around 10,000 Vera Rubin-class GPUs. The tender is set to open in December at the earliest. Ryu openly admits that with 1,000 GPUs per team, there's no catching up with the world's best.

Europe has taken a different path so far. Rather than crowning a model champion, it funds consortia and infrastructure. OpenEuroLLM is financed through the Digital Europe Programme and plans its first models for the end of December 2026, with final ones in January 2028. For the AI Gigafactories, the EU aims to mobilize €20 billion through InvestAI. The formal call launched in July 2026; the earlier informal call for expressions of interest drew 77 proposals from 16 member states.

Our take: if Europe ever does pit models against each other, it can learn four things from Korea. First, publish the formula and score breakdowns in full and in advance. Second, define how benchmark gaps translate into points, because that alone can flip the result. Third, rotate subjective panels and have them score blind. Fourth, don't tie yourself to a single external index that can change mid-competition.

The bottom line

The Korean case doesn't show fraud. It shows a design choice whose consequences only became visible after the fact. The ministry explicitly wanted to reward not just benchmark points but models that people actually choose to use day to day. There are good reasons for that, not least the risk of benchmaxxing. But the execution has weaknesses: the conversion compressed the benchmark gap almost to irrelevance. A small panel of 49 people – two-thirds of them the same as in Round 1, and not blind – tipped the scales. And the appeal was rejected without explanation.

Whether Motif 3 is really worse in everyday use can't be determined from public data. All that's established is that it lost the judges' verdict. For LG, what matters now is less its own "70 percent" figure than February's blind evaluation. For Europe, the case is a useful lesson before it crowns a state-backed AI champion of its own.