Asylum AI With No Error Stats: What Germany’s BAMF Software Can Actually Prove
Germany’s planned AI Migration Administration Act (KIMVG) would let authorities train AI on old asylum files and automatically check applicants’ statements against the internet. We’ve pulled together how well the AI tools the BAMF already uses actually perform. The government doesn’t report a real error rate. And according to the research, the bill’s main safeguard, staff training, is weak.

What’s at stake
On November 23, 2026, the Bundestag’s Committee on Internal Affairs will hear expert testimony on the AI Migration Administration Act (KIMVG, Bundestag document 21/8275). A week earlier, on November 16, 37 organizations published an appeal against the bill, including AlgorithmWatch, Pro Asyl, Amnesty International, AWO and the German Red Cross (DRK). The legal debate is in full swing. A simpler question gets far less attention: How well do the AI tools that the Federal Office for Migration and Refugees (BAMF) has used since 2017 actually work?
The numbers are scattered across the federal government’s answers to parliamentary questions from the Left Party (Die Linke). We’ve pulled them together. The upshot: there are hardly any real error rates. That leaves a law whose main protection against mistakes is trained staff on shaky ground.
What the KIMVG would allow
The Interior Ministry drafted the bill together with the Foreign Office. The cabinet approved it on July 29, 2026, and its first reading in the Bundestag on October 8 took about 20 minutes. It has three core elements:
- Training: Authorities could use personal data from asylum and residence cases to develop, train, validate and test AI systems. There’s no general obligation to anonymize the data. According to the bill’s explanatory notes, anonymization isn’t required if it’s technically impossible, disproportionate or would make results worse. The notes explicitly describe names and dates of birth as useful.
- Process monitoring: Closed cases would be analyzed automatically to find patterns, ways to speed things up and decision patterns. Profiling, meaning assessing individuals based on their data, would be banned.
- Internet checks: If there are “actual indications” of justified doubts about a statement, the BAMF could check it against publicly available internet data (§ 7b of the draft Asylum Act, with similar rules in the Residence Act). netzpolitik.org cites Instagram, LinkedIn and news reports as examples. Biometric data is excluded.
As safeguards, the bill requires humans to review the results manually and staff to be “specially trained.” There are also documentation requirements and an evaluation at least every five years. According to an analysis by Tech Times, the bill never uses the term “automation bias,” the tendency to follow a computer’s suggestion even when it’s wrong. The federal government says the bill won’t create any costs beyond compliance costs. The national associations of local authorities disagree: they say servers, hardware and training would put a heavy burden on immigration offices.
The track record: four tools, few solid numbers
| Tool | What it does | Documented figures | Official error rate? |
|---|---|---|---|
| DIAS (dialect recognition) | assigns speech samples to a region of origin | 2018: error rate “around 20 percent”; test recognition rate around 87% in 2023 (Arabic), 73.07% in 2022 (Dari/Farsi) | no, “not available” since 2022 |
| Data carrier analysis | reads phones of applicants without passports | 2018: 32% confirmed, 2% refuted, 65% without usable result; 2023: 73.4% without findings, 1.7% contradiction | no |
| Name transliteration | converts names into Latin script consistently, gives hints about origin | 2018: 85–90% success for Syria/Iraq, approx. 35% for Maghreb | no, “not available” |
| Photo matching (LiBiAs) | searches for multiple identities via photos | “no further statistical recording takes place” | no |
Sources: Bundestag documents 19/190, 19/6647, 20/3238, 20/9419, 21/1710 and reporting by netzpolitik.org.
Phone analysis is the clearest example of a tool that delivers little. From January to October 2018, 9,710 data carriers were read out, and 3,439 attempts failed. According to VICE, the system was expected to cost around €11.2 million by 2019. In 2023, around 12,500 devices were read out, and almost three quarters of them produced nothing usable. In February 2023, the Federal Administrative Court ruled that a phone may only be analyzed if there are no less intrusive options. The labor-intensive analysis has been suspended since 2024. In the first half of 2025, only 338 data carriers were read out.
Why “not supported” isn’t an error rate
DIAS, on the other hand, is being used more and more: in 9,923 cases in 2020, 15,052 in 2021, 29,632 in 2022 and 22,947 in the first half of 2023 alone. The government reports how often the result “supports” applicants’ statements. In 2022, that was 78 percent, and in the first half of 2023, 81 percent. For 2022, it also gave some striking figures by country. For Sudan, statements weren’t confirmed in 70.6 percent of cases.
But these figures don’t tell you who got it wrong. “Not supported” can mean that someone gave false information. It can also mean that the software made a mistake. The federal government doesn’t distinguish between the two. Asked about the error rate in 2022, it replied: “An error rate within the meaning of the question does not exist.” Instead, it cites recognition rates from internal “overall tests,” meaning tests with known speech samples.
Our take: Let’s run the numbers with those test values. Even at the best figure of 87 percent, the software misclassifies roughly one sample in eight. That alone could explain a large share of the 19 percent of “not supported” cases in the first half of 2023 (around 4,360 cases) as software errors. For Dari and Farsi, the error rate in testing is more than a quarter. Under real-world conditions, the rate is unlikely to be better than in the lab. Nobody can prove it, though, because nobody measures it. The government stresses that the result is “merely one indication among many” and isn’t evidence. But it sits in the file in front of decision-makers and is meant to guide targeted questions during the asylum interview.
Among experts, the basic idea is controversial. Mark Liberman heads the Linguistic Data Consortium, which according to AlgorithmWatch supplies most of the training data. He doubts that spoken Farsi, Dari and Pashto can be reliably told apart by machine. The government announced independent scientific evaluation back in 2018. We found no evidence that it happened. A new parliamentary question from the Left Party on DIAS (Bundestag document 21/9007), filed on November 10, 2026, hasn’t been answered yet.
Names online: a technically tricky check
In practice, the internet checks will often rely on names, since biometrics are excluded. And names are exactly where the BAMF has a weak track record. The transliteration software was trained on around a billion names, yet in 2018 it achieved only about 35 percent success for names from the Maghreb countries. In an analysis of 1,443 cases, 39 percent of the results couldn’t be verified.
Researchers have known about this problem for a long time. In 2006, Freeman and colleagues showed that common fuzzy search methods perform poorly on Arabic-English names. The reasons are inconsistent transliterations and the fact that Arabic script usually leaves out short vowels. Then there’s a simple problem: lots of people share the same name. A match on Instagram can show a completely different person. The European Parliamentary Research Service therefore warns that AI errors can seem to confirm suspicions of false statements. The Bundesrat, Germany’s upper house, still wants to extend the checks to establishing identity and nationality. Those are exactly the cases where names are least reliable. The government says it will look into it.
Does training help against automation bias?
The research here is sobering. In 2010, Parasuraman and Manzey summed it up in the journal Human Factors: automation bias affects laypeople and experts alike, in individual and team decisions, and training or instructions can’t prevent it. A systematic review by Goddard and colleagues (2012, 74 studies) found mixed evidence. Training can help people spot system errors, but one study they cited found no effect. Design choices looked more effective: showing how confident the system is, and having it provide information rather than ready-made recommendations. Time pressure and heavy workloads make the effect worse. In a small meta-analysis from medicine, incorrect advice from a computer increased the relative risk of a wrong decision by 26 percent.
Ben Green analyzed 41 policies that require human oversight of government algorithms. His 2022 conclusion: they rest on an untested assumption and create a false sense of security. Experiments by Alon-Barkat and Busuioc in the Netherlands are especially revealing for public administration. Participants there followed algorithmic recommendations mainly when they matched common stereotypes. That matters for DIAS: a result like “doesn’t speak Sudanese” lands in front of decision-makers who may already have their doubts.
What the AI Act requires
The EU’s AI Act classifies AI systems as high-risk in Annex III No. 7 if they help authorities examine asylum, visa and residence applications, including assessing evidence. That covers exactly what’s at issue here. Article 14 requires the people overseeing these systems to stay aware of the tendency to trust them blindly, and explicitly calls it “automation bias.” Legal scholars Anna-Lena Priebe and Lise Känner consider the monitoring and the internet checks to be high-risk systems. Whether the government can claim an exemption is disputed. However, under the EU’s “Digital Omnibus” package, the obligations for such systems won’t apply until December 2, 2027.
The bigger picture
The KIMVG would allow training on data from procedures whose tools have never been systematically tested for errors. Eight years of BAMF technology add up to a mixed record. Phone analysis was expensive, mostly found nothing and has now largely been dropped. The name software failed in exactly the hard cases. DIAS is used on a massive scale, but there’s no real error rate.
Our take: Training doesn’t hurt, but according to the research, it isn’t enough to counter automation bias. Things the bill doesn’t make mandatory would work better: measured error rates for each language, country and tool, showing uncertainty in the results, independent audits before deployment, and enough time for caseworkers. The Federal Data Protection Commissioner also considers a simple human review of results insufficient. Whether the experts can close this gap on November 23 is an open question. The obvious demand remains an error statistic: a state that lets AI help decide on protection claims should first measure how often it gets things wrong.
Sources
- Bundestag: Drucksache 21/8275 (Gesetzentwurf KIMVG) dserver.bundestag.de
- hib: Künstliche Intelligenz in der Migrationsverwaltung (Gesetzentwurf, Bundesrat, Gegenäußerung) bundestag.de
- Bundestag Textarchiv: Erste Lesung KIMVG bundestag.de
- Bundestagszusammenfasser: KIMVG bundestagszusammenfasser.de
- Tech Times: Germany AI Asylum Law Draft Leans on Staff Training techtimes.com
- AlgorithmWatch: KIMVG-Appell algorithmwatch.org
- netzpolitik.org: Dutzende Organisationen warnen vor KI bei Migrationsverfahren netzpolitik.org
- netzpolitik.org: Es geht um Entscheidungen über Menschenleben netzpolitik.org
- netzpolitik.org: Da sollten die Alarmglocken schrillen netzpolitik.org
- LTO: Erteilt die KI bald Arbeitserlaubnisse? lto.de
- FluchtforschungsBlog: Der Gesetzesentwurf zum KIMVG auf dem Prüfstand fluchtforschung.net
- hib: Software im Bamf zur Dialekt-Erkennung (2017, Fehlerquote ca. 20 %) bundestag.de
- kleineanfragen.de: Drucksache 19/6647 (IT-Assistenzsysteme im BAMF) kleineanfragen.de
- netzpolitik.org: BAMF weitet automatische Sprachanalyse aus (2022) netzpolitik.org
- hib: Einsatz von Dialekterkennungssoftware im Bamf (Antwort 20/3238) bundestag.de
- hib: Sprach- und Dialekterkennungssoftware beim Bamf (Antwort 20/9419) bundestag.de
- hib: Kleine Anfrage 21/9007 zu DIAS (November 2026) bundestag.de
- netzpolitik.org: Asylbehörde liest kaum noch Datenträger aus (2025) netzpolitik.org
- VICE: Wie teuer und nutzlos die IT-Offensive des BAMF ist (2018) vice.com
- AlgorithmWatch: Kontroverse Dialekterkennung algorithmwatch.org
- netzpolitik.org: Wissenschaftlicher Dienst des EU-Parlaments zu KI in Asylverfahren netzpolitik.org
- Goddard et al. 2012: Automation bias – systematic review (JAMIA) pmc.ncbi.nlm.nih.gov
- Parasuraman & Manzey 2010 (PubMed) pubmed.ncbi.nlm.nih.gov
- Ben Green: The Flaws of Policies Requiring Human Oversight of Government Algorithms (arXiv) arxiv.org
- Alon-Barkat & Busuioc: Human-AI Interactions in Public Sector Decision Making (arXiv) arxiv.org
- Freeman et al. 2006: Cross Linguistic Name Matching in English and Arabic aclanthology.org
- EU AI Act: Anhang III artificialintelligenceact.eu
- EU AI Act: Artikel 14 artificialintelligenceact.eu
- Gibson Dunn: EU AI Act Omnibus Agreement – Postponed High-Risk Deadlines gibsondunn.com
- Tech Policy Press: Germany's Draft Law on AI in Migration techpolicy.press
Ask the AI assistant
Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.