Microsoft AI system beats doctors at diagnosis in new study

What this is about

Microsoft has published a study in the journal Science on its AI system, the Microsoft AI Diagnostic Orchestrator (MAI-DxO). It is said to find the correct diagnosis far more often than doctors on difficult medical cases. Microsoft AI chief Mustafa Suleyman told The Verge that medical AI is "finally here." The study, titled "Sequential Diagnosis with Language Models," was already available last year as a preprint on arXiv, before peer review. It has now passed peer review (evaluation by independent experts).

How the system works

The orchestrator works like a panel of physicians. Multiple AI agents, which are programs that use language models to handle subtasks, play different roles. They propose diagnoses, choose tests, challenge assumptions and weigh costs. According to the study, the system continually revises the differential diagnosis, meaning the list of possible conditions. The researchers say this could reduce the cognitive biases of individual doctors, such as sticking with a first diagnosis.

What sets it apart is the sequential process: the system doesn't get all the symptoms, history and test results at once. It starts with the chief complaint and has to decide for itself which questions to ask and which tests to order. That is closer to how real practice works. For the evaluation, Microsoft developed the benchmark SDBench (Sequential Diagnosis Benchmark), which the company plans to release as open source.

The numbers

The comparison group consisted of general practitioners (generalists). Microsoft also says the system worked at lower estimated costs, at one-half to one-third, with fewer tests and procedures. But these are estimated costs in simulated cases, not actual treatment costs. The system achieved its best result with OpenAI's GPT-5. According to Gizmodo, additional tests showed improvements with GPT-5.5. The system was originally developed with GPT-4.1. It also outperformed individual top models on their own. Newer models such as GPT-6 Astra, Gemini 4 Argon or Opus 5.5 are not included in the peer-reviewed paper. According to the researchers, later tests suggest that better models don't necessarily erode the benefit of the multi-agent approach.

Limits and next steps

Microsoft admits there is still a long way to go. The system was tested only on past cases. How well it works in everyday clinical practice is unclear. "It needs clinical validation," Suleyman said, calling it a leap, but a milestone. Dominic King, Microsoft's head of health AI, announced hospital trials with Beth Israel Deaconess Medical Center in Boston. There was no firm timeline, but they are expected to start "in the coming months." The researchers expect the gap would narrow if the system were compared with specialists rather than generalists.

Microsoft's vision

The project is part of an ambition to build "medical superintelligence," meaning AI that outperforms the best human-led healthcare systems. According to a blog post and an article in BMJ Digital Health & AI, Microsoft sees five other areas, including care coordination, treatment decisions and the human side of medicine. Commercialization is conceivable, but Microsoft gave no timing or form. The company already has a presence in US healthcare, and health is among the most common topics for its Copilot chatbot. Suleyman expects "extremely impressive" deployments next year and stresses that doctors should keep a central role.

Our take

Publication in Science is an important step for credibility. Still, the claim "better than doctors" comes with caveats: the comparison was against generalists, the task was unfamiliar and artificial, and only historical cases were used. The benchmark comes from Microsoft itself. Similar results have already been seen for OpenAI's o1-preview in triage. Liability questions remain open if an AI-assisted diagnosis is wrong. A 2025 study also showed that doctors got worse at colonoscopies when working without AI after using it. Experts therefore see such systems as assistants, not replacements.