Microsoft's diagnostic AI beats doctors in a test, but there's a big catch

What this is about

Microsoft has presented a study on its AI diagnostic system MAI-DxO (Microsoft AI Diagnostic Orchestrator) in the journal Science (published October 22). According to the study, it correctly diagnosed complex medical cases in a simulation more than twice as often as general practitioners: 83% versus 36%. Mustafa Suleyman, Microsoft's AI chief, sees it as a step toward "medical superintelligence." This is a research technique and is not available to patients.

How MAI-DxO works

MAI-DxO isn't a single model. It's an orchestrator: several AI agents work together like a panel of doctors. The system starts with little information, asks follow-up questions about the medical history and decides which tests would help. The agents challenge each other's conclusions as new results come in.

This differs from earlier studies, where the AI got a complete case in a single prompt. Here, diagnosis is a step-by-step process that also weighs costs. The "patient" was an AI agent acting as a gatekeeper, answering questions from the underlying case file. According to Fox News, it could generate plausible answers when details were missing.

Numbers and comparisons

The testing used 304 clinicopathological conference cases from the New England Journal of Medicine and 325 cases from the MIMIC-IV-ED database, running from emergency department arrival to admission.

That makes the emergency department score about 1.7 times the doctors' result, not double (according to The News).

How much the orchestration system adds depends on the base model. According to TechCrunch, GPT-5 with a short standard prompt reached 78.9% at an estimated cost of about US$8,400 per case. MAI-DxO cut costs by around US$3,000 and raised accuracy by only about four points. With GPT-5.5, released in April 2026, the rate on complex cases rose from 72% to almost 87%. In the emergency department, both variants landed at about 85%. Weaker models benefit more: Gemini 2.5 Flash scored 46.4% on its own and just under 69% with MAI-DxO. Models from Anthropic, Meta and DeepSeek were also tested. The gap between weak and strong models was larger than the gain from MAI-DxO. In other words, much of the performance comes from the base model.

The cost of diagnostics

The researchers assigned estimated costs to examinations. In a cost-conscious configuration, MAI-DxO reached about 81% at an estimated US$2,241 per case, while doctors with reference materials averaged around US$4,725. In terms of total costs, the authors speak of between half and a third of the testing costs. Configurations tuned for maximum accuracy cost more. These figures are estimates from the simulation, not real bills, and they don't include the AI's running costs.

Limits of the study

• The comparison doctors were family physicians and general internists working alone. In everyday clinical practice, such cases usually go to specialists or teams, who weren't included in the comparison.
• The AI was tested on all 304 cases, while the doctors worked on only 56 held-out cases, according to Fox News. The groups aren't identical.
• The study used existing case reports, not real-time patients. People withhold symptoms, records have gaps, and physical exams yield clues that appear in no summary.
• A correct diagnosis is only one part of treatment.
• The authors work for Microsoft, and independent testing is still pending.

Outlook: clinical trials and ambitions

Microsoft is planning a study with Beth Israel Deaconess Medical Center in Boston, in emergency departments and urgent care facilities. Patients will receive their usual care. Those who consent will additionally interact with MAI-DxO. Its recommendations will be compared with those of the doctors, who remain responsible. Dominic King, Vice President of Health at Microsoft AI, expects "the largest study of its kind."

In October, Google published a study in The Lancet on its AMIE system: 98 patients spoke with the AI before urgent primary care appointments, doctors monitored the conversations, and none had to be stopped for safety reasons.

Suleyman told TechCrunch about a health foundation model trained with the Mayo Clinic, due next year. He expects a breakthrough in predictions from medical records in 2027 and said health models would improve "by an order of magnitude" within 24 to 36 months. Microsoft already sells dictation and radiology tools and wants to lead in clinical decision support. Microsoft defines medical superintelligence as the ability to outperform the strongest human-led health systems in medical reasoning and coordination.

Our take

The study shows that orchestrated AI agents can deliver strong diagnostic performance in controlled tests and may save on unnecessary tests. But "AI beats doctors" is too simple. The comparison involved generalists working without teams, simulated patients and unequal case counts. A large share of the success comes from models like GPT-5 and GPT-5.5. With the best models on emergency cases, orchestration added little. Google and OpenAI make similar claims. Whether MAI-DxO helps patients or cuts costs in real clinical practice remains an open question. For patients, the takeaway is this: AI answers are good for preparing questions for your doctor, not as a diagnosis.