Microsoft AI published a peer-reviewed study in the journal Science on Thursday showing its MAI-DxO diagnostic system correctly diagnosed complex medical cases 83% of the time, more than double the 36% rate of generalist physicians working with reference materials. CEO Mustafa Suleyman called the result a step toward “medical superintelligence.”
What Was Announced?
The paper introduces two foundations, Microsoft AI announced: the Sequential Diagnosis Benchmark (SDBench), which tests step-by-step diagnostic reasoning, and MAI-DxO, the company’s leading AI system on that benchmark. SDBench evaluates performance across 304 complex cases taken from The New England Journal of Medicine and a set of more typical emergency-department presentations from MIMIC-IV-ED. Each becomes an interactive case in which AI models or physicians start with limited information and choose what to ask, which tests to order and how each new finding changes the likely diagnosis. The benchmark measures diagnostic accuracy, the evidence requested and estimated test costs.
MAI-DxO itself works like a panel of virtual physicians: each proposes diagnoses, selects useful questions or tests, and challenges the emerging view over several rounds of debate, updating its assessment as evidence arrives and weighing each test’s likely value against its cost. Microsoft is open-sourcing SDBench for select NEJM cases, including the benchmark, Gatekeeper, Judge and cost-estimation code, so researchers can reproduce and extend the findings.
Graphic: GPTOnline
Key Findings
On the complex NEJM cases, MAI-DxO reached 83% diagnostic accuracy, while physicians scored 22% without external reference resources and 36% with them, making the system about 3.7 and 2.3 times more accurate respectively, according to Microsoft AI. On the more typical emergency-department cases, MAI-DxO reached 80% accuracy versus 47% for physicians, more than 1.7 times higher. A cost-optimized configuration held 81% accuracy at less than half the cost of comparative systems, and Microsoft says that “across both datasets, no other human or AI approach was both more accurate and less costly than MAI-DxO.”
The gains also persisted with newer underlying models, including GPT-5.5, without changing the orchestration method, suggesting the approach is not tied to one model generation. TechCrunch reported cost details from the study: running GPT-5 alone scored 78.9% on complex cases at about $8,400 per case, while DxO cut that cost by about $3,000 and raised accuracy by about four points. With GPT-5.5, DxO lifted accuracy from 72% to nearly 87% at a slightly cheaper cost. The orchestration helped most on complex cases and with weaker models: Gemini 2.5 Flash, for example, rose from 46.4% on its own to nearly 69% with DxO. On emergency-department cases, by contrast, there was little improvement between GPT-5 on its own and with DxO’s help.
Chart: Microsoft AI
What Suleyman Said
Suleyman framed the work as a manifesto for “medical superintelligence,” which Microsoft defines as “the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care.” He told TechCrunch the company is working with the Mayo Clinic to train a foundation model for healthcare, with release planned for next year, and believes it will enable “really enormous leaps.”
“I really believe that in 2027, we will be on the verge of a coding style breakthrough for medical records,” Suleyman said. “I think we’re going to be able to stream highly accurate predictions about how a medical record might unfold, and obviously that kind of foresight for a clinician is gold dust.”
Through a new partnership with Beth Israel Deaconess Medical Center in Boston, Microsoft will evaluate DxO’s clinical reasoning across different hospital settings, comparing its recommendations against human doctors’ in parallel with standard treatment. “This technology is ready for real-world evaluations,” said Dominic King, vice president of Health at Microsoft AI. “There is something very substantial and exciting here. The biggest challenge in healthcare delivery at the moment is the growing mismatch between demand and supply of medical expertise.”
Caveats and Context
The headline numbers come with important limits. Cases this complex are usually handled by specialists or physician teams, who were not part of the comparison. Microsoft’s own researchers noted that accuracy gains naturally shrink as models approach the theoretical maximum, and the DxO orchestration adds the least on routine emergency-department cases.
Microsoft joins Google and OpenAI in claiming AI can match or outperform doctors: Google published a clinical study alongside Beth Israel in The Lancet this month evaluating its AMIE conversational diagnostic AI in real-world urgent care visits. Suleyman, a co-founder of DeepMind who joined Microsoft to lead its AI unit in 2024, expects healthcare models to get “an order of magnitude better in capability within the next 24 to 36 months,” arguing the field is about to get the same production flywheel that transformed coding and chat.
