Microsoft's AI Beat Doctors on the Hard Cases — Now It Wants to Prove It in a Real Hospital

Microsoft has a new claim in the race to put artificial intelligence inside medicine: its diagnostic system solved difficult simulated cases at more than double the rate of generalist physicians. The catch, acknowledged in the research itself, is that the comparison was not quite apples to apples — and the next step has to happen in a real hospital, not a simulation.

A panel of AI doctors

The system, called MAI-DxO, works less like a single chatbot and more like a panel of clinicians. According to TechCrunch, several AI agents question a patient, order tests and debate a diagnosis before settling on an answer. The research behind it was published in Science.

In a simulation built from complex cases published in the New England Journal of Medicine, the system reached the correct diagnosis 83% of the time, TechCrunch reported. Generalist doctors working alone with reference materials got 36% right. But cases this complicated are normally handled by specialists or teams, and those groups were not part of the comparison.

The models did much of the work

The study's own numbers also complicate Microsoft's story. According to TechCrunch, much of the improvement appears to come from the increasingly powerful AI models underneath the system, rather than from Microsoft's orchestration layer alone.

Microsoft tested the system with models from several companies, including OpenAI, Google, Anthropic, Meta and DeepSeek, and its best results came with OpenAI's GPT-5 and GPT-5.5. On its own, GPT-5 scored 78.9% on complex cases at an estimated cost of about $8,400 per case, TechCrunch reported. Adding Microsoft's system cut that cost by roughly $3,000 while lifting accuracy by only about four points.

With GPT-5.5, the system raised accuracy on complex cases from 72% to nearly 87% at a slightly lower cost. But on emergency department cases, it barely moved either accuracy or cost — both versions landed around 85%. Weaker models benefited more: Gemini 2.5 Flash rose from 46.4% alone to nearly 69% with the system.

From benchmarks to hospital corridors

Microsoft is not stopping at benchmarks. Its AI chief, Mustafa Suleyman, told TechCrunch that Microsoft is working with the Mayo Clinic on a foundation model for health care, with a release planned for next year, and predicted a breakthrough in how medical records are handled in 2027.

The more immediate test is a partnership with Beth Israel Deaconess Medical Center in Boston. According to TechCrunch, patients there will receive standard care while, with their consent, the AI system works in parallel. Microsoft and its partners will then compare the system's recommendations against the human doctors' decisions — including the complexity of a Boston emergency department on a Friday evening.

Microsoft is joining Google and OpenAI in claiming its AI can match or beat doctors, and it has defined its goal as "medical superintelligence": surpassing the strongest human-led health systems in reasoning and coordination across the full arc of care. The Boston trial is where that claim starts to get tested against real patients, real time pressure and real costs.

Written by the BidAsk Newsroom. Based on reporting from the sources below.