Microsoft AI Diagnostic System Outperforms Generalist Doctors in Complex Case Simulation
A new study reveals that Microsoft's multi-agent AI framework achieved an 83% accuracy rate on difficult medical scenarios, significantly exceeding the performance of individual primary care physicians.
By Alina Carter ·
A newly published study in Science highlights a widening gap between artificial intelligence and human physicians in diagnosing complex medical conditions. The research demonstrates that Microsoft's AI diagnostic system, known as MAI-DxO, correctly identified the right treatment path in difficult cases more than twice as often as generalist doctors. While the findings suggest a significant leap in automated medical reasoning, the results also underscore the limitations of current AI in specific hospital settings.
The system operates by coordinating multiple AI agents that mimic a panel of doctors. These agents interact with patient data by asking questions, ordering tests, and debating potential diagnoses. In a simulation using complex cases from the New England Journal of Medicine, the AI achieved an accuracy rate of 83%. By comparison, primary care doctors and general internists working individually with standard reference materials were correct only 36% of the time. It is important to note that such complex cases are typically handled by specialists or teams in real-world scenarios, a group that was not included in this specific comparison.
Dominic King, vice president of Health at Microsoft AI, described the findings as substantial. He noted that the primary challenge in healthcare delivery is the growing mismatch between the demand for expertise and the available supply of medical professionals. By publishing these results, Microsoft joins Google and OpenAI in asserting that their AI technologies can match or exceed the capabilities of human doctors.
The data indicates that much of the performance improvement stems from the underlying power of the AI models rather than the orchestration system alone. Microsoft tested its framework with models from various companies, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The best outcomes were achieved using OpenAI's GPT-5 and GPT-5.5. When GPT-5 was used without the multi-agent framework, it scored 78.9% on complex cases at a cost of approximately $8,400 per case. The addition of the orchestration system reduced the cost by about $3,000 but only raised accuracy by four points.
Performance varied significantly depending on the model and the type of case. With the newer GPT-5.5 model, the orchestration system boosted accuracy from 72% to nearly 87% for complex cases while slightly lowering costs. However, for emergency department cases, the gains were minimal, with or without the system, both scoring around 85% at similar costs. The system showed its greatest utility when paired with less capable models, such as Gemini 2.5 Flash, where accuracy jumped from 46.4% to nearly 69%.
Mustafa Suleyman, CEO of Microsoft AI, views this work as a step toward medical superintelligence. He stated that the company is collaborating with the Mayo Clinic to train a healthcare-specific foundation model, with a release planned for next year. Suleyman predicts that by 2027, the technology will enable clinicians to stream highly accurate predictions about how medical records might unfold, providing valuable foresight. He believes that healthcare models will become an order of magnitude more capable within the next 24 to 36 months as the technology matures through real-world usage.
To validate these capabilities outside of simulations, Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston. This initiative will evaluate the clinical reasoning of the AI across different hospital settings, including emergency and urgent care environments. Patients will receive standard treatment, while consenting patients will also engage with the AI system. The team will then compare the AI's recommendations against those of human doctors. King described this as a major study that will provide compelling evidence for deploying such systems at a larger scale in the future.
The study coincides with a broader manifesto from Microsoft defining medical superintelligence as the capacity to surpass the strongest human-led health systems in reasoning and coordination. The company aims to move beyond simple benchmarking and integrate its tools directly into clinical workflows. Suleyman expressed confidence that Microsoft can become the leader in clinical decision support products, leveraging its existing portfolio of dictation and radiology tools to accelerate the adoption of AI in healthcare.
