Microsoft researchers have developed an artificial intelligence system that demonstrated higher diagnostic accuracy than generalist physicians in controlled evaluations involving complex medical cases and emergency department records.
The system, known as Microsoft AI Diagnostic Orchestrator (MAI-DxO), achieved an accuracy rate of 83.2% on challenging clinical cases, compared with 36.1% for physicians who could consult reference materials. In a separate evaluation using historical emergency department records, the system correctly diagnosed 80.3% of cases, compared with 46.8% for participating physicians.
The findings suggest that AI could help clinicians investigate difficult cases, gather relevant patient information, and make more informed diagnostic decisions. However, the research was conducted using previously documented cases, and further clinical testing is needed to determine how the technology would perform in real healthcare settings.
How Microsoft’s AI Diagnostic System Works
MAI-DxO takes a sequential approach to medical diagnosis rather than processing a complete patient record at the beginning. It starts with limited clinical information and determines which questions to ask, which diagnostic tests to request, and when enough evidence has been gathered to reach a conclusion.
The system uses a large language model to coordinate multiple diagnostic perspectives, simulating a panel of medical specialists with different approaches to evaluating a patient. It maintains a list of potential diagnoses, requests additional information, reassesses its conclusions, and considers the cost of further investigations.
The primary evaluations used OpenAI’s GPT-5 as the underlying model. Microsoft researchers also developed the Sequential Diagnosis Benchmark (SDBench), a framework designed to assess how AI systems and physicians make decisions as new clinical evidence becomes available.
SDBench includes 304 complex clinicopathological cases published in The New England Journal of Medicine and 325 emergency department encounters from the MIMIC-IV-ED database.
Patient information is disclosed progressively when requested, requiring participants to decide what information they need rather than relying on a complete case description. Researchers also assigned estimated costs to diagnostic tests to evaluate the relationship between accuracy and resource consumption.
MAI-DxO asked an average of 25.5 questions before ordering its first diagnostic test, compared with 5.6 questions among participating physicians. This allowed the system to gather more information about patient histories before deciding which investigations to pursue.
AI Achieves Higher Diagnostic Accuracy
On the complex medical cases, MAI-DxO correctly diagnosed 83.2% of patients. By comparison, generalist physicians achieved 22.3% accuracy without reference tools and 36.1% when they could consult external resources.
The highest-performing physician in the evaluation reached 51.8% accuracy, still below the AI system’s reported result. However, the comparison involved individual physicians rather than specialist teams collaborating on difficult diagnoses.
The researchers acknowledged that the performance difference could be smaller if the system were compared with specialists experienced in the specific conditions being evaluated. The complex cases were also selected for their diagnostic difficulty and educational value, meaning they do not necessarily represent typical medical consultations.
In the emergency department evaluation, MAI-DxO achieved 80.3% accuracy, compared with 46.8% for participating physicians. The highest-performing physician in this assessment reached 62.5%.
Researchers also observed differences in the types of mistakes made. The AI system’s incorrect answers were generally related to the correct diagnosis but differed in specificity, while physicians’ incorrect answers tended to be further from the established diagnosis.
These results indicate that the system could have potential in complex diagnostic reasoning, although accuracy on historical cases does not establish how effectively it would support physicians treating patients in real time.
Balancing Diagnostic Accuracy With Costs
Beyond diagnostic performance, the research examined how much the system would spend on investigations to reach a diagnosis.
Without budget restrictions, MAI-DxO generated higher estimated testing costs than physicians. On complex cases, its average diagnostic expenditure was $5,431 per case, compared with $4,725 for physicians. For emergency department cases, the corresponding figures were $4,979 and $3,400.
Researchers subsequently evaluated configurations that imposed spending limits, testing whether the system could maintain relatively high diagnostic accuracy while reducing the number or cost of investigations.
On complex cases, one configuration achieved 80.9% accuracy at an estimated cost of $2,241 per case. A lower-budget configuration recorded 79.6% accuracy at $1,303.
For emergency department cases, a budget-constrained version achieved 73.2% accuracy at an estimated cost of $2,634 per case. Another configuration reached 69.5% accuracy at $2,037 per case. Both results exceeded the average diagnostic accuracy recorded by participating physicians.
The findings suggest that budget constraints can help balance diagnostic performance with testing expenditure. However, these figures represent simulated costs calculated using standardised US hospital pricing, not verified savings from actual hospitals.
The analysis also did not account for several important considerations, including patient discomfort, the risks of invasive procedures, and delays while waiting for test results.
Microsoft AI Health’s head of AI, Harsha Nori, highlighted the importance of evaluating not only whether a system reaches the correct diagnosis but also how carefully it approaches the diagnostic process.
Limitations and Questions About Clinical Use
Despite the promising results, the research has several limitations that must be addressed before the technology can be considered a reliable clinical tool.
The benchmark relied on text-based medical records and did not directly evaluate the system’s ability to interpret medical images, pathology slides, or other forms of clinical data. Real consultations also involve incomplete or inaccurate histories, changing symptoms, and uncertainty that may not be fully represented in historical records.
Additionally, the evaluated cases had established diagnoses. The benchmark did not include healthy patients or routine benign consultations, limiting the conclusions that can be drawn about performance across the full spectrum of healthcare.
The study measured diagnostic accuracy rather than treatment decisions, patient safety, or clinical outcomes. Consequently, the findings do not establish whether using MAI-DxO would improve patient health or outperform physicians in everyday clinical practice.
The research was conducted by Microsoft employees who hold equity in the company. Eight authors were also named on a patent application associated with the evaluation framework, an additional consideration when assessing the findings.
Clinical Trials Will Determine the Next Steps
Microsoft’s research highlights the potential for AI systems to support medical diagnosis by gathering patient information systematically, evaluating competing explanations, and considering the financial implications of diagnostic testing.
If validated in clinical settings, systems such as MAI-DxO could potentially assist physicians with difficult cases, provide additional diagnostic perspectives, and help healthcare providers evaluate testing strategies.
However, translating benchmark results into practical improvements will require further evaluation involving real patients, clinical workflows, and measurable health outcomes. Researchers will also need to establish how these systems handle uncertainty, avoid harmful recommendations, and integrate with the judgment of qualified medical professionals.
For now, MAI-DxO represents a promising research development rather than a replacement for doctors. Its ultimate value will depend on whether the reported diagnostic performance can be reproduced safely and consistently in real-world healthcare environments.
Source: https://www.artificialintelligence-news.com/news/microsoft-ai-diagnostic-tool/
