A Microsoft research team reported that an artificial intelligence system designed to work through medical diagnoses step by step performed better than general physicians on a collection of complex cases. The study, published Thursday in the journal Science, tested the system on more than 300 unusual and difficult medical scenarios.
According to the researchers, the AI was more than twice as accurate as general physicians who used textbooks and the internet when working through the cases and identifying diagnoses. The study also found that diagnostic work-ups designed by the AI system cost less than those conducted by human doctors, though the supplied report does not provide specific cost figures or a detailed breakdown of the comparison.
The system uses what Microsoft calls an “orchestrator” to organize the diagnostic process. Instead of receiving a full set of symptoms, test results and imaging at once and returning a diagnosis, the tool determines what information it needs to gather. It selects questions about a patient’s medical history, considers relevant findings from a physical examination and decides which tests to order. It can then assess results, seek additional testing and revise its working hypotheses as the process continues.
Microsoft researchers described the approach as different from the all-at-once method used by many existing AI models. Those systems typically analyze the information already provided and produce an answer, rather than directing a staged investigation.
The research team also tested the tool on scenarios based on a major database of emergency-room admissions. The report said conventional comparisons of AI systems with physicians often rely on multiple-choice questions. Microsoft AI health-unit corporate vice president Dr. Dominic King said such tests do not reflect the way patients present in practice, where clinicians must determine what information and examinations are needed.
Harsha Nori, head of AI at Microsoft AI Health, said the researchers evaluated not only diagnostic accuracy but also how carefully and efficiently the system conducts a work-up. The report did not provide further figures on the system’s performance in the emergency-admission scenarios or the amount saved on testing.
The next planned evaluation will take place through a collaboration with Beth Israel Deaconess Medical Center in Boston. Researchers will assess whether the tool can help triage patients waiting in emergency department waiting rooms. That work is a planned test; the supplied material does not report results from the hospital evaluation or establish that the AI system is being used for patient care there.
The findings are from Microsoft’s research and do not, on their own, show how the tool would perform across routine clinical settings or replace a physician’s judgment. The next hospital-based test is intended to examine a specific use—emergency-department triage—beyond the cases used in the study.
