Computer scientists and healthcare experts affiliated with the University of Michigan have delved into the utilization of artificial intelligence in identifying patients undergoing hospitalization, as outlined in a recent article in JAMA.
The research delved into professionals’ comprehension of AI model functionality’s influence on diagnostic accuracy and potential biases. Despite attempts to utilize image-based AI model explanations to pinpoint consistently biased algorithms, the team discovered that these guides did not effectively assist practitioners in recognizing such biases.
In order to assess the impact of persistent bias in AI on diagnostic precision, a randomized clinical picture study was carried out involving hospitalist physicians, nurse practitioners, and physician assistants from 13 states in the U.S. The study presented nine medical vignettes depicting patients with chronic pulmonary failure, incorporating clinical data like symptoms, physical examinations, laboratory findings, and chest X-rays.
The clinicians were assigned the task of determining the likelihood of specific conditions causing acute respiratory failure in each patient, namely pneumonia, heart failure, or chronic obstructive pulmonary disease. The evaluation commenced with clinicians examining two scenarios devoid of AI input, followed by an additional six scenarios with or without AI explanations. Among these scenarios, three featured standard-model forecasts, while the remaining three comprised intentionally biased predictions.
The findings revealed that when clinicians reviewed patient cases with standard AI predictions and explanations, diagnostic accuracy saw a notable enhancement of 4.4% compared to the baseline accuracy. However, exposure to systematically biased AI model estimations resulted in a significant drop in accuracy by more than 11%, with explanations proving ineffective in mitigating the impact of such biased forecasts.
While the conventional image-based AI explanations fell short in alleviating the adverse effects of bias, the study underscored that while standard AI models could boost clinical accuracy, systematic bias had an adverse impact on diagnostic results.
