Overview of Various Assessment Schemes
A recent study conducted by the University of Michigan has revealed that a proprietary artificial intelligence system, known as the Epic Sepsis Model, integrated into Epic’s electronic medical record software, lacks the ability to distinguish between high and low-risk patients before they undergo treatment.
This tool, which automatically generates sepsis risk assessments for hospitalized patients every 20 minutes, aims to aid clinicians in identifying potential sepsis cases early on. However, the study indicates that the AI predictions do not provide additional insights beyond what clinicians already discern from patient data.
The research, led by associate professor Tom Valley and computer science expert Jenna Wiens, highlights a crucial issue concerning the timing of data availability to the AI system. The evaluation, based on 77,000 adult patients at the University of Michigan Health, demonstrated that the AI’s performance varied significantly depending on when the risk assessments were conducted.
While the AI displayed an 87% accuracy rate in identifying high-risk patients when considering all data points throughout a patient’s hospitalization, its accuracy dropped to 62% when utilizing data recorded before the onset of sepsis symptoms. Particularly noteworthy was the model’s ability to predict sepsis risk before blood culture tests were ordered, where its accuracy decreased to only 53%.
The study suggests that the AI model heavily relies on cues from diagnostic tests and treatments, which are often administered after clinicians already suspect sepsis. Consequently, the AI’s predictions may not offer substantial benefits in clinical decision-making.
In conclusion, the research emphasizes the importance of evaluating AI models within the context of clinical workflows to determine their practical utility for healthcare professionals.
Source:
