Large Language Models May Aid Emergency Department Triage

TUESDAY, May 14, 2024 (HealthDay News) -- Large language models (LLMs) could enhance emergency department triage workflows, according to a study published online May 7 in JAMA Network Open.

Christopher Y.K. Williams, M.B., B.Chir., from the Bakar Computational Health Sciences Institute at the University of California, San Francisco, and colleagues examined whether an LLM accurately assessed clinical acuity in the emergency department. The analysis included 10,000 pairs of emergency department visits (Jan. 1, 2012, to Jan. 17, 2023) with nonequivalent Emergency Severity Index (ESI) scores balanced for each of the 10 possible pairs of five ESI scores. The LLM examined acuity based on presenting histories extracted from physician notes.

The researchers found that the LLM correctly inferred the patient with higher acuity for 8,940 of 10,000 pairs (accuracy, 0.89). A previous comparator LLM had a lower performance (accuracy, 0.84). For the manually classified 500-pair subsample, LLM performance (accuracy, 0.88) was similar to that of the physician reviewer (accuracy, 0.86).

"Our findings suggest that an LLM could perform the complex task of evaluating clinical acuity," the authors write. "The integration of LLMs into emergency department workflows could enhance triage processes while maintaining triage quality and warrants further investigation."

Several authors disclosed ties to the technology or pharmaceutical industries.

Abstract/Full Text

Editorial