Background
Researchers have discovered that artificial intelligence systems called large language models (LLMs), like ChatGPT and Claude, can effectively analyze patient and community feedback in emergency research settings. In a new study published in Scientific Reports, investigators evaluated how well these AI tools could assess community perspectives within the Exception from Informed Consent (EFIC) process, which is critical for studying lifesaving interventions where traditional consent isn’t possible [1].
Study Question
Could large language models analyze patient perspectives in EFIC interviews with performance comparable to human reviewers?
Study Design
The research team analyzed 102 EFIC community interviews from 9 sites participating in the PediDOSE trial, which is investigating optimal medication dosing for children having seizures during ambulance transport. They used multiple AI systems, including GPT-4, to analyze over 3,600 responses and compared their performance to human reviewers on two key tasks:

- Assessing sentiment (positive to negative feelings)
- Organizing responses into meaningful thematic categories (e.g., trust, safety, consent concerns)
Key Findings
The AI systems showed substantial agreement with human reviewers. When identifying whether responses expressed support or concern, LLMs matched human reviewers more than 95% of the time when identifying whether responses showed support or concern— demonstrating strong potential to assist in interpreting large volumes of community feedback. The AI tools were also highly effective at grouping similar responses into thematic categories, performing at nearly the same level of accuracy as trained human reviewers.
Caution
While promising, the AI systems sometimes interpreted emotional content differently than humans, showing a tendency to classify responses in more extreme ways. This highlights that these tools should complement rather than replace human judgment in understanding community feedback. The goal is to make the process more efficient without losing the human element in decision-making.
Take home messages |
---|
|

Why is this important for patients and caregivers
EFIC is essential for researching critical interventions like automated defibrillators or emergency seizure treatments when traditional consent isn’t feasible. The process requires interviewing community stakeholders to understand concerns and support before studies begin. However, analyzing hundreds of interviews across multiple research sites becomes challenging for research teams and ethics boards. Better tools could ensure community voices are properly understood and incorporated into research design.
Acknowledgements
Special thanks to SimTech for their assistance in this study.
References
- Kornblith AE, Singh C, Innes JC, et al. Analyzing patient perspectives with large language models: a cross-sectional study of sentiment and thematic classification on exception from informed consent. Sci Rep. 2025;15(1):6179. Published 2025 Feb 20. doi:10.1038/s41598-025-89996-w. PMID 39979559
- Ward CE, Adelgais KM, Holsti M, et al. Public support for and concerns regarding pediatric dose optimization for seizures in emergency medical services: An exception from informed consent (EFIC) trial. Acad Emerg Med. 2024;31(7):656-666. doi:10.1111/acem.14884. PMID 38450918