Can AI Help Us Better Understand Community Perspectives in Clinical Trials?

Background

Researchers have discovered that artificial intelligence systems called large language models (LLMs), like ChatGPT and Claude, can effectively analyze patient and community feedback in emergency research settings. In a new study published in Scientific Reports, investigators evaluated how well these AI tools could assess community perspectives within the Exception from Informed Consent (EFIC) process, which is critical for studying lifesaving interventions where traditional consent isn’t possible [1].

Study Question

Could large language models analyze patient perspectives in EFIC interviews with performance comparable to human reviewers?

Study Design

The research team analyzed 102 EFIC community interviews from 9 sites participating in the PediDOSE trial, which is investigating optimal medication dosing for children having seizures during ambulance transport. They used multiple AI systems, including GPT-4, to analyze over 3,600 responses and compared their performance to human reviewers on two key tasks:

exception from informed consent (EFIC) LLM AI analysis
  • Assessing sentiment (positive to negative feelings)
  • Organizing responses into meaningful thematic categories (e.g., trust, safety, consent concerns)

Key Findings

The AI systems showed substantial agreement with human reviewers. When identifying whether responses expressed support or concern, LLMs matched human reviewers more than 95% of the time when identifying whether responses showed support or concern— demonstrating strong potential to assist in interpreting large volumes of community feedback. The AI tools were also highly effective at grouping similar responses into thematic categories, performing at nearly the same level of accuracy as trained human reviewers.

Caution

While promising, the AI systems sometimes interpreted emotional content differently than humans, showing a tendency to classify responses in more extreme ways. This highlights that these tools should complement rather than replace human judgment in understanding community feedback. The goal is to make the process more efficient without losing the human element in decision-making.

Take home messages

  1. LLMs may help investigators and ethics boards (IRBs) rapidly identify concerns, spot patterns, and enhance decision-making—especially in large, complex EFIC datasets. For research involving emergency situations like seizures, cardiac arrest, or trauma, these tools could improve how community perspectives are incorporated into study design and communication.

  1. The future of this technology holds promise: while LLMs won’t replace human judgment, they can amplify our ability to listen at scale in clinical trial design and oversight, potentially making emergency research more responsive to community needs and concerns.

Why is this important for patients and caregivers

EFIC is essential for researching critical interventions like automated defibrillators or emergency seizure treatments when traditional consent isn’t feasible. The process requires interviewing community stakeholders to understand concerns and support before studies begin. However, analyzing hundreds of interviews across multiple research sites becomes challenging for research teams and ethics boards. Better tools could ensure community voices are properly understood and incorporated into research design.

Acknowledgements

Special thanks to SimTech for their assistance in this study.

References

  1. Kornblith AE, Singh C, Innes JC, et al. Analyzing patient perspectives with large language models: a cross-sectional study of sentiment and thematic classification on exception from informed consent. Sci Rep. 2025;15(1):6179. Published 2025 Feb 20. doi:10.1038/s41598-025-89996-w. PMID 39979559
  2. Ward CE, Adelgais KM, Holsti M, et al. Public support for and concerns regarding pediatric dose optimization for seizures in emergency medical services: An exception from informed consent (EFIC) trial. Acad Emerg Med. 2024;31(7):656-666. doi:10.1111/acem.14884. PMID 38450918
Aaron Kornblith, MD

Aaron Kornblith, MD

University of California, San Francisco

Michelle Lin, MD

Michelle Lin, MD

University of California, San Francisco

This website is supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) as part of the Emergency Medical Services for Children Data Center award totaling $3,200,000 with 0% financed with non-governmental sources. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by HRSA, HHS, or the U.S. Government. For more information, please visit HRSA.gov.