AI Agent Finds 6x More Cases Than Previous System
An artificial intelligence agent developed by Parkland Health in Dallas demonstrated 99% sensitivity in identifying patients who require follow-up procedures from radiology reports — six times more effective than the macro-based system previously in use. The study, published in NEJM Catalyst, offers one of the first concrete, validated use cases for agentic AI in radiology, a topic that has quickly become one of the hottest in the specialty.
What Is Agentic AI?
Agentic AI is a modality of artificial intelligence capable of working autonomously to complete tasks with minimal human supervision. Unlike traditional AI models that respond to specific prompts, AI agents can analyze information, make intermediate decisions, and execute actions in sequence — such as reading a report, extracting follow-up recommendations, classifying urgency, and integrating findings into departmental workflow.
In healthcare, agentic AI is being applied to a wide range of tasks, from improving health system operations to clinical and administrative functions. The Parkland Health study addresses one of the trickiest tasks in radiology: ensuring patients with suspicious findings comply with recommendations for follow-up procedures.
The Problem of Unfulfilled Recommendations
Previous studies have documented low rates of adherence to radiologist recommendations for follow-up imaging — possibly as low as 50%. This creates the uncomfortable possibility of missed opportunities that could have major patient-care ramifications. The dilemma is compounded by the use of structured note templates in EHRs, as improper use or modification of these macros can lead to missed notifications.
Considering Parkland’s annual volume of 500,000 imaging studies, the AI agent could identify 21,500 follow-up cases per year. Many of these could be serious issues, such as new cancer diagnoses or pathologies requiring surgical intervention. The application of AI to diagnostic imaging has already demonstrated value in multiple scenarios, and automated follow-up represents another critical dimension.
Study Results
To address the problem, Parkland researchers developed an AI agent based on Meta’s open-source large language model Llama 3 70B, which reviews clinical impressions, extracts important details for follow-up, and integrates findings into departmental workflow to enable patient outreach.
In tests on 10,000 radiologist notes, the AI agent achieved: an overall detection rate of approximately 5.1% (slightly lower than other published studies at 8% to 12%); far higher sensitivity than Parkland’s previous macro-based notification system (99% vs. 16%), correctly flagging 6X more cases (513 vs. 83); higher accuracy (99% vs. 58%); and 94% accuracy for characterizing follow-up timing, recommended procedure, and underlying abnormality.
Implications for Radiology Practice
The study shows that agentic AI isn’t some technogeek’s far-off dream — it’s a useful tool on the verge of real-world implementation, with the potential to improve patient care without overburdening radiology staff. For imaging services using AI tools for triage, adding a follow-up agent complements the diagnostic cycle, ensuring important findings don’t get lost between report issuance and clinical action.
In healthcare systems where care continuity can be fragmented across different providers and systems, an automated follow-up agent could have even greater impact, tracking patients who changed services or didn’t return for follow-up examinations.
The Future of Agentic AI
The use of an open-source model (Llama 3) is particularly relevant, as it demonstrates that agentic AI doesn’t necessarily require expensive proprietary solutions. Healthcare institutions with technical capacity could adapt and train their own agents, customizing them for their specific follow-up protocols. The combination of agentic AI with EHR data and PACS systems represents the next frontier of augmented radiology — where AI not only assists in diagnosis but also ensures diagnostic recommendations translate into concrete clinical actions.
Source: The Imaging Wire