RadOncRAG: A Novel Retrieval-Augmented Generation Framework Improves Large Language Model Benchmark Performance in Radiation Oncology.

Authors

• Nikhil Gautam Thaker
• Navid Redjal
• Adam Dicker
• Arturo Loaiza-Bonilla
• Trevor Royce
• Vivek Subbiah
• Vikash Deendyal
• Jonathan R Gabriel
• Neena Shetty
• Ajay Choudhri
• Gautam H Thaker

Publication/Presentation Date

11-1-2025

Abstract

PURPOSE: Large language models (LLMs) show promise in assisting knowledge-intensive fields such as oncology, where up-to-date information and multidisciplinary expertise are critical. Traditional LLMs risk hallucinations and reliance on static, possibly outdated data that lack domain-specific context. Retrieval-augmented generation (RAG) has emerged as a strategy to address these issues by incorporating domain-specific information from external knowledge repositories.

METHODS: We evaluated 15 LLMs, including Meta Llama-2/3, generative pretrained transformer (GPT)-3.5/4/4o variants, Claude-3, Gemini-2.0, and DeepSeek-R1. In a zero-shot workflow, each LLM answered 298 scorable questions from the 2021 American College of Radiology in-training examination. We implemented a RAG pipeline (Iridium Model) that transforms user prompts into vector embeddings, queries a specialized radiation oncology database, and merges relevant text with the original prompt to form an augmented query. We compared zero-shot versus RAG-augmented performance.

RESULTS: Larger-parameter LLMs had higher zero-shot accuracy, with six models outscoring graduating residents (

CONCLUSION: Radiation-oncology-specific retrieval-augmented generation pipeline enhances nonreasoning LLM performance in radiation oncology by integrating domain-specific evidence, whereas it does not improve performance of reasoning models. These findings demonstrate that RAG can elevate clinical decision support by enabling simpler, cost-effective nonreasoning models to tackle complex tasks through retrieval capabilities-an efficient alternative to extensive model training that also yields citable, evidence-based explanations.

Volume

9

First Page

2500220

Last Page

2500220

ISSN

2473-4276

Published In/Presented At

Thaker, N. G., Redjal, N., Dicker, A., Loaiza-Bonilla, A., Royce, T., Subbiah, V., Deendyal, V., Gabriel, J. R., Shetty, N., Choudhri, A., & Thaker, G. H. (2025). RadOncRAG: A Novel Retrieval-Augmented Generation Framework Improves Large Language Model Benchmark Performance in Radiation Oncology. JCO clinical cancer informatics, 9, e2500220. https://doi.org/10.1200/CCI-25-00220

Disciplines

Business Administration, Management, and Operations | Health and Medical Administration | Management Sciences and Quantitative Methods

PubMedID

41237352

Department(s)

Administration and Leadership

Document Type

Article