Evaluating Artificial Intelligence Chatbots for Patient Education on Thyroid Radiofrequency Ablation: An Analysis of Accuracy, Quality, and Readability.

Publication/Presentation Date

2-1-2026

Abstract

BACKGROUND: Artificial intelligence (AI) chatbots are increasingly being used by patients to obtain medical information. Comparison between platforms with specialty-specific physician assessment remains limited. This study compares the quality, factual accuracy, readability, and consistency of responses generated by four publicly available AI chatbots when answering patient-centered questions about thyroid radiofrequency ablation (RFA).

METHODS: We conducted a cross-sectional analysis of chatbot-generated responses using 20 standardized clinical questions about thyroid RFA. Responses from ChatGPT-4, Gemini, Copilot, and Perplexity were evaluated by six blinded physician reviewers experienced in thyroid RFA using 5-point Likert scales for global quality and factual accuracy. Higher Likert scale scores indicated better performance. Readability and response length were analyzed with established metrics. Statistical significance was defined as

RESULTS: Gemini achieved the highest mean scores for global quality (4.08 ± 0.87) and accuracy (3.76 ± 1.05), with significantly better performance than ChatGPT and Copilot (

CONCLUSIONS: AI chatbot performance varied across platforms for thyroid RFA queries. Chatbots were generally reliable for straightforward factual information but were less dependable for judgment or context-dependent assessments. These AI tools should supplement, not replace, clinician-vetted patient education and institutional materials.

Volume

36

Issue

2

First Page

162

Last Page

168

ISSN

1557-9077

Disciplines

Business Administration, Management, and Operations | Health and Medical Administration | Management Sciences and Quantitative Methods

PubMedID

41791887

Department(s)

Administration and Leadership

Document Type

Article

Link to Full Text

Share

COinS