Artificial intelligence improves urologic oncology patient education and counseling.
Publication/Presentation Date
10-1-2024
Abstract
INTRODUCTION: Patients seek support from online resources when facing a troubling urologic cancer diagnosis. Physician-written resources exceed the recommended 6-8th grade reading level, creating confusion and driving patients towards unregulated online materials like AI chatbots. We aim to compare the readability and quality of patient education on ChatGPT against Epic and Urology Care Foundation (UCF).
MATERIALS AND METHODS: We analyzed prostate, bladder, and kidney cancer content from ChatGPT, Epic, and UCF. We further studied readability-adjusted responses using specific AI prompting (ChatGPT-a) and Epic material designated as Easy to Read. Blinded reviewers completed descriptive textual analysis, readability analysis via six validated formulas, and quality analysis via DISCERN, PEMAT, and Likert tools.
RESULTS: Epic met the recommended grade level, while UCF and ChatGPT exceeded it (5.81 vs. 8.44 vs. 12.16, p < 0.001). ChatGPT text was longer with more complex wording (p < 0.001). Quality was fair for Epic, good for UCF, and excellent for ChatGPT (49.5 vs. 61.67 vs. 64.33). Actionability was overall poor but particularly lowest (37%) for Epic. On qualitative analysis, Epic lagged on all quality measures. When adjusted for user education level (ChatGPT-a and Epic Easy to Read), readability improved (7.50 and 3.53), but only ChatGPT-a retained high quality.
CONCLUSIONS: Online urologic oncology patient materials largely exceed the average American's literacy level and often lack real-world utility for patients. Our ChatGPT-a model indicates that AI technology can improve accessibility and usefulness. With development, a healthcare-specific AI program may help providers create content that is accessible and personalized to improve shared decision-making for urology patients.
Volume
31
Issue
5
First Page
12013
Last Page
12018
ISSN
1488-5581
Published In/Presented At
Shah, Y. B., Ghosh, A., Hochberg, A., Mark, J. R., Lallas, C. D., & Shah, M. S. (2024). Artificial intelligence improves urologic oncology patient education and counseling. The Canadian journal of urology, 31(5), 12013–12018.
Disciplines
Business Administration, Management, and Operations | Health and Medical Administration | Management Sciences and Quantitative Methods
PubMedID
39462532
Department(s)
Administration and Leadership
Document Type
Article