BACKGROUND: Venous thromboembolism (VTE) prophylaxis is underutilized in hospitalized medical patients. Underutilization might occur as a result of resident practice variation incurred by using a complex risk assessment tool.
OBJECTIVE: To examine what impact repetitive exposure to an electronic point-based VTE risk assessment tool has on resident inter-rater reliability and protocol adherence.
DESIGN: Pre and post intervention cross-sectional cohort study.
SETTING: Single academic center.
PATIENTS: Convenience samples of Internal Medicine residents.
INTERVENTIONS: Residents completed clinical vignettes before and after any exposure to an electronic risk assessment tool and reminder alert. They were asked to make three determinations using a point-based VTE risk assessment tool: risk stratification, identify contraindications, and VTE prevention strategy.
MEASUREMENTS: Inter-rater reliability for risk assessment, contraindications, and VTE prophylaxis strategy and protocol adherence.
RESULTS: Kappa scores for VTE risk assessment did not change, but improved for VTE plan increasing from 0.28 to 0.37. Protocol adherence improved from 71% in 2008 to 79% (P = 0.06). There was a significant decrease in under-prophylaxis (22% to 6%, P < 0.0001) but a significant increase in over-prophylaxis (7% to 16%, P = 0.001).
CONCLUSIONS: Using clinical vignettes, we determined that daily exposure to an electronic risk assessment tool did not improve the inter-rater reliability of a point-based risk assessment tool when used by medical residents. This might be due to inexperienced providers using a complex point-based tool. Overall, adherence improved, and under-prophylaxis decreased, but over-prophylaxis increased. Clinical vignettes are a generalizable method to monitor resident prophylaxis practices and way to identify educational and process improvement opportunities.
KEYWORDS: Resident; Inter-rater reliability; Venous thromboembolism; Agreement; Risk assessment score.
Published In/Presented At
Todoric K, Lehman E, Beck MJ. Using Clinical Vignettes to Evaluate VTE Protocol Adherence. J Clin Med Res. 2012 Apr;4(2):87-94. doi: 10.4021/jocmr766w. Epub 2012 Mar 23. 8.
Medicine and Health Sciences
Department of Medicine