Beyond the Score: Bias Investigations to Improve the Fairness of Board Certification Exams.

Publication/Presentation Date

4-1-2026

Abstract

OBJECTIVE: High-stakes examinations, such as those used for board certification, must be valid and fair across demographic groups. The American Board of Emergency Medicine (ABEM) developed a structured process for bias and fairness assessment to identify and refine potentially biased examination items.

METHODS: ABEM implemented a three-phase innovation: (1) statistical flagging of potentially biased items using differential item functioning (DIF) analysis; (2) expert panel qualitative review; and (3) holistic content review by the editorial team.

RESULTS: Over an 8-year period, 3736 items were analyzed. DIF flagged 597 items (16.0%) for review. The expert Bias and Fairness Panel recommended deletion of 62 (10.4% of flagged items) due to construct-irrelevant bias, most often related to racial bias (53.2% of items recommended for deletion), followed by regional jargon or practice variation (43.5%). The process has been adopted consistently and is being extended to new examination formats.

CONCLUSION: A structured, theory-informed bias and fairness assessment process can reduce construct-irrelevant variance in high-stakes learner assessments. This can serve as a replicable model for other certifying bodies and medical educators seeking to enhance their approach to assessment.

Volume

10

Issue

2

First Page

70170

Last Page

70170

ISSN

2472-5390

Disciplines

Medicine and Health Sciences

PubMedID

42079398

Department(s)

Department of Emergency Medicine

Document Type

Article

Link to Full Text

Share

COinS