Publication/Presentation Date

7-25-2014

Abstract

Abstract

The main objective of this study is to determine whether or not a correlation factor exists between USMLE and COMLEX-USA scores. Due to the large number of residencies that accept both DO and MD applicants, it would potentially hold great value if the two scores were correlated and they could be interchanged. In this study, ERAS was used to sort through and collect data from past emergency medicine applicants. Due to the vast number of applicants, a relatively large sample size will be used. The secondary objective of the study is to determine the correlation factor by manipulating various demographics such as age, gender, examination year and application year. Although the data has been collected and coded, the database is yet to be analyzed by a biostatistician making it impossible to determine if a correlation factor does or does not exist. Future endeavors of this study include having the data analyzed and possibly trying to find formula that can be used to convert either a USMLE or COMLEX score into the other.

Background / Literature Review

The Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) and the United States Medical Licensing Examination (USMLE) are a series of standardized medical licensing examinations used by osteopathic (DO) and allopathic (MD) medical schools, respectively. The comprehensive COMLEX-USA is comprised of Level 1, which assesses basic science knowledge and mechanisms of medicine and health, and Level 2 which tests medical problem solving skills and clinical concepts and principles. Similarly, the USMLE is partly comprised of Step 1 which assesses sciences basic to practice of medicine and mechanisms underlying health, disease, and modes of therapy, and Step 2 which assesses ability to apply medical knowledge, skills, and understanding of clinical science.

Due to the similarities between the two examinations and the large number of residency programs that accept both COMLEX-USA and USMLE scores, determining if and how the scores are interchangeable could allow residency program directors to better evaluate DO and MD applicants. Studies in the past have tried to find the correlation between USMLE Step 1 and COMLEX-USA Level 1 and USMLE Step 2 and COMLEX-USA Level 2, however the results varied from study to study. One study that only used a sample size of 90 EM residents found that a correlation did not exist between USMLE Step 1 and COMLEX-USA Level 1(Sarko et al 2010), while another study that used 580+ internal medicine residents did find a correlation of 0.85 between COMLEX-USA Level 1 and USMLE Step 1(Chick et al 2010).

Utilizing the thousands of osteopathic emergency medicine residency applicants received by Lehigh Valley Health Network over the years, this study seeks to find the correlation factor of both USMLE Step 1 and COMLEX Level 1 and USMLE Step 2 and COMLEX Level 2 scores. Unlike studies in the past, an EM-focused study with a relatively large sample size has never been done. Also, this study utilizes applicant test scores dating back to 2006, allowing for comparison of the correlation factor in various years.

Determining the existence of a correlation factor between USMLE and COMLEX-USA scores has become more important in light of recent events. In 2012, the non-profit private council responsible for accrediting and overseeing all MD residencies and most DO residencies in the United States, the American Council for Graduate Medical Education (ACGME), plans to assimilate with two other osteopathic medicine-focused organizations, the American Osteopathic Association (AOA) and the American Association of Colleges of Osteopathic Medicine (AACOM) by 2015. This merger will create a single accreditation system responsible for overseeing all medicine residencies in the United States, exacerbating the need for a method to fairly evaluate resident applicants and candidates.

Research Question

If any, what is the correlation factor between COMLEX-USA and USMLE scores of osteopathic emergency medicine residency applicants?

Secondary Questions

How does the correlation factor among applicants differ from year to year (both application year and examination year)?

If and how is the correlation factor of USMLE and COMLEX-USA scores related to various demographics such as age, gender, and the osteopathic medical school attended?

How do the USMLE and COMLEX-USA scores of Lehigh Valley Health Network EM residency applicants compare to those of other EM residencies across the nation?

Study Hypothesis

An analysis of USMLE Step 1 and COMLEX-USA Level 1 scores will result in a definitive correlation factor. A separate correlation factor will arise when USMLE Step 2 and COMLEX-USA Level 2 are analyzed.

Study Goals

The primary objective of this study is to determine the correlation factor between USMLE and COMLEX scores of EM residency applicants, if one does exist. Another goal of this study is to determine if the correlation factor differs when looking at it from year to year or when comparing the correlation factor among various demographics such as age, gender, and the school attended. The last aim of this study is to determine if the USMLE and COMLEX-USA scores of applicants received by Lehigh Valley Health Network differ from those received by EM residencies across the US.

Methods

A database was created by collecting data from Electronic Residency Application Service (ERAS) of past osteopathic medical school applicants that had taken equivalent parts of the USMLE and COMLEX-USA (i.e. COMLEX-USA Level 1 with USMLE Step 1, COMLEX-USA Level 2 with USMLE Step 2, or COMLEX-USA Levels 1 & 2 with USMLE Steps 1 & 2). Only the applicants that applied between July 1, 2006 and December 31, 2013 were used. Information other than age, gender, examination year, and the name of osteopathic medical school attended was de-identified. To make sure data from ERAS was recorded correctly into the database, 10% of the sample size was reviewed and checked for quality assurance.

Results

After compiling the database, the sample size was determined to be 556 eligible applicants. Of those applicants, 359 or 64.6% were male and 197 or 35.4% were female. Also, the sample included applicants from 27 different osteopathic medical schools. The age of the applicants ranged from 23 to 54 with 28 being the average age.

Discussion / Conclusion

Due to the lack of a complete analysis, it is impossible to definitively conclude whether or not a correlation factor between USMLE and COMLEX-USA scores of emergency medicine residents exists, and therefore also impossible to prove the hypothesis to be correct or incorrect.

The next step in this study will be to analyze the database of matched scores. In order to do so, yearly means, standard deviations, and Pearson correlation coefficients will be compared. To determine if the correlation factor is changing from either one application year to another or from one examination year to another, a time series analysis will be performed on the correlation coefficients. To determine if the age of the applicant is linked with the correlation factor between USMLE and COMLEX-USA scores, a t-test will be used. On the other hand, a chi-square test will be used for gender.

If the preliminary analysis of the database is promising, future analysis could include data from emergency medicine applicants of 2014. Secondly, if a strong or fairly significant correlation factor exists between the scores, future ventures could include trying to determine a formula or equation that could be used to convert COMLEX-USA scores to USMLE or vice versa.

References

Chick, D. A., Friedman, H. P., Young, V. B., & Solomon, D. (2010). Relationship Between COMLEX and USMLE Scores Among Osteopathic Medical Students who Take Both Examinations. Teaching and Learning in Medicine, 22(1), 3-7.

Sarko, J., Svoren, E., & Katz, E. (2010). COMLEX-1 and USMLE-1 Are Not Interchangeable Examinations. Academic Emergency Medicine, 17(2), 218-220.

Comments

Mentor: Kathleen Kane

Department(s)

Research Scholars, Research Scholars - Posters

Document Type

Poster

Download

Share

COinS