Research Scholars Poster Presentation

Resident Confidence and Retained Medical Knowledge in Cardiopulmonary Resuscitation Affected By Simulated Mock Code Blue Session

Miranda HaslamFollow
Rachel Lipner Do, Lehigh Valley Health NetworkFollow
Katie Mastoris DO, Lehigh Valley Health NetworkFollow
Matthew Miles DO, Lehigh Valley Health NetworkFollow
Sarah Park DO, Lehigh Valley Health NetworkFollow
Ryan Rogers DO, Lehigh Valley Health NetworkFollow
Stacey Smith, Lehigh Valley Health NetworkFollow

Publication/Presentation Date

7-31-2015

Abstract

Resident Confidence and Retained Medical Knowledge in Cardiopulmonary Resuscitation Affected By Simulated Mock Code Blue Session

Research Scholar: Miranda Haslam, The College of William & Mary, Williamsburg, Virginia

Authors: Rachel Lipner, DO; Katie Mastoris, DO; Matthew Miles, DO; Sarah Park, DO; Ryan Rogers, DO; Stacey Smith, MD; and Miranda Haslam

Department of Internal Medicine, Lehigh Valley Health Network, Allentown, Pennsylvania

Abstract

The purpose of this observational quality improvement study was to evaluate the impact of mock code blue simulation on internal medicine residents’ knowledge of Advanced Cardiovascular Life Support (ACLS) guidelines for in-hospital cardiopulmonary arrest, as well as the residents’ self-perceived confidence in their ability to lead a response. Prior to beginning the academic year, internal medicine residents at the Lehigh Valley Health Network (LVHN) Cedar Crest Campus completed two surveys: a confidence survey and a knowledge survey based on ACLS guidelines. Following participation in a scheduled mock code blue event during their rotation in the Medical Intensive Care Unit (MICU), residents completed the same two surveys. Prior to simulation, the average 5-point Likert scale score for the confidence survey was a 3.7, and the mode confidence was 4. After simulation, the average confidence increased to 3.9 and the mode confidence increased to 5. Prior to simulation, residents answered 72.8% of the ACLS knowledge survey questions correctly. After simulation, residents answered 76.0% of the ACLS knowledge survey questions correctly. Internal medicine residents reported an increase in confidence in responding to cardiopulmonary resuscitation events following a mock code blue simulation session. Confidence increased particularly in the areas of leadership, placing IO and CVC lines, and choosing medications. Clinical knowledge of ACLS guidelines also improved after simulation. Increased confidence and clinical knowledge retention following simulation suggest that mock code blue training can improve leadership and adherence to ACLS protocols, ultimately improving patient outcomes during in-hospital cardiopulmonary arrests.

Key Words: resident, cardiopulmonary arrest, cardiopulmonary resuscitation (CPR), code blue, education, simulation Advanced Care Life Support (ACLS), knowledge, confidence, surveys, internal medicine

Introduction

Making the transition from resident to attending physician requires accurate and timely decisions regarding patient care. This is exemplified during cardiopulmonary resuscitation events, which demand physicians respond with confidence, skill, and strong leadership abilities. One key element of residency training is to provide resident physicians with the confidence, exposure, and knowledge necessary to be the leader of a cardiac arrest response team during a cardiopulmonary resuscitation event.

There have been many studies, reviews, and commentaries throughout the medical literature that highlighted flaws within resident training regarding cardiopulmonary resuscitation. Hayes et al. revealed that internal medicine residents often feel unprepared as leaders of cardiac arrest teams. They determined that residents perceive deficits in their training and supervision to care for critically ill patients as cardiac arrest team leaders. Further compounding the issue, Mickelsen et al. voiced concern that current numbers of in-hospital code blue events were potentially insufficient to provide adequate experience without supplemental practice for trainees. They conducted a single-centered, retrospective review of monthly code blue frequency and detected a 41% overall reduction in code blue events, as well as elucidated the fact that at their facility, code blue events decreased by 13% annually from 2002 to 2008. Concerned for the overall reduction in medical training, Yang et al. discussed possible strategies to compensate for less “in-the-field” exposure by maximizing the “learning yield per event” and using simulation training methods. In 2006, use of simulation-based education programs enabled Wayne et al. to show improved skill and knowledge of resident performance with simulated ACLS events and maintenance of those skills after 14 months.

The focus of our study is to evaluate the impact of a simulation-based education program, utilizing mock cardiopulmonary arrest simulation sessions, on residents’ self-perceived confidence and skills in handling cardiopulmonary resuscitation situations. Mock cardiopulmonary arrest simulation sessions have been implemented by the LVHN internal medicine residency program during the 2014-2015 academic year. Our focus was to determine the effect the mock sessions have on current internal medicine residents across all post-graduate years. We hypothesized that these simulation-training sessions would not only lead to improvement in resident confidence and skills, but ultimately improved technique and accuracy in fulfillment of ACLS guidelines during in-hospital cardiopulmonary arrests.

Materials and Methods

The subject group was comprised of internal medicine residents at LVHN. All residents, post-graduate years (PGY) 1 through 3, were invited to complete the surveys, excluding the residents involved in the study design. A total of 43 residents completed the pre-simulation surveys. The 43 participants were comprised of 29 PGY1 residents, six PGY2 residents, and seven PGY3 residents, with one participant’s post-graduate year not reported. There were 17 males and 25 females, with one participant’s gender not reported. A total of 14 residents were able to experience a mock code simulation and complete the subsequent surveys during the duration of the study, with eight males and six females responding. The responding participants were comprised of eight residents in PGY1, two in PGY2, and four in PGY3.

The observational quality improvement study was carried out over a one-year duration, beginning with the commencement of the 2014-2015 academic year. Pre-simulation surveys were completed in June and July of 2014, and post-simulation surveys were completed on the date of the scheduled simulation. All simulations were conducted during residents’ 4-week rotation in the Medical Intensive Care Unit (MICU) at LVHN Cedar Crest campus, and the simulations were run by one of three participating academic intensivists. The mock code blue simulations included a computerized and automated patient simulator with real-time hemodynamic displays.

The residents completed two surveys: a confidence survey and a 13-question knowledge survey based on ACLS guidelines. The confidence survey was comprised of two parts, adapted from a previous validated study by Schaik et al. The first part of the survey was an assessment of self-perceived confidence in areas of technical and leadership skills. Technical skills assessed were broken down during survey design into three levels: basic, advanced, and expert. Basic technical skills assessed included recognizing when and knowing how to get additional help, ability to position and clear the airway, ability to perform bag-valve-mask ventilation, ability to identify hemodynamic instability, and ability to perform adequate chest compressions. Advanced technical skills assessed included abilities to perform and choose medications for endotracheal intubation, place intravenous (IV) lines and intraosseous (IO) lines, recognize and treat different cardiac arrhythmias, choose synchronized cardioversion or defibrillation, and operate the defibrillator. The expert technical skill assessed was the ability to perform a central line (CVC). Leadership skills assessed included abilities to take charge as team leader, delegate tasks, and supervise team members. Self-reported confidence levels for each skill were scored on a 5-point Likert scale, with 1 being the lowest confidence and 5 being the highest confidence.

The second part of the confidence survey was an informational section where residents reported on the number of codes they had attended during their residency thus far, both simulated mock codes and real codes, as well as what roles they played during the codes. Residents also reported whether debriefing was part of the code experience and whether or not they found it helpful if it had occurred.

The 13-question knowledge survey was based off of ACLS cardiopulmonary resuscitation event response guidelines and was designed to be similar to the ACLS certification test taken biennially by physicians. The survey assessed clinical knowledge in a four-stem multiple-choice question format and included questions regarding medication selection and dosing, Basic Life Support (BLS) protocols, and rhythm strip interpretation.

Results

Prior to simulation, residents felt most confident (5 on the Likert scale) with recognizing when and how to get additional help, being able to position the airway and perform bag-valve-mask ventilation, and performing chest compressions. Residents felt least confident (2 or lower on the Likert scale) with choosing medications for endotracheal intubation and placing an IO line. Resident confidence increased after simulation in the areas of placing IO lines, operating defibrillators, knowledge of medications for various cardiac arrhythmias, performing CVC lines, running the code as team leader, delegating tasks, and supervising team members. Resident confidence decreased after simulation in the areas of positioning the airway, clearing the airway, performing bag-valve-mask ventilation, performing endotracheal intubation, and placing IV lines. Prior to simulation, the overall average confidence expressed on the 5-point Likert scale was a 3.7 and the overall mode confidence expressed was 4. After simulation, the overall average confidence expressed on the 5-point Likert scale was 3.9 and the overall mode confidence expressed was 5. Confidence survey results are depicted in a table in Figure 1.

Following simulation, residents reported an increase in the number of times they had played the roles of airway manager and team leader, and the number of times they had operated the defibrillator during mock codes. They also reported an increase in the number of real codes they had attended, as well as the number of times they had done chest compressions during real codes. Few residents reported that debriefing sessions occurred after real codes, though they unanimously reported them as useful when they did occur. Similarly, debriefing sessions were reported as useful after mock codes, where a debriefing session was most often reported as having occurred.

Prior to simulation, residents on average answered 72.8% of ACLS knowledge survey questions correctly. The most commonly missed question, answered correctly by only 30% of residents, was the question concerning proper precautions for transcutaneous pacing. The question regarding medication administration via endotracheal tube was answered correctly 40% of the time, and the question regarding depth of chest compressions for adult CPR was answered correctly 51% of the time. After simulation, residents on average answered 76.0% of ACLS knowledge survey questions correctly. Responses to the question regarding proper precautions for transcutaneous pacing decreased from 30% correct before simulation to 14% correct after simulation, making it again the most commonly missed question. The percent of correct responses received increased after simulation for all questions regarding medication selection and dosing, from 67.1% to 87.1%. Additionally, after simulation 100% of residents correctly answered all questions regarding BLS protocol and reading rhythm strips. After simulation, the residents performed better on all ACLS survey questions except questions regarding 0₂ saturation monitoring following return to spontaneous circulation, the most common reversible causes of PEA, and precautions for transcutaneous pacing. Knowledge survey results are depicted in a table in Figure 2.

Discussion

In an effort to explore the affects of mock cardiopulmonary resuscitation simulation on internal medicine residents’ knowledge of ACLS guidelines for in-hospital cardiopulmonary arrest response, as well as the residents’ self-perceived confidence in their ability to lead a response, we administered 43 pre-simulation and 14 post-simulation survey sets. The survey sets contained one survey to assess confidence and another to assess clinical knowledge of ACLS guidelines. On average, resident confidence increased and knowledge survey scores improved following simulation, supporting the hypothesis that simulation-training sessions would lead to improvement in resident confidence and skill in implementation of ACLS guidelines for cardiopulmonary resuscitation.

Our results support the finding by Wayne et al. that a simulation-based educational program improves the quality of care provided by residents during a real time ACLS event. Improved knowledge of ACLS guidelines following simulation, leading to better implementation of those guidelines during real-time events, would ultimately improve patient care. Our results also support findings by Sam et al., who revealed that simulation-trained residents show better adherence to clinical standards, a finding supported by the improvement in knowledge survey scores documented following simulation in our study. Schaik et al. discovered that confidence in resuscitation skills among pediatric residents increases following mock codes, which is supported by our finding that confidence among internal medicine residents similarly increased following simulated mock codes. Compared to what is found in the literature, our study design was unique in that it allowed us to analyze the affects of simulation training on both the self-perceived confidence and clinical knowledge retention of internal medicine residents. While Schaik et al. indicated in their study that self-assessed confidence does not necessarily equate positively with actual skills, our two-fold analysis of confidence and clinical knowledge allowed us to show that both measures were positively correlated with simulation-based cardiopulmonary resuscitation response training.

On the ACLS knowledge survey, the most commonly missed question, answered correctly by only 30% of residents before simulation and 14% after simulation, was the question concerning proper precautions for transcutaneous pacing. It was retrospectively decided by study designers that the wording of the question was confusing, potentially leading to artificially high rates of incorrect responses. The question regarding depth of chest compressions for adult CPR was answered correctly only 51% of the time before simulation, which is of interest, as most residents reportedly felt very confident (5 on the Likert scale) at performing chest compressions. After simulation, however, 100% of respondents answered the question correctly. Resident responses improved following simulation for all other ACLS survey questions, except for the question concerning 0₂ saturation monitoring, the question regarding the most common reversible causes of PEA, and the question regarding transcutaneous pacing.

A possible explanation for the decline in correct response to the question concerning O₂saturation is that LVHN hospital protocol dictates that anesthesia reports to all code blues and is responsible for airway management. This could also explain why resident confidence decreased after simulation in the areas of positioning the airway, clearing the airway, performing bag-valve-mask ventilation, and performing endotracheal intubation. Similar studies conducted at other facilities where having an anesthesia team manage the airway is not the protocol could be comparatively used to elucidate if resident confidence in these areas would increase following simulation if the residents were responsible for airway management.

While 43 residents completed the pre-simulation surveys, only 14 residents were able to experience a mock code blue simulation during the duration of the study. The small sample group of the post-simulation responses could have affected the study with confounding variables and outlier responses. This could also explain why the percentage of correct responses to some knowledge survey questions was higher before simulation than after simulation. While scheduling is a common problem with simulation-based medical education, as the pressures of clinical duties can often take precedence over simulation sessions (McGaghie et al.), future studies would be strengthened by a greater number of post-simulation responses. This could be facilitated by a better practice of scheduling simulation in the MICU, or by scheduling simulation sessions during a lighter rotation.

This study could also have been strengthened by the use of survey response identifiers, giving researchers the ability to compare an individual resident’s responses before and after simulation and allowing correlation between individuals’ confidence and knowledge scores to be made. It would have been interesting to analyze the correlation between confidence and knowledge scores, as Hayes et al. suggested that ACLS competency does not necessarily contribute to perceived adequacy of training, but the increase in both confidence and knowledge shown in this study may indicate otherwise.

In summary, the results of this study indicate that simulation-based medical education in the form of mock code blue events is beneficial for internal medicine residents’ confidence and clinical knowledge retention, and can be utilized to improve leadership and overall adherence to ACLS protocol. Additional studies with larger post-simulation response and the use of participant identifiers will allow for a better analysis of how cardiopulmonary resuscitation simulation can be used to improve patient outcomes. These studies would be facilitated by an effort to make simulation-training sessions more accessible to resident physicians, despite common scheduling difficulties arising from the pressures of clinical duties. Further studies conducted with resident populations outside of internal medicine who also respond to code blue events would also allow for a better understanding of how mock code blue simulation may be used to improve patient outcomes on a hospital-wide scale. In the future, long-term studies used to analyze the affects of implementation of a mock code blue simulation-based medical education program on patient outcomes at this facility would be able to further validate the importance of simulation-based training methods.

Conclusions

Internal medicine residents report increased confidence in responding to cardiopulmonary resuscitation events following a mock code blue simulation, particularly in the areas of leadership, placing IO and CVC lines, and choosing medications. The educational benefit of mock code blue simulation is further supported by improved clinical knowledge, as assessed by the ACLS protocol survey, following simulation. Increased confidence and improved clinical knowledge retention following mock code blue simulation indicate that simulation training can be utilized to improve overall adherence to ACLS cardiopulmonary resuscitation protocols, ultimately improving patient outcomes during in-hospital cardiopulmonary arrests.

References

1. Brett-Fleegler, M., Vinci, R., Weiner, D., Harris, S., Shih, M., & Kleinman, M. (2008). A Simulator-Based Tool That Assesses Pediatric Resident Resuscitation Competency. Pediatrics, 121(3), 597-603. doi:10.1542/peds.2005-1259

2. Hayes, C. W., Rhee, A., Detsky, M. E., Leblanc, V. R., & Wax, R. S. (2007). Residents Feel Unprepared and Unsupervised as Leaders of Cardiac Arrest Teams in Teaching Hospitals: A Survey of Internal Medicine Residents. Critical Care Medicine, 35(7), 1668-1672.

3. McGaghie, W., Issenberg, S. B., Petrusa, E., & Scalese, R. (2010). A Critical Review of Simulation-Based Medical Education Research: 2003-2009. Medical Education, 44. 50-63.

4. Mickelsen, S., Mcneil, R., Parikh, P., & Persoff, J. (2011). Reduced Resident “Code Blue” Experience in the Era of Quality Improvement: New Challenges in Physician Training. Academic Medicine, 86(6), 726-730.

5. Sam, J., Pierse, M., Al-Qahtani, A., & Cheng, A. (2012). Implementation and Evaluation of a Simulation Curriculum for Paediatric Residency Programs Including Just-In-Time In Situ Mock Codes. Paediatric Child Health, 17(2). 16-20.

6. Schaik, S., Kohorn, I., & O'Sullivan, P. (2008). Pediatric Resident Confidence in Resuscitation Skills Relates to Mock Code Experience. Clinical Pediatrics, 47(8), 777-783.

7. Wayne, D., Didwania, A., Feinglass, J., Fudala, M., Barsuk, J., & McGaghie, W. (2008). Simulation-Based Education Improves Quality of Care During Cardiac Arrest Team Responses at an Academic Teaching Hospital. CHEST Journal, 133(1), 56-61.

8. Wayne, D., Siddall, V., Butter, J., Fudala, M., Wade, L., Feinglass, J., & McGaghie, W. (2006). A Longitudinal Study of Internal Medicine Residents Retention of Advanced Cardiac Life Support Skills. Academic Medicine, 81(10), 9-12.

9. Yang, J., & Howell, M. (2011). Commentary: Is the Glass Half Empty? Code Blue Training in the Modern Era. Academic Medicine, 86(6), 680-683.

Figures

Figure 1. Confidence Survey Results. The confidence survey administered to residents was designed to evaluate self-perceived confidence in both technical and leadership skills. Confidence was scored on a 5-point Likert scale and the mode response to each question was calculated using descriptive statistics.

Mode

Survey Question

BEFORE

AFTER

1. I recognize when to get additional help

2. I know how to get additional help

3. I am able to position the airway

4. I am able to clear the airway

5. I am able to perform bag-valve-mask ventilation

6. I am able to identify hemodynamic instability

7. I am able to perform adequate chest compressions (2 inches deep, for 2 minutes)

8. I am able to perform endotracheal intubation

9. I am able to choose medications for endotracheal intubation

10. I am able to place IV line

11. I am able to place an IO line

12. I am able to recognize different cardiac arrhythmias

13. I know when to choose synchronized cardioversion or defibrillation

14. I am able to operate the defibrillator

15. I know which medications to use for different cardiac arrhythmias

16. I am able to perform an central line

17. I am able to take charge and run the code as team leader

18. I am able to delegate tasks

19. I am able to supervise team members

Figure 2. Knowledge Survey Results. The knowledge survey administered to residents was designed to evaluate residents’ clinical knowledge of ACLS cardiopulmonary resuscitation protocols. Clinical knowledge was assessed using 13 4-stem multiple-choice questions, and the quiz was designed to resemble the ACLS certification test taken biennially by physicians. The percent of residents who answered each question correctly was calculated using descriptive statistics.

Appendix

Attachment 1. Confidence Survey. This confidence survey was adapted from a validated study by Schaik et al. and was designed to evaluate residents’ self-perceived confidence in both technical and leadership skills necessary for responding to cardiopulmonary resuscitation events. The second part of the survey was designed to give researchers insight into the residents’ clinical cardiopulmonary arrest response experience.

Date:____________________________ Gender: ________________________

PGY: ____________________________ Age: ____________________________

Please circle the number that represents your confidence level during a cardiopulmonary resuscitation code (code blue) with each of the following statements:

I recognize when to get additional help