Publication/Presentation Date

7-31-2015

Abstract

Education of Residents in the Operating Room

Megan Arnold1 and Lauren Alden1

Hubert K. Huang, Ph.D.1,2

1Division of Education, Lehigh Valley Health Network

2Research Scholar Program Mentor

Abstract

It is of paramount importance that the technical skills of surgical residents is evaluated. There are obstacles in the operating room setting, however, that interfere with teaching, such as limited time because of efficiency and patient safety. Debriefing after the case has been found to increase the educational value of a case. For this study, residents from the obstetrics and gynecology training program were observed performing four types of sentinel cases. The number of questions the resident asked and the number of teaching points the attending surgeon asked intraoperatively were observed and recorded. During a debriefing session following each case, the video footage collected was available for review and playback. Of the 40 cases that were observed throughout the study, sufficient data to analyze the educational value of the case was only collected from 15 of the cases. The overall average learning experience reported from the attending and resident surgeons for all collected cases was 4.5 on a 1 to 5 Likert scale. In conclusion, higher-level residents were found to ask less questions and were given fewer teaching points. SimCapture technology was found to be a useful tool for resident evaluation, and should be implemented into other specialties beyond obstetrics and gynecology.

Keywords

SimCapture, video capture, obstetrics and gynecology, OPRS, surgery, technical skill, evaluation, assessment

Introduction

Since the Accreditation Council for Graduate Medical Education (ACGME) created six general competencies to be used to evaluate medical education, the complicated system of assessing the proficiency of residents in the operating room has been based heavily on these competencies in many specialties. Patient care, medical knowledge, and interpersonal skills are among the competencies identified by the ACGME.3 Technical skill, however, is critical in the operating room and must also be evaluated. It is important for surgical residents to be objectively assessed concerning their technical skills during surgery. There are many ways to do so, ranging from direct observation to animal models to virtual simulation of procedures.1 Technical skill can be measured using standard validated scoring tools such as the Operative Performance Rating System (OPRS) and milestone evaluations.

Financial pressure to increase productivity can limit the volume of surgeries residents are exposed to during their training period,1 therefore each teaching opportunity in the operating room is highly valuable. Residents must become proficient quickly. Video recording of residents performing surgical procedures is a method that may be utilized to assess technical skills without requiring extra staff members to be present in the operating room to watch the live procedure. The videos could be sent to various blinded faculty members for evaluation, which would increase the objectivity of the assessment.1

SimCapture is a portable, multi-camera system complete with video software that has been purchased by Lehigh Valley Health Network (LVHN). Implementation of this system is intended to enhance the education and feedback surgical residents receive after operative cases.2 The system is capable of connecting to the existing endoscopic towers in the operating rooms to record laparoscopic cases. Specific events that occur during the cases can be referenced during debriefing sessions to show residents areas of strength or weakness in their technical skills. The present study aims to assess whether video-capture technology, specifically the SimCapture system, is an effective evaluative tool for resident surgeons. Performance of the resident is to be evaluated by a proctoring surgeon as well as the resident themselves, using several evaluative tools. The patient as well as the operating surgeons will not be identifiable, as only the video from the endoscopic tower and the hands of the resident surgeon will be videotaped.2

Crowdsourcing is collecting services from a large number of people, generally through an online database.4 This study intends to eventually utilize crowdsourcing to evaluate resident performance of technical skills, in order increase objectivity and efficiency. In a recent study, Crowd Sourced Assessment of Technical Skill (CSATS) was used to rate skills of surgical trainees. Results of the study showed that CSATS is an accurate and fast way to gather technical skills assessments for residents, as inter-rater reliability between surgeons and the crowd was found to be excellent.4

Verbal interactions among resident and attending surgeons can serve to keep the operation flow moving forward, as well as provide teaching opportunities. The operating room is one of the most critical places for the education of residents to occur, despite the limited time that is available for teaching during surgical procedures.5 Educational value can be further increased by the attending and resident surgeons having a short debriefing following the case. Discussing the procedure and revisiting teaching points that were touched on during the operation helps solidify the information and leads to better performance in the future.5

Methods

Participants comprised of different surgical residents from the obstetrics and gynecology training program. Attending surgeons in the respective programs were also involved in the study in that they participated in evaluation of the residents and debriefing sessions. All participants are employed by Lehigh Valley Health Network. Participants were selected based on operative schedules during weekdays; only resident surgeons performing five specified sentinel case types were recorded and evaluated. A SimCapture team made up of two summer research scholars oversaw the setup and recording of the selected procedures in the operating room.

The SimCapture system, provided by B-Line Medical, was utilized for this project. OPRS and milestone evaluation forms were used by residents and attending physicians to rate the performance of the residents during the surgical procedures. An additional form was created for use by the SimCapture team to keep a record of the number of questions asked by the resident and teaching points given by the attending surgeon during each case.

Four case types were evaluated for this study, including cesarean sections, operative hysteroscopy, laparoscopic bilateral tubal ligation, and robotic hysterectomy. A SimCapture team set up the system and cameras in the operating room immediately before or after the patient was brought into the operating room. Parts of the surgery that were performed by the resident surgeon were annotated using the SimCapture software for easy playback. During the procedure, a member of the SimCapture team counted the number of questions the resident surgeon asked and the number of teaching points that were offered by the attending surgeon throughout the procedure. After each case, both the resident surgeon and attending surgeon completed two evaluation forms, the OPRS (specific to the particular procedure) and milestone evaluations. A member of the SimCapture team then mediated a brief, five- to ten-minute discussion between resident and attending regarding the resident’s performance. This conversation allowed a comparison of the resident and attending surgeons’ perceptions of how the case went. Lastly, a short survey was sent electronically to both the resident and attending surgeon about their perception of the educational value of the procedure.

Results

After the collection of the needed data, the SimCapture team collected 40 cases that had usable material to be sent out to third party experts. Most of the results were coming from cesarean sections because they are simply the most common procedures that the SimCapture team was recording (almost always 3 a day are performed). We did run into some problems in the first couple weeks because of issues with the actual technology, and trial and error occurrences. Problems also occurred when cases were cancelled, pushed back, or the resident was simply not going to be performing any of the surgery. Some of the cases that were recorded could not be used due to HIPPA violations. Therefore, the SimCapture team has less data than was expected, but still enough for outside party members to create a fair assessment. Almost always, the attending surgeon, resident, and staff were receptive to the project overall. The SimCapture team was able to find that residents who are at a higher level generally ask less questions and are also given less teaching points, with the exception of some outliers due to the variety in attending surgeons and residents. Most residents who are their second year asked a lot of questions during the procedure as opposed to a fourth year resident. The debriefing sessions varied in length and quality; some were quick and precise and others were lengthy with a lot of description. Some debriefing sessions did not occur at all because of time or schedule conflicts The results from the outside party members is still to be collected although they have been sent out. The SimCapture team will then compare the results from a third party to the results of the attending evaluation of the same case.

Discussion

SimCapture technology provides an overall beneficial educational value of the residents’ crucial learning period. With the ability to watch the video at their leisure, and have a formal debriefing session with an attending, residents can become efficiently more competent. After analyzing data, we were able to find that both residents and attending physicians find the overall educational value to be very high, averaging a 4.2 on a 1 to 5 scale. The level of a resident was found to impact the number of teaching points as well as the amount of questions asked in the operating room. Many upper level residents asked fewer questions and received fewer teaching points whereas lower level resident tended to do the opposite. In these cases, the overall educational value was still ranked high, even if there were fewer questions asked and fewer teaching points. It was also found that there was no difference in the overall OPRS evaluation and the overall educational value, meaning that the OPRS is an evaluation tool of quality in which attending surgeons can formally recognize how well a resident is performing. Also, the OPRS provides a means for the resident and attending to sit down and have an open discussion about his or her performance, therefore increasing the education value. Because of this, SimCapture technology is crucial to a resident in training. They themselves can go back and watch the footage during a debriefing session and can assess their strengths and weakness. It would be beneficial to continue using SimCapture technology throughout not only the OB/GYN department but all other departments so that residents can more effectively and efficiently learn during a crucial time in their careers.

Acknowledgements

We would like to thank everyone who assisted us in our research and clinical observation. Thank you to Joe Patruno, President Elect of Medical Staff at Lehigh Valley Health Network, for his guidance and supervision. We could not have made the progress we did without his enthusiasm and help getting into the operating room. Thank you also to Martin A. Martino, gynecologic oncologist, and Timothy M. Pellini, obstetrics and gynecology, for their patience and assistance in getting this project off the ground. Thank you to our mentor and the Summer Research Scholars program director, Hubert Huang, for providing this opportunity for all of our personal and professional growth.

References

Moorthy Krishna, Munz Yaron, Sarker Sudip K,Darzi Ara. Objective assessment of technical skills in surgery BMJ 2003; 327 :1032

Patruno JE, Pellini T, Huang H, Cesanek P, Martino M. The use of video-capture technology to optimize the education, evaluate technical performance in residents.

Greenberg JA, Irani JL, Greenberg CC, et al. The ACGME competencies in the operating room. Surgery.2007 Aug;142(2):180–4.

Holst Daniel, Kowalewski Timothy M., White Lee W., Brand Timothy C., Harper Jonathan D., Sorenson Mathew D., Kirsch Sarah, and Lendvay Thomas S.. Journal of Endourology. May 2015, 29(5): 604-609. doi:10.1089/end.2014.0616.

Roberts Nicole, Brenner Michael, Williams Reed, Kim Michael, Dunnington Gary. Capturing the teachable moment: A grounded theory study of verbal teaching interactions in the operating room. Surgery. 2012 May; 151(5):643-50.

Comments

Mentor: Hubert K. Huang, PhD

Department(s)

Research Scholars (Acknowledgements and Co-authored Publications), Research Scholars - Posters

Document Type

Poster

Additional Files
Arnold Megan - Manuscript.pdf (92 kB)
Download

Share

COinS