The Feasibility of the use of Video Capture, Feedback Process in the Obstetrics and Gynecology Residents
Martin A. Martino, MD; Joseph E. Patruno, MD; Timothy M. Pellini, MD
Educating a resident and proving that he is capable of consistently performing a procedure is a difficult task. This is vastly important for the patients safety. One of the key parts of becoming a quality surgeon is technical skill. To assess the technical skills of the obstetrics and gynecology residents, myself and four other students performed a video capture, feedback process using a product called SimCapture. This process involved us video recording four different types of surgeries performed by residents. After each case, the resident and attending participated in a feedback process to review the strengths and weaknesses of the resident for that particular case. The feasibility of the process is very significant in helping determine if this method is effective and if it should be used in the future. Often times, we planned on recording a case for our data in which something went wrong such as a resident who didn’t operate at all or a change in the operating room (OR) schedule we were not made aware of. This caused us to miss quite a few cases that we planned on recording. Nurses, attendings, and residents were slightly skeptical about our study initially, but over time they became more comfortable with us recording the surgeries. The consistent issues and obstacles forced us to be very proactive and flexible. Ultimately, we were successful in collecting the research we wanted.
Becoming an expert surgeon is a long and difficult process including many years of medical school and residency. This calls for thousands of hours of learning information, practicing on simulators, and performing surgeries with an attending surgeon. It is important that the attending surgeon mentors and assists the resident(1). The education of these residents must be maximized for the purpose of patient safety. While there are many factors in determining a surgeons abilities, technical skill is the most related to the outcome of the patient (2). Evaluating the technical skill of residents is an especially challenging subject. An educational program called FLS (The Fundamentals of Laproscopic Surgery) was developed to improve the knowledge and technical skills of residents (3). A study done by three Washington D.C hospitals compared the FLS examination scores of residents to their objective OR skill evaluations (3). The results showed a clear relationship as residents with higher FLS scores also received better evaluations in the OR (3). The FLS test includes a written section to express knowledge of laproscopic surgery as well as skills tests on box trainers (3). Another way to evaluate the technical skill of surgeons is through video analysis. A study done by the Michigan Bariatric Surgery Collaborative (MBSC) had 20 surgeons submit a video of themselves performing a laproscopic gastric bypass (2). The skills of these surgeons were then evaluated by blinded surgeons and rated on a 1 to 5 scale. These scores were then compared to the outcomes of surgeries done by the same 20 surgeons on around 10,000 patients (2). The results showed that the top quartile of scores were related to lower complication rates (2). The bottom quartile were related to higher complication rates (2). The use of videotape assessment is a very effective way for residency programs to improve the technical skill of residents through feedback. It is also a way to determine whether or not their residents are competent performing certain surgical procedures. If hospitals are considering a form of video taping and feedback process, it is important that they know the feasibility of the process. How accepting were the residents and attending surgeons to the process? Were participating in the debriefing after? Was the process overcrowding the OR? Was it difficult to record the cases? A type of video recording technology called SimCapture was purchased by the Lehigh Valley Hospital for the purpose of recording residents. The technology includes a software programmed into a laptop computer that then allows you to record surgeries from a webcam. The data recorded on the webcam and through connection to the endoscopic tower is then stored on the software for further analysis.
The purpose of my project is to determine the feasibility of a video capture, feedback process that will be used to improve and calculate the technical skill of obstetrics and gynecology residents at the Lehigh Valley Hospital. The video capture, feedback process involves recording the surgeries performed by a number of different residents. The attending surgeons will provide feedback based on their real time analysis of the residents performance. The videotapes of each case will be sent out to blinded experts for analysis using the same evaluation forms that were completed by the attending surgeon and resident that were present at the case. In the future, the recorded surgeries could also be used to build a portfolio for each resident to show their competency performing certain procedures. My complete focus will be on determining the degree of how convenient a video capture, feedback process can be done.
For our data collection, we recorded the obstetrics and gynecology residents performing four different types of surgeries. These different surgeries included C section, Hysteroscopy, Robotic Hysterectomy, and Laparoscopic tubal ligation (BTL). Our exact methods for recording the surgeries varied slightly depending on the case. The residents varied in experience from first through fourth year. The fourth year residents would often do large portions of the cases while the less experienced residents did less. The night before the cases, we would look at the OR schedule and plan out the cases we were going to record the next day. A group of two scholars would show up about a half hour before the first surgery. At this time they would find the resident and attending for the case to introduce themselves, tell them about the project, and ask them to participate in a debrief session following the case. The scholars would then enter into the OR to set up the technology. The computer with the Sim Capture program is connected to the endoscopic tower using the appropriate adaptors. This allows us to get an internal view of the patient. A camera attached to an extension chord is mounted on top of an IV pole using a clamp and then angled toward the incision made on the patient. For C sections, there is no endoscopic tower used. For robotic hysterectomy cases, we record the resident’s hands when using the robot instead of the incisions. One scholar then logs into the SimCapture program and runs a new session. To run the session, they have to input information such as the resident number, name of scholar operating the system, and the type of surgery. Once everything is set up, both scholars leave the OR until the patient is all ready to be operated on. Upon re-entering the OR, the scholar working the laptop then starts recording once time out is called. Every time the resident stops or starts operating, an annotation is made in the video tape. The scholar who is not working the SimCapture program fills out the form which keeps track of each time the attending teaches the resident. That scholar also completes the feasibility form which notes if the resident, attending, and other staff were understanding of our study as well as any problems that occurred. Once the surgery is completed, recording is stopped and both scholars disassemble the equipment. After leaving the OR; both scholars, the resident, and the attending participate in a short debriefing session. In this debriefing session, the attending and resident each complete the appropriate OPRS and milestone forms. The resident then discusses what he thought he did well and also what could be improved. The attending follows up with his feedback. Once the debriefing is completed, the scholars then plug all the forms into the SimCapture program and discuss the case.
Figure 1: Number of cases recorded for each procedure
Number of cases recorded
Figure 2: Percentage of cases recorded
Percentage of Cases Recorded
Figure 4: Summary of feasibility form
Average time to set up equipment
Average time to disassemble equipment
Average time to debrief
Percent of cases where debrief occurred
Percent of cases where resident was receptive to process
Percent of cases where attending was receptive to process
Percent of cases where OR staff was receptive to process
Conclusion / discussion
After about six weeks of collecting data, we were able to record 39 cases as shown in figure 1. You can see in figure 2 that we did miss quite a few of the cases that we had planned on recording. Our 61% success rate had to do with the number of issues and obstacles that occurred throughout the process. Some of the obstacles included technology issues, resident not operating, changes in the OR schedule, surgeon preference, and patient preference. Figure 3 allows you to see the breakdown of how often certain issues occurred in the 25 cases we missed. As we became more comfortable with our video capture feedback process, the issues became less and less frequent. Some of the initial technology issues included delays in the camera feed, one camera not working at all, SimCaptu re not picking up the camera and tower feeds, and not having the correct adaptor to connect the program into the tower. We sorted out almost all of these issues in the first few weeks. Experience also allowed us to perform the process more efficiently, including setting up the equipment. Over time, we had developed a consistent system to record cases and stay out of the way of the staff. This allowed our process to gain acceptance by attendings, residents, and OR staff. After performing a number of cases, they became more comfortable with us recording the cases in the OR. Figure 4 shows a summary of the results for the feasibility forms we completed for each case. One of the other key difficulties of our process was getting the resident and attendings to participate in the debriefing after the case. This is shown by the fact that only 61% of the cases had debriefings. The attendings, residents, or both of them were often needed elsewhere and had little time after the cases.Throughout the 6 weeks, we were forced to make quite and few innovations and be flexible with our process. For example, we needed to purchase an adaptor that would allow us to connect the SimCapture program into the endoscopic tower. Often times, the nurses and OR staff got very nervous about all the wires that we were bringing into the OR. We often had to be innovative by attaching the camera to the IV pole to get a good angle and laying a mat over all the wires. Even with all these issues, we were able to record a very good amount of cases over the 6 week period. This shows that a video capture, feedback process using a system such as SimCapture can be very effective in evaluating the technical skills of residents.
Levy, B. (2012). Experience Counts. American College of Obstetricians and Gynecologists, 119(4), 693-694.
- Birkmeyer, J. (2013, October 10). Surgical Skill and Complication Rates after Bariatric Surgery. www.nejm.org.
- 3. Antosh, D. (2012, December 8). Blinded Assessment of Operative Performance After Fundamentals of Laparoscopic Surgery in Gynecology Training. www.jmig.org.
Published In/Presented At
Adams, Sean; Martino, Martin A., MD; Patruno, Joseph E., MD; Pellini, Timothy M., MD (2015, July 31). The Feasibility of the use of Video Capture, Feedback Process in the Obstetrics and Gynecology Residents. Poster presented at LVHN Research Scholar Program Poster Session, Lehigh Valley Health Network, Allentown, PA.
Research Scholars (Acknowledgements and Co-authored Publications), Research Scholars - Posters