close

Вход

Забыли?

вход по аккаунту

?

j.amjsurg.2017.09.010

код для вставкиСкачать
The American Journal of Surgery xxx (2017) 1e7
Contents lists available at ScienceDirect
The American Journal of Surgery
journal homepage: www.americanjournalofsurgery.com
Preparing for the American Board of Surgery Flexible Endoscopy
Curriculum: Development of multi-institutional proficiency-based
training standards and pilot testing of a simulation-based mastery
learning curriculum for the Endoscopy Training System
Brenton R. Franklin a, *, Sarah B. Placek a, Aimee K. Gardner b, James R. Korndorffer Jr. c,
Mercy D. Wagner a, Jonathan P. Pearl d, E. Matthew Ritter a
a
The Department of Surgery at the Uniformed Services University and the Walter Reed National Military Medical Center, Bethesda, MD, USA
Baylor College of Medicine, 1 Baylor Plaza, Houston, TX 77030, USA
c
Department of Surgery, Tulane University School of Medicine, 1430 Tulane Ave, New Orleans, LA 70112, USA
d
Department of Surgery, University of Maryland, 22 S. Greene Street, Baltimore, MD 21201, USA
b
a r t i c l e i n f o
a b s t r a c t
Article history:
Received 20 July 2017
Received in revised form
9 September 2017
Accepted 16 September 2017
Background: The Fundamentals of Endoscopic Surgery (FES) exam is required for American Board of
Surgery certification. The purpose of this study was to develop performance standards for a simulationbased mastery learning (SBML) curriculum for the FES performance exam using the Endoscopy Training
System (ETS).
Methods: Experienced endoscopists from multiple institutions and specialties performed each ETS task
(scope manipulation (SM), tool targeting (TT), retroflexion (RF), loop management (LM), and mucosal
inspection (MI)) with scores used to develop performance standards for a SBML training curriculum.
Trainees completed the curriculum to determine feasibility, and effect on FES performance.
Results: Task specific training standards were determined (SM-121sec, TT-243sec, RF-159sec, LM-261sec,
MI-180-480sec, 7 polyps). Trainees required 29.5 ± 3.7 training trials over 2.75 ± 0.5 training sessions to
complete the SBML curriculum. Despite high baseline FES performance, scores improved (pre 73.4 ± 7,
post 78.1 ± 5.2; effect size ¼ 0.76, p > 0.1), but this was not statistically discernable.
Conclusions: This SBML curriculum was feasible and improved FES scores in a group of high performers.
This curriculum should be applied to novice endoscopists to determine effectiveness for FES exam
preparation.
Published by Elsevier Inc.
Keywords:
Endoscopy
Simulation
Mastery learning
Endoscopic surgery
Standard setting
1. Introduction
Endoscopy is a large component of many general and colorectal
surgeons' practices, and the need for proficiency continues to increase in more rural general surgery practices.1,2 Standardized
training and assessment for endoscopy is in its early stages, and
only recently was the Fundamentals of Endoscopic Surgery (FES)
exam developed as a way to identify individuals with a level of
competency required to safely perform basic endoscopy.3e5 The
need to ensure that surgeons are proficient in basic endoscopic
skills prior to entering independent practice has been recognized
by the American Board of Surgery, who will now require FES certification for board eligibility starting with the graduating residents
of 2018 as part of the Flexible Endoscopy Curriculum.6
Abbreviations: FES, Fundamentals of Endoscopic Surgery; FLS, Fundamentals of Laparoscopic Surgery; SAGES, Society of American Gastrointestinal and Endoscopic
Surgeons; USU, Uniformed Services University of the Health Sciences; SBML, Simulation Based Mastery Learning; VR, Virtual Reality; ETS, Endoscopy Training System; SD,
Standard Deviation; MIS, Foregut Minimally Invasive Surgery; CR, Colorectal Surgery; GI, Gastroenterology; GAGES, Gastrointestinal Global Assessment of Gastrointestinal
Endoscopic Skills.
* Corresponding author. 8901 Rockville Pike, Bethesda, MD 20889, USA.
E-mail address: brentonfranklin@gmail.com (B.R. Franklin).
https://doi.org/10.1016/j.amjsurg.2017.09.010
0002-9610/Published by Elsevier Inc.
Please cite this article in press as: Franklin BR, et al., Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development
of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the
Endoscopy Training System, The American Journal of Surgery (2017), https://doi.org/10.1016/j.amjsurg.2017.09.010
2
B.R. Franklin et al. / The American Journal of Surgery xxx (2017) 1e7
Similar to the Fundamentals of Laparoscopic Surgery (FLS), the
FES program was developed by members of the Society of American
Gastrointestinal and Endoscopic Surgeons (SAGES), and is
comprised of three partsdan online curriculum, a knowledge
exam, and a technical skills exam. Expert endoscopists developed
the specific tasks of the technical skills portion of the FES exam to
represent the core skills required for gastrointestinal endoscopy.
For consistency, predictability and objectivity of scoring, the FES
skills test uses a virtual reality (VR) platform that has been shown to
have considerable validity evidence.5
While the online curriculum prepares trainees for the knowledge portion of the exam, to date no optimal curriculum exists to
prepare individuals to pass the technical skills exam. Recent studies
have also shown that relying on completion of the endoscopy case
requirements for a general surgery residency results in a 25% first
time fail rate on the FES skills exam.7 Similarly, a group of residency
graduates who pursued fellowship in minimally invasive surgery
demonstrated a skills exam failure rate of 40%.8 An ideal endoscopic
training curriculum would not only promote first-time pass rates
for the FES skills test, but would also translate into improved performance in the clinical environment. There are few VR modules to
train for the FES exam, and the cost of the available modules, like
the VR system itself, can be prohibitive9; therefore, practicing for
the FES skills exam on a VR system is often not possible or feasible.
As a whole, VR simulators also have other training drawbacks
including lack of haptic feedback, technical failure, high maintenance cost and suboptimal durability. An affordable and easy to use
physical simulator can overcome many of these drawbacks with
improved haptic feedback, lower cost, and less technical failures.
Previous work has developed a Simulated Colonoscopy
Objective Performance Evaluation which is a physical endoscopy
training platform based on the Kyoto Kagaku colonoscopy model.10
With the development of a task for retroflexion, the platform was
renamed as the Endoscopy Training System (ETS) (Limbs and
Things and Kyoto Kagaku). The ETS platform consists of two separate models encompassing five tasks: scope manipulation, tool
targeting, retroflexion, loop management and mucosal inspection
(Figs. 1 and 2). Previous work has shown validity evidence for four
of these tasks, with a new retroflexed tool-targeting task added to
wholly represent the domain of flexible gastrointestinal endoscopic
skills.11
In addition to the platform itself, an ideal curriculum for trainees
must take into account the varying levels of endoscopic experience
and skill. Employing mastery learning principles using an expert
performance based standard is appealing because the total time
required for training is dependent on the underlying skill and
performance of the trainee.12e15 This strategy has been used for
many different simulation based training curricula, including FLS,
with extraordinary results.16e21
The objective of this study was to conduct multi-institutional
standard setting using experienced endoscopists to develop performance standards for a mastery learning, proficiency-based
endoscopic training curriculum using the ETS training platform,
and to pilot test the standards for feasibility on a small group of
surgical trainees.
2. Methods
2.1. ETS training platform
The ETS has been developed over the last 4 years through a
Collaborative Research and Development Agreement between the
Henry Jackson Foundation for the Advancement of Military Medicine, the Department of Surgery at the Uniformed Services University of the Health Sciences (USU), Limbs & Things Inc. (Bristol,
UK), and Kyoto Kagaku Co, LTD (Kyoto, Japan).
The ETS contains 5 training tasks housed in two tabletop units
(Figs. 1 and 2). All tasks are performed using a standard endoscope
and tower unit, which is not provided with the system. The first
unit is linear and contains the Scope Manipulation (1), Tool Targeting (2) and Retroflexion (3) tasks. This unit is equipped with a
simplistic endoscopic tool and uses basic circuitry to allow for both
auditory and visual feedback for tasks 2 & 3. Task 1 requires a
transparent screen overlay that is provided as part of the ETS. The
second tabletop unit is a stylized body form, and is a modified
version of Kyoto's previous colonoscopy trainer. The Loop Management (4) and Mucosal Inspection (5) tasks are performed in the
second unit. Tasks 4 and 5 require setup by a trainer to ensure
proper lubrication and orientation of the rubber colon. Instructions
for this setup are standardized and require less than 5 min to
complete. Tasks 4 and 5 also require use of suction, insufflation, and
the lens cleaning function of the endoscope, thus requiring a fully
functional endoscopy setup. Brief descriptions of the 5 ETS tasks are
shown here:
2.2. Task 1: scope manipulation
Fig. 1. Endoscopy training system platform; straight and body model.
The purpose of this task is to perform basic endoscope navigation using tip deflection and torque of the scope. The task is to align
the white numbered triangle presented in the colonic lumen within
the two black triangle outlines on the overlay that is placed onto
the display monitor. The edges of the white triangle must be
positioned upright and completely within the two black triangles of
the overlay. Alignment must be held long enough to freeze and
unfreeze the image on the screen. There are a total of 10 shapes and
Please cite this article in press as: Franklin BR, et al., Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development
of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the
Endoscopy Training System, The American Journal of Surgery (2017), https://doi.org/10.1016/j.amjsurg.2017.09.010
B.R. Franklin et al. / The American Journal of Surgery xxx (2017) 1e7
3
Fig. 2. Endoscopy training system tasks. Scope manipulation (top left), tool targeting (top right), retroflexion (bottom left), mucosal inspection (bottom right); loop reduction task
not depicted.
they must be completed in numerical order. Timing begins when
the first shape is seen and ends when the last shape is completed.
2.3. Task 2: tool targeting
The purpose of this task is to perform hand-eye coordination
with a simulated biopsy tool and the endoscope. The task is to
touch the tool to the metal disc within the colonic lumen. The
subject will need to manipulate both the endoscope and the tool
without assistance. Successful contact with the metal disc is
signified by the sound of a tone. If contact is lost, a second distinct
tone alerts the subject to make a correction. A double beep is
audible when contact is held for 5 continuous seconds, and a corresponding indicator light illuminates on the control panel, signifying completion of the target. The metal disc must be kept in view
on the screen for the entire length of the tone, and the targeting
tool must be withdrawn out of the view after each completed disc.
There are a total of 10 discs that must be completed in reverse
order. Timing begins when the scope is withdrawn from the white
cap at the end of the straight model and stops after the last target
indicator illuminates.
2.4. Task 3: retroflexion
The purpose of this task is to perform hand-eye coordination
with the same simulated biopsy tool and the endoscope in the
retroflexed position. During this task, the scope is retroflexed in the
bulbous vestibule of the straight colon model and the tool is
advanced 1.5e2.5 cm into view, with appropriate length indicated
by a pre-marked line on the tool. Unlike tool targeting, the biopsy
tool may NOT be adjusted during the entire task. Successful
completion of each target is equivalent to what is described for Task
2 above. For the purposes of standardization, only the white shaped
targets (5) are completed in any order. Timing begins when scope is
withdrawn into the vestibule and ends when the last target indicator illuminates.
2.5. Task 4: loop management
The purpose of this task is to perform scope navigation during a
simulated colonoscopy, and correctly manage the formation of a
standard alpha loop. The task begins with a standardized loop
which will require reduction in order to continue advancing the
endoscope. The task is performed on the body model in the supine
position. Abdominal pressure can be provided by a proctor where
and when directed by the subject, and insufflation, irrigation, and
suction can be used. To facilitate recognition of proper reduction
timing and technique, a “reduction zone” is marked in the transverse colon to prevent novices from attempting reduction too early.
Timing begins with scope insertion and ends when the stop marker
in the simulated cecum is contacted by the tip of the scope.
2.6. Task 5: mucosal inspection
The purpose of this task is to perform complete mucosal inspection after completion of Task 4. Simulated polyps must be
identified within the colonic mucosa during withdrawal of the
scope using standard endoscopic maneuvers, including retroflexion
of the endoscope in the rectum to visualize the anorectal junction.
The endoscope may be advanced at any time to re-visualize a
segment, and insufflation, irrigation and suction can be used. Newly
identified polyps are counted, and there are between 5 and 20
simulated polyps present in the colonic segment. Timing begins
when the scope is withdrawn from the white cap in the cecum and
Please cite this article in press as: Franklin BR, et al., Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development
of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the
Endoscopy Training System, The American Journal of Surgery (2017), https://doi.org/10.1016/j.amjsurg.2017.09.010
4
B.R. Franklin et al. / The American Journal of Surgery xxx (2017) 1e7
ends when the scope exits the simulated anus.
2.7. Task scoring
Scores are based on performance efficiency which incorporates
time and the number of potential errors. Start and end times for
each task are outlined above. All tasks incorporate visual and/or
auditory feedback which designates accuracy during task performance. The total time elapsed after completion of the task correlates with score, with a shorter time signifying a superior level of
performance.
2.8. Proficiency level development
Experienced subjects were recruited from four different institutions (USU/Walter Reed, Tulane, UT Southwestern, and the
University of Maryland) and included individuals who regularly
perform endoscopy in their practice. All of our cohort were
fellowship trained and included four minimally invasive foregut
surgeons/surgical endoscopists (MIS), four colorectal surgeons (CR)
and three gastroenterologists (GI). The endoscopic skill of each
subject was assessed with either a clinical Global Assessment of
Gastrointestinal Endoscopic Skills (GAGES) score for upper or lower
endoscopy or FES skills exam score (Table 1).22 Additionally, the
majority of subjects had experience with virtual and physical
endoscopic simulation.
The majority of subjects were unfamiliar with the ETS tasks.
Prior to each task, verbal instruction with corresponding pictures of
each task were given. Subjects then completed three repetitions of
each task proctored by one of the investigators. All tasks were
completed in one or two sessions. An adult colonoscope (Olympus
America, Center Valley, PA) was used for all tasks using an Olympus
imaging system on a standard-definition imaging monitor. Subjects
used standard endoscopic capabilities including insufflation, irrigation, and suction as dictated by task instructions. The overall
mean score and standard deviation (SD) was determined for the
third repetition of each task. Any score greater than 2SDs from the
mean would be excluded and a new mean and SD calculated and
used for proficiency level determination. Descriptive statistics for
all scores were analyzed for trends and outliers, and were adjusted
on a task-by-task basis.
2.9. Feasibility testing
Once training standards were established for each of the 5 ETS
tasks, the simulation-based mastery learning (SBML) standards
were pilot tested on a convenience sample of four surgical trainees.
Demographic information collected included gender, post-graduate
year, number of upper and lower endoscopies performed, hand
dominance, glove size, and the type and amount of previous flexible endoscopy simulation experience. All pilot subjects completed
Table 1
Experienced endoscopist demographics, experience and proficiency scores;
MIS ¼ minimally invasive foregut surgery, FES ¼ Fundamentals of Laparoscopic
Surgery, GAGES ¼ Global Assessment of Gastrointestinal Endoscopic Skills.
Table 2
Experienced endoscopist standard setting data, adjusted means presented in seconds as average of 3rd trial; ETS ¼ Endoscopy Training system, SD ¼ standard
deviation.
ETS Task
Mean Time (SD)
Completion Standard
1- Scope manipulation
2- Tool targeting
3- Retroflexion
4- Loop management
5- Mucosal inspection (time)
Mucosal inspection (polyps)
121 (31)
197 (46)
138 (21)
158 (103)
313 (176)
8 (1)
10/10 targets
10/10 targets
5/5 white targets
Loop Reduced
a pre-training assessment consisting of all 5 FES tasks on the GI
Mentor II. Subjects then trained to the previously determined
performance benchmarks on each of 5 ETS tasks. Tasks could be
trained in any order, and subjects were provided instruction on
how to correctly perform each ETS task with correct performance
verified prior to being allowed to practice the tasks independently.
Subjects were then instructed to practice for no more than 90 minutes per session with no more than 2 sessions per calendar day to
ensure training was consistent with the standards of distributed
and deliberate practice. Once training benchmarks were reached
for all tasks, an identical post training assessment was performed
with all 5 FES tasks. FES task and total scores were calculated and
resulted using a proprietary algorithm and were supplied to the
investigators directly from the SAGES FES Program.
3. Results
The experienced endoscopists had practiced for 11.2 (±8) years
after completion of graduate medical education training, and performed 328 (±306) endoscopies in the past year (Table 1). Six had
taken and passed the FES skills test (mean 83.5 ± 11). Eight were
evaluated by GAGES (mean 19.6 ± 1.1 on 20 pt scale). ETS performance parameters for each task are listed in Table 2. Only the third
trial for each task is presented as there was marked improvement
after the first trial for all tasks with stable scores on the remaining 2
trials. The presented mean and SD for task 3 excludes performance
of experienced CR endoscopists, and for task 4 excludes the performance of experienced MIS endoscopists. These groups were
outliers compared to others on these tasks and the decision to
exclude them is further discussed below.
One experienced endoscopists score for task 4 was excluded, as
it was greater than 2 SDs above the adjusted mean. The final trial
for task 4 was not completed by one subject and was excluded from
all calculations. Once available, these data were used by the investigators to develop time- and error-based mastery learning
standards for each of the 5 tasks (Table 3). Decisions to adjust the
performance standard were made based on the consistency of the
experienced endoscopists performance on the task, as well as the
Table 3
Experienced Endoscopist-derived simulation based mastery learning training
standards; ETS ¼ Endoscopy training system.
ETS Task
Time (sec)
Performance
requirements
Repetitions &
overtraining
(consec/þ non consec)
12345-
121
243
159
261
180e480
10/10 targets
10/10 targets
5/5 targets
Loop reduced
>6 polyps seen
(2/þ5)a
(2/þ5)a
(2/þ5)a
(2)
(2)
Experienced Endoscopists (n ¼ 11)
MIS/Surgical endoscopists
Colorectal surgeons
Gastroenterologists
Years in practice
Endoscopies in last 12 months
FES score
GAGES score (maximum 20)
4
3
4
11 (8)
328 (306)
83.5 (11)
19.6 (1.1)
>6 polyps identified
Scope Manipulation
Tool Targeting
Retroflexion
Loop Management
Mucosal Inspection
a
Overtraining required on tasks showing high correlations between ETS and FES
scores (data not shown).
Please cite this article in press as: Franklin BR, et al., Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development
of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the
Endoscopy Training System, The American Journal of Surgery (2017), https://doi.org/10.1016/j.amjsurg.2017.09.010
B.R. Franklin et al. / The American Journal of Surgery xxx (2017) 1e7
Table 4
Pilot testing of simulation based mastery learning standards, presented as mean
(standard deviation); ETS ¼ Endoscopy training system.
ETS Task
Pre-test Score
Trials
Post-test Score*
12345-
74.3
96.0
76.2
47.0
73.5
9.8 (2.2)
7.3 (0.5)
8.3 (1.3)
2.3 (0.5)
2 (0)
85.5 (10.2)
103.9 (2.1)
80.5 (8.2)
42.3 (19.7)
77.9 (6.4)
29.5 (3.7)
78.1 (5.3)
Scope Manipulation
Tool Targeting
Retroflexion
Loop Management
Mucosal Inspection
Total
(19.7)
(4.5)
(24.9)
(21.1)
(10.4)
73.4 (7.1)
*p > 0.05 for all scoring differences.
practicality and difficulty of performing the task repeatedly. To
improve reliability, each task required at least 2 consecutive performances at the designated standard to advance to the next task.
ETS tasks 1e3 (scope manipulation, tool targeting and retroflexion)
added an overtraining component of 5 additional non-consecutive
performances, given the consistency of the experienced endoscopists' performance on these tasks and strong correlations with
the corresponding FES skills (data not shown).
Pilot trainees (4) were similar with respect to post-graduate
year, number of upper and lower endoscopies performed, handedness, glove size, and simulation experience. They required
29.5 ± 3.7 training trials distributed over 2.75 ± 0.5 training sessions to achieve the set standards on all 5 ETS tasks. Despite earning
high/passing FES scores at baseline (73.4 ± 7), scores still improved
post-training with a strong effect size (78.1 ± 5.2; effect size ¼ 0.76,
p > 0.1); however, this improvement was not statistically discernable given the small sample size (Table 4). Similarly, there was a
non-statistically discernable improvement on each of the FES tasks
with the exception of FES task 2 (ETS task 4) e loop management
(Table 4).
4. Discussion
Simulation-based mastery learning is an innovative instructional approach that harnesses the power of measurement to drive
learning. Unlike traditional instructional methods where the
amount of training is predetermined and quantified by time or
numbers of repetitions, SBML allows performance to be directly
measured and used as the sole indicator of when training is complete. SBML curricula have resulted in improved performance
translating to the clinical arena in performance of laparoscopy,
endoscopy, arthroscopy, and critical care among others.16e21,23,24
Correct implementation of SBML curricula requires special considerations when it comes to standard setting. Unlike traditional
standard setting which often aims for a goal of “minimally
competent” or ”borderline” performance, SBML standard setting
seeks to raise the bar of performance above that level. One recommended method for SBML standard setting is known as the
proficient group approach.15 This approach first requires the proficient group to be defined, ideally with an objective measurement
of performance. Measures of experience alone such as years of
practice do not consistently predict technical excellence.25,26 In our
study, experienced endoscopists were included from different institutions and specialties in order to represent multiple clinical
domains performing gastrointestinal endoscopy. Each endoscopist
in the cohort was assessed with either a clinical GAGES or FES skills
exam score as an objective measure of their endoscopic skill. Like
FES, GAGES scoring has shown validity evidence for the assessment
of flexible endoscopic skills in the clinical setting.22 Our entire
cohort who was assessed with GAGES scored well within one
standard deviation of what would be expected for an experienced
endoscopist. Those assessed with the FES skills exam all passed,
5
with the mean score of the group falling 2.2 standard deviations
above the passing score. Additionally, all means and standard deviations were based on the third trial for each task, which allowed
the endoscopist to familiarize themselves with the mechanics of
the task and the platform and reach a steady state performance.
The number of upper versus lower endoscopies performed in
the previous year was considerably different between specialties
(Table 1). No endoscopist in the CR group performed more than 5
upper endoscopies and only one in the MIS group performed lower
endoscopies in the previous year. The GI endoscopists were the
only group who consistently performed both upper and lower endoscopies. For these reasons, all endoscopist data were used when
determining task standards for skills that are fundamental for both
upper and lower endoscopy (scope manipulation, tool targeting,
mucosal inspection). The CR group was excluded for skills more
specific to upper endoscopy (retroflexion), and the MIS group was
excluded for skills more specific to lower endoscopy (loop
management).
The training standards for each task were determined by performance time and task requirements (Table 3). The ETS platform
has built in auditory and visual feedback that facilitates quantification of these additional metrics. This helps ensure that posttraining performance is efficient and not just a function of sacrificed quality for speed.
Each task requires a minimum of 2 consecutive trials that meet
the standard for the skill to be considered completed. This
requirement is designed to increase the reliability of the standard
by reducing the chance that the standard could be met randomly.
While this requirement is somewhat arbitrary, it is commonly used
in mastery training curricula, including FLS, with good results.27e34
Overtraining with five additional non-consecutive trials at or
exceeding the standard is required for the three most consistent
skills (scope manipulation, tool targeting, and retroflexion). Overtraining to automaticity has been shown to improve performance
above SBML alone.35 While this 5 trial overtraining does not
represent automaticity, we feel it is a practical compromise to
improve reliability while maintaining practical feasibility.
Scope manipulation is arguably the most important endoscopic
skill, and all subsequent tasks rely on having a high level of proficiency in scope manipulation. The mean experienced endoscopist
time for the scope manipulation task was 121 s and the mean was
chosen as the training standard. While it may be difficult for novices to achieve this high proficiency level initially, the extra repetitive practice on this task will theoretically build a strong
foundation for the remaining tasks. This is similar to the argument
that was made for the peg transfer tasks in the FLS curriculum.27 In
the FLS proficiency-based curriculum, peg transfer is the first and
most core fundamental skill and the standard was set at the expert
mean forcing learners to strive to a high proficiency level for this
core skill. Pilot data showed that this task required the greatest
number of trials to reach proficiency (mean 9.8), but was met with
the marked increases in task score on post-testing (Table 4). It also
showed that these trainees can attain proficiency within a single
training session which is in contrast to the length of time required
to attain proficiency for the peg transfer task in FLS training.
Tool targeting is also a skill fundamental to upper and lower
endoscopy so all experienced endoscopists' scores were included in
the mean for this skill. The mean time to complete this task was
197 s (±46 s), and the standard was set at the mean plus one
standard deviation, which resulted in a training standard of 243 s
with completion of all ten targets. Despite outstanding pre-test
scores on this task, all trainees still improved on post-testing
(Table 4).
Retroflexion with biopsy is a skill that is likely more commonly
performed by upper endoscopists during thorough examination of
Please cite this article in press as: Franklin BR, et al., Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development
of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the
Endoscopy Training System, The American Journal of Surgery (2017), https://doi.org/10.1016/j.amjsurg.2017.09.010
6
B.R. Franklin et al. / The American Journal of Surgery xxx (2017) 1e7
the cardia of the stomach and gastroesophageal junction. There was
a statistically discernable difference in retroflexion task performance between the CR group and the rest of the cohort that
regularly performs upper endoscopy as part of their practice. Since
our goal is to develop the skill for a general gastrointestinal endoscopist, we chose to use the higher skilled cohort to set the standard for this task. For this reason, the CR subjects were excluded
resulting in an adjusted mean of 138 s (±21 s), with the standard set
at the mean plus one standard deviation (159 s with all five white
targets completed). Changes in scores were variable on beta testing,
likely due to high pre-test scores in three of the trainees; however,
there was still an overall increase in mean score (Table 4).
There was a statistically discernable scoring difference for loop
management between those that regularly perform lower endoscopy (CR, GI) and those that do not (MIS). Despite both disciplines
using loop reduction techniques, loop reduction on both FES and on
the ETS model is based heavily on a lower endoscopy model.
Additionally, we saw marked differences in performance on the
loop reduction task between those who do colonoscopy (CR, GI)
and those who primarily perform upper endoscopy (MIS). To avoid
weakening performance post training, the MIS group was excluded
from this task, which resulted in an adjusted mean of 158 s (±103 s).
The standard was set at 261 s, which is the adjusted mean plus one
standard deviation. This task demonstrated the most variability in
performance, and for that reason only 2 consecutive tasks are
required to reach proficiency. Interestingly, this task also showed a
decrease in mean FES task score on post-test. Our trainees reached
proficiency after a mean of only 2.25 trials. Thus the post training
performance likely represents lack of training benefit for the relatively experienced group of pilot subjects or could also represent
that the standard is set too low. Future performance effect on
novice subjects will help answer this question. Additionally, performance variability could be due to differences in the VR testing
platform compared with the ETS physical simulator.
The total number of polyps in the simulated colon is constant
(10), and the instructions give a wide range to prevent trainees
from knowing if they “found them all.” The mean time for
completion of the mucosal inspection task was 313 s (±176 s), with
a mean of 8 polyps (±1 polyp) identified. All experienced endoscopists were included and the standard was set at 180e480 s with
identification of 7 polyps, which corresponds to approximately plus
one standard deviation for time and minus one standard deviation
for polyp identification. The original intent was to target a six
minute mucosal inspection to train the muscle memory of a real
colonoscopy; however, the simulated colon in this model is shorter
than a real colon, and in beta testing this was too much allotted
time for most learners who would then just pause in the rectum
prior to finishing to meet the time requirement. This task, like loop
reduction, demonstrated a great deal of performance variability
with a large standard deviation so only 2 consecutive trials at or
below the standard is required. Only an average of 2 trials were
required by all of our trainees to achieve the training standard, but
even this short training resulted in improved post-training scores
(Table 4).
The pilot trainees were chosen from a convenience sample and
represented a group of intermediately trained endoscopists.
Despite each trainee attaining a passing FES score prior to training,
all individuals showed marked improvement from baseline total
and task specific FES scores (Table 4). A mean of 2.75 training
sessions which equates to approximately 3e4 h of deliberate
practice were required to complete the curriculum. We anticipate
that novice trainees will require more time to attain the proposed
standards, but this timeframe represents a reasonable amount of
time to reach the set goals.
Lastly, the cost of simulation training can be prohibitive for
programs due to the financial burden associated with many simulators. Virtual reality platforms are the most expensive and typically cost in excess of $60,000.36 The estimated cost of the ETS
platform is approximately $8500 with the only consumable cost
being lubrication; however, during this study we did not exhaust
the supply sent by the manufacturer. Similar to majority of
commercially available physical simulators, a functional endoscopy
complement is required to use the ETS platform.36 Our group used
repurposed equipment from our institution that was no longer in
clinical use and incurred no extra cost. If this equipment is not
available, a full endoscopy set (training grade endoscopy towers,
including endoscope, video processor, light source, and monitor)
can be found from various internet sources with costs ranging from
$8500-$18,000 (eBay), making the maximum total ETS startup cost
approximately $26,500.
5. Conclusions
The training standards for this SBML curriculum were developed
using a multi-specialty/institutional standard setting approach and
resulted in attainable standards, and improved FES scores even in a
group of high performers. These standards should be applied to
novice endoscopic trainees to determine its effectiveness in
developing fundamental endoscopic skill and preparing trainees
for the FES exam.
Funding
This work was supported by the Society of American Gastrointestinal and Endoscopic Surgeons [grant number 64089] and the
Henry Jackson Foundation Cooperative Research and Development
Agreement [project number gs2].
Conflicts of interest
Dr. Ritter receives research support from the Henry M. Jackson
Foundation for the Advancement of Military Medicine. Dr. Gardner
has ownership interest in SurgWise Consulting, LLC. Dr. Korndorffer
receives honoraria from Becton Dickinson Medical. Dr.'s Wolf,
Taylor, Franklin, and Placek have nothing to disclose. The views
expressed are those of the author and do not reflect the opinions of
the United States Army, Department of Defense, or U.S.
Government.
References
1. Zuckerman R, Doty B, Bark K, Heneghan S. Rural versus non-rural differences in
surgeon performed endoscopy: results of a national survey. Am Surg.
2007;73(9):903e905.
2. Heneghan SJ, Bordley 4th J, Dietz PA, et al. Comparison of urban and rural
general surgeons: motivations for practice location, practice patterns, and
education requirements. J Am Coll Surg. 2005;201(5):732e736.
3. Hazey JW, Marks JM, Mellinger JD, et al. Why fundamentals of endoscopic
surgery (FES)? Surg Endosc. 2014;28(3):701e703.
4. Poulose BK, Vassiliou MC, Dunkin BJ, et al. Fundamentals of Endoscopic Surgery
cognitive examination: development and validity evidence. Surg Endosc.
2014;28(2):631e638.
5. Vassiliou MC, Dunkin BJ, Fried GM, et al. Fundamentals of endoscopic surgery:
creation and validation of the hands-on test. Surg Endosc. 2014;28(3):704e711.
6. ABS Establishes New Requirement for Endoscopic Training and Assessment; 2014.
Available at: https://www.absurgery.org/default.jsp?news_fec. Accessed June
6, 2016.
7. Gardner A, Scott D, Willis R, et al. Is current surgery resident and GI fellow
training adequate to pass FES? Surg Endosc. 2016;31(1):352e358.
8. Gardner AK, Ujiki MB, Dunkin BJ. Passing the Fundamentals of Endoscopic
Surgery (FES) Exam: Linking specialty choice and attitudes about endoscopic
surgery to success [abstract]. In: the Annual Meeting of the Society of American
Gastrointestinal and Endoscopic Surgeons; 2017 Mar 22e25. Houston TX.
9. Desilets DJ, Banerjee S, Barth BA. Endoscopic simulators. Gastrointest Endosc.
2011;73(5):861e867.
Please cite this article in press as: Franklin BR, et al., Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development
of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the
Endoscopy Training System, The American Journal of Surgery (2017), https://doi.org/10.1016/j.amjsurg.2017.09.010
B.R. Franklin et al. / The American Journal of Surgery xxx (2017) 1e7
10. Ritter EM, Cox TC, Trinca KD, Pearl JP. Simulated Colonoscopy Objective Performance Evaluation (SCOPE): a non-computer-based tool for assessment of
endoscopic skills. Surg Endosc. 2013;27(11):4073e4080.
11. Trinca KD, Cox TC, Pearl JP, Ritter EM. Validity evidence for the simulated colonoscopy objective performance evaluation scoring system. Am J Surg.
2014;207(2):218e225.
12. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert
performance in medicine and related domains. Acad Med J Assoc Am Med Coll.
2004;79(10 Suppl):S70eS81.
13. McGaghie WC, Issenberg SB, Barsuk JH, Wayne DB. A critical review of
simulation-based mastery learning with translational outcomes. Med Educ.
2014;48(4):375e385.
14. McGaghie WC, Siddal VJ, Mazmanian PE, et al. Lessons for continuing medical
education from simulation research in undergraduate and graduate medical
education: effectiveness of continuing medical education: American College of
Chest Physicians Evidence-Based Educational Guidelines. Chest. 2009;135(3
Suppl):62Se68S.
15. Yudkowsky R, Park YS, Lineberry M, Knox A, Ritter EM. Setting mastery
learning standards. Acad Med J Assoc Am Med Coll. 2015;90(11):1495e1500.
16. Scott DJ, Ritter EM, Tesfay ST, Pimentel EA, Nagji A, Fried GM. Certification pass
rate of 100% for fundamentals of laparoscopic surgery skills after proficiencybased training. Surg Endosc. 2008;22(8):1887e1893.
17. Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R. Mastery learning for
health professionals using technology-enhanced simulation: a systematic review and meta-analysis. Acad Med J Assoc Am Med Coll. 2013;88(8):1178e1186.
18. Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulationbased education with mastery learning improves residents' lumbar puncture
skills. Neurology. 2012;79(2):132e137.
19. Zendejas B, Cook DA, Bingener J, et al. Simulation-based mastery learning
improves patient outcomes in laparoscopic inguinal hernia repair: a randomized controlled trial. Ann Surg. 2011;254(3):502e509. discussion 509-11.
20. Barsuk JH, Cohen ER, Potts S, et al. Dissemination of a simulation-based
mastery learning intervention reduces central line-associated bloodstream
infections. BMJ Qual Saf. 2014;23(9):749e756.
21. Barsuk JH, Cohen ER, McGaghie WC, Wayne DB. Long-term retention of central
venous catheter insertion skills after simulation-based mastery learning. Acad
Med J Assoc Am Med Coll. 2010;85(10 Suppl):S9eS12.
22. Vassiliou MC, Kaneva PA, Poulose BK, et al. Global Assessment of Gastrointestinal Endoscopic Skills (GAGES): a valid measurement tool for technical
skills in flexible endoscopy. Surg Endosc. 2010;24(8):1834e1841.
23. Angelo RL, Ryu RK, Pedowitz RA, et al. A proficiency-based progression training
curriculum coupled with a model simulator results in the acquisition of a
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
7
superior arthroscopic bankart skill set. Arthrosc J Arthrosc Relat Surg Off Publ
Arthrosc Assoc N. Am Int Arthrosc Assoc. 2015;31(10):1854e1871.
Ahlberg G, Hultcrantz R, Jaramillo E, Lindblom A, Arvidsson D. Virtual reality
colonoscopy simulation: a compulsory practice for the future colonoscopist?
Endoscopy. 2005;37(12):1198e1204.
Ericsson KA, Whyte 4th J, Ward P. Expert performance in nursing: reviewing
research on expertise in nursing within the framework of the expertperformance approach. ANS Adv Nurs Sci. 2007 Jan-Mar;30(1):E58eE71.
Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship
between clinical experience and quality of health care. Ann Intern Med.
2005;142(4):260e273.
Ritter EM, Scott DJ. Design of a proficiency-based skills training curriculum for
the fundamentals of laparoscopic surgery. Surg Innov. 2007;14(2):107e112.
McClusky D, Gallagher A, Ritter E, et al. Virtual reality training improves junior
residents' operating room performance: results of a prospective randomized,
double-blinded study of the complete laparoscopic cholecystectomy. J Am Coll
Surg. 2004;199:S73.
Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves
operating room performance: results of a randomized, double-blinded study.
Ann Surg. 2002;236(4):458e463. discussion 463-4.
Korndorffer Jr JR, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ.
Simulator training for laparoscopic suturing using performance goals translates
to the operating room. J Am Coll Surg. 2005;201(1):23e29.
Stefanidis D, Korndorffer Jr JR, Markley S, Sierra R, Scott DJ. Proficiency maintenance: impact of ongoing simulator training on laparoscopic skill retention.
J Am Coll Surg. 2006;202(4):599e603.
Stefanidis D, Sierra R, Korndorffer Jr JR, et al. Intensive continuing medical
education course training on simulators results in proficiency for laparoscopic
suturing. Am J Surg. 2006;191(1):23e27.
Thomsen Ann Sofia Skou, Bach-Holm Daniella, Kjærbo Hadi, et al. Operating
room performance improves after proficiency-based virtual reality cataract
surgery training. Ophthalmology. 2017;124(4):524e531.
Gomez PP, Willis RE, Schiffer BL, Gardner AK, Scott DJ. External validation and
evaluation of an intermediate proficiency-based knot-tying and suturing curriculum. J Surg Educ. 2014 Nov-Dec;71(6):839e845.
Stefanidis D, Scerbo MW, Montero PN, Acker CE, Smith WD. Simulator training
to automaticity leads to improved skill transfer compared with traditional
proficiency-based training: a randomized controlled trial. Ann Surg.
2012;255(1):30e37.
Desilets DJ, Banerjee S, Barth BA, et al. Endoscopic simulators. Gastrointest
Endosc. 2011;73(5):861e867.
Please cite this article in press as: Franklin BR, et al., Preparing for the American Board of Surgery Flexible Endoscopy Curriculum: Development
of multi-institutional proficiency-based training standards and pilot testing of a simulation-based mastery learning curriculum for the
Endoscopy Training System, The American Journal of Surgery (2017), https://doi.org/10.1016/j.amjsurg.2017.09.010
Документ
Категория
Без категории
Просмотров
1
Размер файла
813 Кб
Теги
2017, amjsurg, 010
1/--страниц
Пожаловаться на содержимое документа