2910jum_online.qxp:Layout 1 9/20/10 3:09 PM Page 1423 Article CME Learning Curve of Emergency Physicians Using Emergency Bedside Sonography for Symptomatic First-Trimester Pregnancy Timothy B. Jang, MD, Wendy Ruggeri, MD, Pamela Dyne, MD, Amy H. Kaji, MD, PhD Objective. The purpose of this study was to prospectively assess the learning curve of emergency physician training in emergency bedside sonography (EBS) for first-trimester pregnancy complications. Methods. This was a prospective study at an urban academic emergency department from August 1999 through July 2006. Patients with first-trimester vaginal bleeding or pain underwent EBS followed by pelvic sonography (PS) by the Department of Radiology. Results of EBS were compared with those of PS using a predesigned standardized data sheet. Results. A total of 670 patients underwent EBS for first-trimester pregnancy complications by 1 of 25 physicians who would go on to perform at least 25 examinations. The sensitivity and specificity of EBS for an intrauterine pregnancy increased from 80% (95% confidence interval [CI], 71%–87%) and 86% (95% CI, 76%–93%), respectively, for a physician’s first 10 examinations to 100% (95% CI, 73%–100%) and 100% (95% CI, 63%–100%) for those performed after 40 examinations. Likewise, the sensitivity and specificity for an adnexal mass or ectopic pregnancy changed from 43% (95% CI, 28%–64%) and 94% (95% CI, 89%–97%) to 75% (95% CI, 22%–99%) and 89% (95% CI, 65%–98%), whereas the sensitivity and specificity for a molar pregnancy changed from 71% (95% CI, 30%–95%) and 98% (95% CI, 94%–99%) to 100% (95% CI, 20%–100%) and 100% (95% CI, 81%–100%). Although detection of an intrauterine or a molar pregnancy improved with training, even with experience including 40 examinations, the sensitivity of EBS for an adnexal mass or ectopic pregnancy was less than 90%. Conclusions. There is an appreciable learning curve among physicians learning to perform EBS for first-trimester pregnancy complications that persists past 40 examinations. Key words: ectopic pregnancy; first-trimester pregnancy; intrauterine pregnancy; learning curve. Abbreviations ACEP, American College of Emergency Physicians; CI, confidence interval; EBS, emergency bedside sonography; ED, emergency department; IUP, intrauterine pregnancy; OR, odds ratio; PGY, postgraduate year; PS, pelvic sonography; SAEM, Society for Academic Emergency Medicine Received April 20, 2010, from the Department of Emergency Medicine, David Geffen School of Medicine, Olive View Medical Center and UCLA Medical Center, Sylmar, California USA (T.B.J., W.R., P.D.); and Department of Emergency Medicine, David Geffen School of Medicine, Harbor-UCLA Medical Center, Torrance, California USA (T.B.J., A.H.K.). Revision requested May 17, 2010. Revised manuscript accepted for publication May 26, 2010. Address correspondence to Timothy B. Jang, MD, Department of Emergency Medicine, David Geffen School of Medicine, Olive View Medical Center and UCLA Medical Center, 14445 Olive View Dr, North Annex, Sylmar, CA 91342 USA. E-mail: tbj@ucla.edu CME Article includes CME test T he training of clinicians performing bedside sonography in the emergency department (ED) has been controversial, and there are different guidelines for their instruction.1–5 Although several studies have described the learning curve of clinicians performing bedside sonography for the evaluation of biliary disease and trauma,4–8 we are unaware of any study assessing the learning curve of physicians training in emergency bedside sonography (EBS) for the evaluation of first-trimester pregnancy complications. Therefore, the purpose of this study was to prospectively assess the learning curve of physicians training in EBS for first-trimester pregnancy complications. © 2010 by the American Institute of Ultrasound in Medicine • J Ultrasound Med 2010; 29:1423–1428 • 0278-4297/10/$3.50 2910jum_online.qxp:Layout 1 9/20/10 3:09 PM Page 1424 Learning Curve for Emergency First-Trimester Obstetric Sonography Materials and Methods Study Design This was an Institutional Review Board– approved prospective cohort study of emergency physicians training in EBS, using a convenience sample of patients enrolled between August 1, 1999, and July 31, 2006. Patients consented and underwent EBS before pelvic sonography (PS) by the Department of Radiology for the evaluation of first-trimester pregnancy complications. The study physicians performed EBS to detect sonographic signs of intrauterine and ectopic pregnancies, and results were recorded on a predesigned data sheet. Study Setting This study was conducted at an urban academic ED with 49,000 annual adult visits, a postgraduate year 2 (PGY-2) through PGY-4 emergency medicine residency and a PGY-1 through PGY5 combined internal medicine/emergency medicine residency. Selection of Participants The participating physicians were emergency physicians training in EBS and had completed a 2-day introductory course on emergency sonography, which included 2 EBS examinations on healthy pregnant volunteers. Both attending and resident physicians were included because the emergency ultrasound program was started in 1999, and none of the physicians at our institution met the training guidelines of the American College of Emergency Physicians (ACEP) or the Society for Academic Emergency Medicine (SAEM) regarding obstetric sonography. All patients presenting to the ED with firsttrimester vaginal bleeding, pain, or nausea and vomiting were eligible for participation if their treating physicians were ordering PS for firsttrimester pregnancy complications. Patients were excluded if they could not speak English, had known results of PS done within 60 days, or were otherwise unable to give informed consent. Protocol Participating physicians obtained consent from the patients and performed EBS to detect specific sonographic signs of intrauterine and ectopic 1424 pregnancies and recorded their results before PS was performed. Research assistants, trained in data abstraction and blinded to the results of the EBS examinations, then reviewed the results of PS read by board-certified radiologists from the Division of Ultrasound and Mammography within the Department of Radiology for subsequent comparison. Emergency bedside sonographic examinations done with the assistance of a more experienced physician were tracked by the examination experience of the senior physician, but EBS examinations done for training purposes after PS was performed in the Department of Radiology were excluded. These training examinations were tracked only for the purpose of assessing the overall experience of the physician. All EBS examinations were reviewed for education and feedback within 72 hours by a sonographer certified by the American Registry for Diagnostic Medical Sonography from 1999 to 2001 and a physician who met the ACEP and SAEM training guidelines from 2001 to 2006. The results of these reviews were not used to alter the completed data sheets because the performance of EBS depends on the technical and interpretive skills of the operator. Study Measurements Emergency bedside sonographic examinations were performed using an SSD-1400 system (Aloka Co, Ltd, Tokyo, Japan) with a 3- to 5-MHz curvilinear probe and a 7.5-MHz endocavitary probe and involved both transabdominal and endovaginal imaging to evaluate the following: (1) the presence of an intrauterine pregnancy (IUP), defined as a gestational sac with a yolk sac or the presence of an intrauterine fetal pole; (2) the presence of a molar pregnancy, defined sonographically by a “cluster of grapes” or “snowstorm” appearance; and (3) the presence of a definite ectopic pregnancy, defined as an extrauterine fetal pole or gestational sac with a yolk sac, or an abnormal adnexal mass, defined as an abnormal-appearing echogenic mass in the adnexa, an “adnexal ring” (ie, trophoblastic reaction in the fallopian tube), or an “interstitial line sign” (ie, cavitary line formed by myometrium pointing to an ectopic focus) of concern for a possible ectopic pregnancy. A simple ovarian cyst J Ultrasound Med 2010; 29:1423–1428 2910jum_online.qxp:Layout 1 9/20/10 3:09 PM Page 1425 Jang et al was not considered an abnormal adnexal mass (see Appendix). The criterion standard for each of these measures was the final reading of PS by board-certified radiologists blinded to the EBS results, which was performed using an iU22 system (Philips Healthcare, Bothell, WA) with an endocavitary probe. Data Analysis Data were collected in an Excel database (Microsoft Corporation, Redmond, WA) and translated into a native SAS format using DBMS/ Copy (Dataflux Corporation, Cary, NC). Analyses were conducted using SAS version 9.1 (SAS Institute Inc, Cary, NC). The sensitivity and specificity of EUS were calculated using 95% confidence intervals (CIs) to assess both statistical significance and the clinical effect.9 It was predetermined to track the EBS examinations per physician by experience level in blocks of 10 (ie, 1–10, 11–20, 21–30, 31–40, and >40 examinations). Thus, the data were hierarchical such that the second block of 10 EBS examinations for every physician was in group 2 (11–20); the third block for every physician was in group 3 (21–30); the fourth block for every physician was in group 4 (31–40); and the fifth block for every physician was in group 5 (>40). Consequently, the number of physicians in each successive group was anticipated to be smaller than that in the preceding group. Furthermore, it was predetermined to include only the data from the physicians who completed at least 25 training examinations, consistent with the ACEP minimum training guideline.10 Mantel-Haenszel χ2 testing was done to assess for changes in performance between each group. Finally, the GENMOD procedure was used to perform a clustered regression analysis to account for the fact that examinations by one operator are typically more similar than those performed by another operator; thus, the data analysis took intraoperator cluster correlation into account rather than assuming independence among all observations. Adequacy of the regression model fit was assessed by using the generalized score statistic criterion (P > .05) and quasi-likelihood information criterion, which is a modification of the Akaike information criterion. J Ultrasound Med 2010; 29:1423–1428 Results A total of 670 patients underwent EBS for firsttrimester pregnancy complications by 1 of 25 physicians who performed more than 25 examinations. The sensitivity and specificity of EBS for an IUP, a molar pregnancy, and an adnexal mass or ectopic pregnancy are shown in Table 1, showing an overall improvement with training. However, even with experience including 40 examinations, the sensitivity of EBS for an adnexal mass or ectopic pregnancy was less than 90%. The Mantel-Haenszel χ2 test for trend showed a statistically significant increase for true-positive identification of an IUP (χ2 = 8.6; P = .0034), as well as a statistically significant decrease in falsepositive identification of an IUP (χ2 = 4.1; P = .04), false-negative identification of an IUP (χ2 = 9.3; P = .002), and false-negative identification of an adnexal mass (χ2 = 9.1; P = .003). However, there was no statistically significant increase in trend for true-negative identification of an IUP (χ2 = 0.3; P = .6), true-positive identification of an adnexal mass (χ2 = 2.6; P = .1), or true-positive (χ2 = 1.1; P = .3) or true-negative (χ2 = 0.2; P = .6) identification of a molar pregnancy, nor was there a statistically significant decreasing trend Table 1. Sensitivity and Specificity of EBS for Specific Findings Condition, n (%) IUP (n = 408 total) 108 (57) 125 (62) 104 (59) 57 (70) 14 (61) Adnexal mass (n = 103 total) 31 (16) 30 (15) 24 (14) 14 (17) 4 (17) Molar pregnancy (n = 29 total) 7 (4) 10 (5) 7 (4) 3 (4) 2 (9) Group (n) Sensitivity, % (95% CI) Specificity, % (95% CI) 1 (188) 2 (202) 3 (175) 4 (82) 5 (23) 80 (71–87) 83 (75–89) 92 (85–96) 93 (82–98) 100 (73–100) 86 (76–93) 91 (82–96) 94 (85–98) 92 (72–99) 100 (63–100) 1 (188) 2 (202) 3 (175) 4 (82) 5 (23) 43 (28–64) 63 (44–79) 100 (78–100) 79 (49–94) 75 (22–99) 94 (89–97) 96 (91–98) 100 (97–100) 100 (93–100) 89 (65–98) 1 (188) 2 (202) 3 (175) 4 (82) 5 (23) 71 (30–95) 80 (44–96) 100 (56–100) 100 (31–100) 100 (20–100) 98 (94–99) 98 (95–99) 100 (96–100) 100 (94–100) 100 (81–100) A total of 670 patients were enrolled: 408 with IUPs, 103 with adnexal masses or ectopic pregnancies, 29 with molar pregnancies, and 130 without definite IUPs, adnexal masses or ectopic pregnancies, or molar pregnancies. 1425 2910jum_online.qxp:Layout 1 9/20/10 3:09 PM Page 1426 Learning Curve for Emergency First-Trimester Obstetric Sonography for false-negative identification of an adnexal mass (χ2 = 3.7; P = .06) or false-positive (χ2 = 3.2; P = .07), or false-negative (χ2 = 2.2; P = .1) identification of a molar pregnancy. When using a random effects model and cluster regression analysis to account for intraoperator correlation, for every incremental increase in 10 examinations, there was a statistically significant increase in true-positive identification of an IUP (odds ratio [OR], 1.13; 95% CI, 1.06–1.22; P = .0006) and an adnexal mass/ectopic pregnancy (OR, 1.10; 95% CI, 1.03–1.17; P = .005). There was also a statistically significant decrease in false-negative identification of an IUP (OR, 0.79; 95% CI, 0.68–0.93; P = .004) and an adnexal mass/ectopic pregnancy (OR= 0.78; 95% CI, 0.64–0.96; P = .02), as well as false-positive (OR, 0.75; 95% CI, 0.56–0.96; P = .02) and falsenegative (OR, 0.73; 95% CI, 0.55–0.96; P = .02) identification of a molar pregnancy. However, there was no significant increase in true-negative identification of an IUP (OR, 0.97; 95% CI, 0.93–1.02; P = .2), true-negative identification of an adnexal mass/ectopic pregnancy (OR, 1.02; 95% CI, 0.94–1.12; P = .5), or true- positive (OR, 1.1; 95% CI, 0.99–1.22; P = .07) or true-negative (OR, 0.98; 95% CI, 0.11–1.1; P = .7) identification of a molar pregnancy, nor was there a decrease in false-positive identification of an IUP (OR, 0.86; 95% CI, 0.71–1.05; P = .1) or false-positive identification of an adnexal mass/ectopic pregnancy (OR, 0.87; 95% CI, 0.69–1.09; P = .2). Discussion Several different guidelines have been proposed for training clinicians to perform various sonographic examinations outside the radiology suite.1–3,5 The American Institute of Ultrasound in Medicine recommends at least 300 sonographic examinations for obstetric indications.11 However, because EBS is focused and goal oriented, being done at the point of care, rather than comprehensive and extensive, being done in the radiology suite, the ACEP recommends a minimum of 25 examinations for training in each EBS indication,10 consistent with similar findings from the family medicine12 and obstetric nursing13 literature. 1426 Our data suggest that the detection of IUPs with EBS among women presenting to the ED with first-trimester complications improves with training but requires more than the 15 examinations suggested by data regarding focused sonography done by obstetric nurses in later pregnancy13. We suspect that the need for increased training is due to the difficulty of making an accurate diagnosis in early pregnancy as opposed to visualizing a large intrauterine fetus in later pregnancy. In our study sample, the ability to detect IUPs at earlier gestational ages improved with training. In group 1, there were 22 false-negative results for an IUP, with a mean gestational age of 5 weeks 1 day, compared with groups 4 and 5, where there were 4 false-negative results for an IUP, with a mean gestational age of 4 weeks 1 day. In groups 4 and 5, there were 19 pregnancies detected with gestational ages between 3 and 6 weeks. On the other hand, although detection of an IUP improved with training, even with experience including 40 examinations, the sensitivity of EBS for an abnormal adnexal mass or ectopic pregnancy was less than 90%. This suggests that even more training in EBS is required to rule out an ectopic pregnancy. Several studies in the literature have shown that emergency physicians could perform EBS for the diagnosis of an IUP and expedite care in the ED.14–16 Our data are consistent with these prior studies but clearly show a learning curve for acquiring such skills. Although our data suggest an asymptotic leveling of the learning curve at around 40 examinations, the low number of physicians with more than 25 examinations limits the conclusions that can be drawn. In our experience, physicians gain facility with EBS in the 26- to 40examination range that allows them to reliably detect an IUP. However, because the diagnosis of an ectopic pregnancy can be more difficult due to subtleties related to imaging of the adnexa, training including 50 to 75 examinations is required before the sensitivity approaches 90% for an ectopic pregnancy. Therefore, our institutions require more than 25 EBS examinations, a minimum of 4 IUPs at less than 8 weeks’ gestation, a minimum of 4 ectopic pregnancies, and a minimum accuracy rate of 90% for privileges to perform EBS for first-trimester pregnancy complications. J Ultrasound Med 2010; 29:1423–1428 2910jum_online.qxp:Layout 1 9/20/10 3:09 PM Page 1427 Jang et al Because the Residency Review Committee for emergency medicine only required 40 total sonographic examinations per resident before 2007, it is not surprising that few physicians in our sample performed more than 25 EBS examinations. This was also the case in recent studies of emergency sonography for biliary disease and trauma.5,8,17 This could be improved with a multicenter study examining the changes in competency that occur during an emergency ultrasound fellowship, in which fellows are expected to perform 1000 sonographic examinations and more than 150 obstetric examinations. This study had several limitations. First, only 25 of 137 eligible physicians (18%) performed more than 25 examinations, giving us a small sample of physicians who met the ACEP training guidelines. This represents an “ultrasound interest” bias on the part of the physicians who were clearly interested in performing the examinations and knew that their results were being studied, raising the potential for a Hawthorne effect. Our findings may not apply to other physicians because sonography is operator dependent, and poor skills may persist despite training including up to 200 examinations.18 We suspect this is one reason why the Residency Review Committee for emergency medicine recently changed the EBS training requirement from a minimum of 40 total examinations to documentation of competency.19 Second, whereas hands-on experience and the number of examinations are related to competency, they may not be the most important factors in developing competency. It may be that the number of examinations with positive findings is more important than the total number of examinations performed. That was not assessed in this study but should be addressed in the future. Third, this was a single-center study with a particular training and quality assurance protocol. Emergency medicine residencies with different resources (eg, number of ultrasound machines and available CD learning modules), personnel (eg, registered diagnostic medical sonographers and fellowship-trained ultrasound directors), and training requirements (eg, hours of didactic training, amount of hands-on teaching, and J Ultrasound Med 2010; 29:1423–1428 required number of sonographic examinations for graduation) might have different learning curves. This could be improved with a multicenter study in the future. Unfortunately, whereas the ACEP and SAEM have clear guidelines, fewer than one-third of residencies meet them,20 which would make it difficult to control for such variations in training. Finally, sonography is operator dependent and depends on the “inherent ability of trainees to learn sonographic skills.”18 That was not the focus of this investigation but should be studied further. In a study of radiology residents, every resident had a learning curve, but those who started out with low competency scores continued to have poor performance, even with training and experience including up to 200 examinations.18 However, when addressing minimum training standards, requirements should be set at a level to ensure that most physicians, not just the naturally adept, can competently perform EBS. Our data suggest that a minimum standard of 25 examinations is inadequate for ensuring competency in EBS for the diagnosis of ectopic pregnancy. In conclusion, there is an appreciable learning curve among physicians learning to perform EBS for first-trimester pregnancy complications that persists past 40 examinations. Appendix: Data Sheet Items Identifying information: Patient name______________________________________________ Hospital number __________________________________________ Date of service ____________________________________________ Physician-sonographer name ________________________________ Sonographic findings: ___ IUP (EGA ___ weeks ___ days by ___ GS with yolk sac __ CRL __ BPD __ FL) ___ Ectopic pregnancy (extrauterine gestational sac with yolk sac or fetal pole) or __ abnormal adnexal mass ___ Molar pregnancy ___ None of the above BPD indicates biparietal diameter; CRL, crown-rump length; EGA, estimated gestational age; FL, femur length; and GS, gestational sac. 1427 2910jum_online.qxp:Layout 1 9/20/10 3:09 PM Page 1428 Learning Curve for Emergency First-Trimester Obstetric Sonography References 1. 1428 Robinson NA, Clancy MJ. Should UK emergency physicians undertake diagnostic ultrasound examinations? J Accid Emerg Med 1999; 16:248–249. 2. Bullock C. AIUM and SAEM square off over ultrasound. Emerg Med News 1993; 15:1. 3. Lanoix R. Credentialing issues in emergency ultrasonography. Emerg Med Clin North Am 1997; 15:913–920. 4. Shackford SR, Rogers FB, Osler TM, Trabusky ME, Clauss DW, Vane DW. Focused abdominal sonogram for trauma: the learning curve of nonradiologist clinicians in detecting hemoperitoneum. J Trauma 1999; 46:553–564. 5. Jang T, Sineff S, Naunheim R, Aubin C. Residents should not independently perform focused abdominal sonography for trauma after 10 training examinations. J Ultrasound Med 2004; 23:793–797. 6. Smith RS, Kern SJ, Fry WR, Helmer SD. Institutional learning curve of surgeon-performed trauma ultrasound. Arch Surg 1998; 133:530–536. 7 Ma OJ, Mateer JR, Ogata M, Kefer MP, Wittmann D, Aprahamian C. Prospective analysis of a rapid trauma ultrasound examination performed by emergency physicians. J Trauma 1995; 38:879–885. 8. Jang T, Aubin C, Naunheim R. Minimum training for right upper quadrant ultrasonography. Am J Emerg Med 2004; 22:439–443. 9. Hanley JA. The place of statistical methods in radiology (and in the bigger picture). Invest Radiol 1989; 24:10–16. 10. American College of Emergency Physicians Board of Directors. ACEP emergency ultrasound guidelines, 2008. American College of Emergency Physicians website. http://www.acep.org/WorkArea/DownloadAsset.aspx?id= 32878. Accessed August 31, 2009. 11. American Institute of Ultrasound in Medicine. Training guidelines for physicians who evaluate and interpret diagnostic ultrasound examinations. American Institute of Ultrasound in Medicine website. http://www.aium.org/ publications/statements.aspx. Accessed August 31, 2009. 12. Dresang LT, Rodney WM, Dees J. Teaching prenatal ultrasound to family medicine residents. Fam Med 2004; 36:98–107. 13. Stringer M, Miesnik SR, Brown LP, Menei L, Macones GA. Limited obstetric ultrasound examinations: competency and cost. J Obstet Gynecol Neonatal Nurs 2003; 32: 307–312. 14. Shih CHY. Effect of emergency physician-performed pelvic sonography on length of stay in the emergency department. Ann Emerg Med 1997; 29:348–352. 15. Burgher SW, Tandy TK, Dawdy MR. Transvaginal ultrasonography by emergency physicians decreases patient time in the emergency department. Acad Emerg Med 1998; 5:802–807. 16. Blaivas M, Sierzenski P, Plecque D, Lambert M. Do emergency physicians save time when locating a live intrauterine pregnancy with bedside ultrasonography? Acad Emerg Med 2000; 7:988–993. 17. Ma JO, Gaddis GM, Robinson LH, et al. Accuracy of FAST examination interpretation as influenced by anechoic stripe size and physician experience level [abstract]. Acad Emerg Med 2004; 11(suppl):581. 18. Hertzberg BS, Kliewer MA, Bowie JD, et al. Physician training requirements in sonography: how many cases are needed for competence? AJR Am J Roentgenol 2000; 174:1221–1227. 19. Accreditation Council for Graduate Medical Education. Emergency medicine guidelines. Accreditation Council for Graduate Medical Education website. http://www. acgme.org/acWebsite/RRC_110/110_guidelines.asp#res. Accessed August 31, 2009. 20. Bakhtiari P, Gough JE, Brewer KL. Do emergency medicine residency programs meet American College of Emergency Physicians requirement of ultrasound curricula [abstract]? Ann Emerg Med 2005; 46(suppl):S79. J Ultrasound Med 2010; 29:1423–1428
Автор
1/--страниц