Defining the learning curve of point-of-care ultrasound for confirming endotracheal tube placement by emergency physicians
© Chenkin et al. 2015
Received: 16 June 2015
Accepted: 26 August 2015
Published: 17 September 2015
Unrecognized esophageal intubations are associated with significant patient morbidity and mortality. No single confirmatory device has been shown to be 100 % accurate at ruling out esophageal intubations in the emergency department. Recent studies have demonstrated that point-of-care ultrasound (POCUS) may be a useful adjunct for confirming endotracheal tube placement; however, the amount of practice required to become proficient at this technique is unclear. The purpose of this study is to determine the amount of practice required by emergency physicians to become proficient at interpreting ultrasound video clips of esophageal and endotracheal intubations.
Emergency physicians and emergency medicine residents completed a baseline interpretation test followed by a 10 min online tutorial. They then interpreted POCUS clips of esophageal and endotracheal intubations in a randomly selected order. If an incorrect response was provided, the participant completed another practice session with feedback. This process continued until they correctly interpreted ten consecutive ultrasound clips. Descriptive statistics were used to summarize the data.
Of the 87 eligible physicians, 66 (75.9 %) completed the study. The mean score on the baseline test was 42.9 % (SD 32.7 %). After the tutorial, 90.9 % (60/66) of the participants achieved proficiency after one practice attempt and 100 % achieved proficiency after two practice attempts. Six intubation ultrasound clips were misinterpreted, for a total error rate of 0.9 % (6/684). Overall, the participants had a sensitivity of 98.3 % (95 % CI 96.3–99.4 %) and specificity of 100 % (95 % CI 98.9–100 %) for detecting correct tube location. Scans were interpreted within an average of 4 s (SD 2.9 s) of the intubation.
After a brief online tutorial and only two practice attempts, emergency physicians were able to quickly and accurately interpret ultrasound intubation clips of esophageal and endotracheal intubations.
Emergency intubations are associated with a significant risk of esophageal intubation, which can be rapidly fatal if not recognized and corrected quickly . Qualitative color capnography is commonly used to help to confirm endotracheal tube position; however, it has been shown to be indeterminate or unreliable in a significant proportion of emergency department patients . In addition, capnography requires that ventilations be delivered to the patient which can increase the risk of aspiration if the tube is misplaced in the esophagus. No single airway confirmation device has been shown to be 100 % accurate in all patient scenarios; therefore, emergency physicians should consider the use of multiple confirmation techniques to reduce the risk of an unrecognized esophageal intubation.
Recently, there has been increasing evidence supporting the use of point-of-care ultrasound (POCUS) for confirmation of endotracheal tube placement. Ultrasound can be used to identify the endotracheal tube position in real time with a high degree of accuracy and without the need for any ventilations being delivered to the patient [3–14]. Two recent meta-analyses found a combined sensitivity of 93–98 % and specificity of 97–98 % for the use of ultrasound for confirming endotracheal tube location [15, 16]. A strategy of combining POCUS and capnography may significantly reduce the risk of an esophageal intubation going undetected and prevent the associated patient morbidity and mortality .
Despite the increasing evidence to support the use of POCUS for confirming endotracheal intubation, there is limited data to guide training protocols for this technique. Specifically, the required amount and optimal format of training as well as the number of scan interpretations required to achieve competence with the technique have not been evaluated. Since the consequences of misinterpreting an intubation ultrasound could be devastating, it is important that any training protocol is sufficient for the learners to achieve a very high accuracy at image interpretation. The objective of this study is to determine the amount of practice required by emergency physicians to achieve proficiency with interpretation of POCUS video clips of endotracheal tube placement.
We conducted a prospective educational study to evaluate the learning curve of interpreting POCUS video clips for endotracheal intubation. Using the principles of effective Web-based educational material development, we developed an online educational module covering the technique of using ultrasound for confirming endotracheal intubation . We designed an online assessment tool that allowed us to track the number of practice attempts and correct and incorrect interpretations from each participant. The educational module and assessment tool were pilot tested by several physicians with experience in the technique and modifications were made based on their feedback. The study was approved by the institutional Research Ethics Board and each participant provided informed consent.
Study setting and population
The study was performed between July and October 2014. All emergency physicians from a single academic emergency department and emergency medicine residents from the associated university program were invited to participate, regardless of their prior experience level with POCUS. The emergency department has an active ultrasound program; however, there is significant variation in its clinical use among the emergency physicians. The residency training program includes a 1-month rotation in emergency ultrasound; however, the curriculum does not include any training on the use of ultrasound for endotracheal tube confirmation.
Demographics of intubated patients
N = 20
Age, mean (year)
64.1 ± 15.2
Female sex, no. (%)
Weight, mean (kg)
80.1 ± 13.3
Height, mean (cm)
171 ± 10.4
BMI, mean (kg/m2)
27.3 ± 3.3
Difficult airway, no. (%)
Emergency physicians and residents were invited to participate in the study by email invitations sent by a research coordinator. Participants were directed to an online tutorial where they first completed a baseline questionnaire and a baseline image interpretation test consisting of five esophageal and five tracheal intubation videos in random sequence. For each video clip, participants were asked whether the tube was placed in the trachea or the esophagus. Following this baseline test, participants completed a 10-min Web-based tutorial which covered the background of POCUS for confirming endotracheal intubation, ultrasound technique, image interpretation, and common pitfalls. Our tutorial included commonly accepted ultrasound findings for tracheal intubation (fluttering movement of the tube within the trachea and no change in the appearance of the esophagus) and esophageal intubation (appearance of a ‘double tract’ second shadow adjacent to the trachea). Following the tutorial, the participants completed a single practice interpretation with specific feedback provided based on their response.
To evaluate the learning curve, participants then completed an online test asking for interpretation of ultrasound clips of esophageal and endotracheal intubations selected in a random order. If the clip was correctly interpreted, they proceeded to the next clip. If the clip was incorrectly interpreted, the participant was required to complete more practice with feedback. No clip was seen more than one time per participant. This process continued until the participant achieved ten correct interpretations in a row. We selected ten sequential correct interpretations as our end point based on pilot testing with several staff, which suggested that the number of interpretations required to achieve proficiency was significantly less than ten. Following the assessment, a questionnaire was completed to assess participants’ comfort level with the technique.
Self-reported comfort scores were measured using an anchored 5-point Likert scale. The assessment protocol was scored based on the number of practice sessions required, as well as the amount of time required to interpret the scans. Clip interpretation time was measured from when the intubation clip first started playing until interpretation was completed.
Data were analyzed using Microsoft Excel (Microsoft Corp, WA, USA). Descriptive statistics were used to report pre-test scores, number of practice attempts, and time to interpretation. Sensitivity and specificity of image interpretation were calculated along with their 95 % confidence intervals by pooling all of the interpretations performed in the assessment phase of the study. Comfort scores were compared using the Wilcoxon signed-rank test using a two-tailed p value.
Demographics of study participants
N = 66 (%)
Basic ultrasound certification
Attended advanced ultrasound course
Previous intubation ultrasound experience
Comfort levels with image interpretation and performance of ultrasound scan before and after the tutorial
Comfortable or very comfortable with interpretation of images
1/66 (1.5 %)
63/66 (95.5 %)
Comfortable or very comfortable with performance of technique
1/66 (1.5 %)
48/66 (72.7 %)
The use of point-of-care ultrasound for confirming endotracheal tube placement is a relatively new, yet promising technique that may help to avoid the morbidity associated with unrecognized esophageal intubations. With any new procedure where the learning curve is not yet defined, it is important to understand the amount of practice required to achieve competence. There is a wide variation in learning curves for the various point-of-care ultrasound techniques used by emergency physicians. Some scans require a significant amount of practice, whereas others are quick to master . With endotracheal tube confirmation, any new technique must be easy to learn, and results must be able to be interpreted quickly with a high degree of accuracy.
The results from this study demonstrate that after a brief online tutorial and only one or two practice sessions with feedback, emergency physicians can quickly and accurately interpret ultrasound videos of esophageal and tracheal intubations. There are several possible reasons that may be contributing to the relatively short learning curve for this technique. Unlike many other emergency ultrasound applications, the transducer is placed in a uniform location and once placed does not need to move. The anatomy is relatively consistent between patients and is easy to identify. Body habitus plays less of a factor than in abdominal scans, as even in obese patients the airway anatomy is easily visible on ultrasound.
Previous studies have demonstrated that point-of-care ultrasound is very accurate at differentiating esophageal from tracheal tube placement. However, many of these studies are limited to including very few sonologists or by having the investigators themselves performing the ultrasound scans [3–5, 8, 9, 12, 19]. This limits the generalizability of the results and provides no information on whether the scan technique can be performed by emergency physicians with limited experience with ultrasound for intubation.
Other intubation ultrasound studies that included non-expert sonologists reported accuracy levels similar to our study findings after only brief training interventions. Goksu et al. found that after a 15-min presentation, seven physicians achieved an overall sensitivity of 95.7 % and specificity of 98.2 % . Ma found that after a 5-min briefing, seven residents achieved a sensitivity of 97 % and specificity of 100 % on cadaver models . Uya et al. found that after a 20-min didactic session and a 30-min practice session, eight novice pediatric emergency medicine fellows achieved a sensitivity of 96 % for tracheal location . These studies were limited by the small numbers of participants and the use of a cadaver model for the intubations.
One previously published study attempted to evaluate the level of difficulty of endotracheal tube localization using ultrasound . In this study, 29 participants with different levels of general point-of-care ultrasound experience identified endotracheal tube location on cadavers. In this study, experienced sonologists (defined as having >150 scans) demonstrated a higher sensitivity and specificity compared with less experienced sonologists (sensitivity 75.0 vs 62.0 %, specificity 62.5 vs 37.9 %). This study was limited by the use of cadaver models and a static technique after the intubation was completed, which may have led to the relatively poor accuracy .
Our study adds to the previously published literature in several ways. First, we included a large cohort of staff and resident physicians with a broad range of ultrasound experience from complete novices to some who have completed an ultrasound fellowship. Unlike many of the previous studies, we used live patient intubations as opposed to cadaver models. We also included patients with a wide range of body habitus and scan difficulty ranging from easy to difficult to interpret. Our study is the first to use repeated practice with tailored feedback to determine the learning curve for interpreting intubation ultrasound clips. Despite the variation in experience level and patient population, we found that all participants were highly accurate with their ultrasound interpretations after only two practice attempts with feedback.
All of the clip misinterpretations in our study were due to tracheal intubations thought to be in the esophagus. While not as dangerous as missing an esophageal intubation, this error may still result in patient harm if a correctly placed tube is unnecessarily removed. The findings of this study fit with our real-world experience, where the fluttering movement seen with correct tube placement is more subtle than the double tract sign of an esophageal intubation. To avoid this pitfall, one strategy is to locate the esophagus before the intubation and observe if it changes during the passage of the tube. If it does not, the tube is located in the trachea. If a second shadow appears where the esophagus lies, the tube is in the esophagus. Another important pitfall which was not seen in this study was the esophagus that lies posterior to the trachea. In this situation, identification of an esophageal tube can be difficult. This pitfall can be avoided by keeping the transducer placement as low as possible on the neck and applying slight pressure to help bring the esophagus into view.
This study has several limitations. For logistical reasons, we only evaluated image interpretation and not image generation. In actual practice, emergency physicians must be able to simultaneously generate and interpret images to make clinical decisions in real time. However, we have found that generating ultrasound images for endotracheal tube confirmation is relatively easy compared with other ultrasound techniques. Our test bank of ultrasound clips was relatively small, consisting of 20 esophageal and 20 tracheal intubations. However, we included clips taken from patients with a broad range of neck anatomy and difficulty of intubation. We selected ten consecutive correct interpretations as a marker for proficiency, and it is possible that a larger sample would have revealed different results. However, we noted that the vast majority of participants were successful after only one practice attempt, supporting our theory that these images are relatively easy to learn. It seems unlikely that many more cases would have made a difference in the study findings. For logistical reasons, the tutorial and evaluations were completed on the same day. Ideally, a washout period would be preferred to test knowledge retention. Finally, the ultrasound clips taken of elective intubations in the controlled environment of the operating room may not be generalizable to emergency department intubations, where images may be more difficult to generate or interpret.
In this study, we found that emergency physicians were able to accurately interpret ultrasound clips for endotracheal intubation after a brief online tutorial and only two practice attempts. Given that image interpretation appears to be easily learned, this technique may be a useful adjunct to other airway confirmation devices in the emergency department. Future studies are needed to determine the learning curve for airway image generation and to confirm these findings in emergency department intubations.
JC conceived the study, participated in its design and execution, and drafted the manuscript. CM helped to acquire the video clips used in the study and revised the manuscript. TJ assisted with data collection and analysis, and revised the manuscript. MR assisted with data analysis and revised the manuscript. CH assisted with data analysis and revised the manuscript. GB assisted with the study design, data analysis, and revised the manuscript. All authors read and approved the final manuscript.
Compliance with ethical guidelines
Competing interests The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Cook TM, Woodall N, Harper J, Benger J, Fourth National Audit P (2011) Major complications of airway management in the UK: results of the Fourth National Audit Project of the Royal College of Anaesthetists and the Difficult Airway Society. Part 2: intensive care and emergency departments. Br J Anaesth 106(5):632–642. doi:10.1093/bja/aer059 View ArticlePubMedGoogle Scholar
- Li J (2001) Capnography alone is imperfect for endotracheal tube placement confirmation during emergency intubation. J Emerg Med 20(3):223–229View ArticlePubMedGoogle Scholar
- Chou HC, Chong KM, Sim SS, Ma MH, Liu SH, Chen NC, Wu MC, Fu CM, Wang CH, Lee CC, Lien WC, Chen SC (2013) Real-time tracheal ultrasonography for confirmation of endotracheal tube placement during cardiopulmonary resuscitation. Resuscitation 84(12):1708–1712. doi:10.1016/j.resuscitation.2013.06.018 View ArticlePubMedGoogle Scholar
- Adi O, Chuan TW, Rishya M (2013) A feasibility study on bedside upper airway ultrasonography compared to waveform capnography for verifying endotracheal tube location after intubation. Crit Ultrasound J 5(1):7. doi:10.1186/2036-7902-5-7 PubMed CentralView ArticlePubMedGoogle Scholar
- Chou HC, Tseng WP, Wang CH, Ma MH, Wang HP, Huang PC, Sim SS, Liao YC, Liao YC, Chen SY, Hsu CY, Yen ZS, Chang WT, Huang CH, Lien WC, Chen SC (2011) Tracheal rapid ultrasound exam (T.R.U.E.) for confirming endotracheal tube placement during emergency intubation. Resuscitation 82(10):1279–1284. doi:10.1016/j.resuscitation.2011.05.016 View ArticlePubMedGoogle Scholar
- Galicinao J, Bush AJ, Godambe SA (2007) Use of bedside ultrasonography for endotracheal tube placement in pediatric patients: a feasibility study. Pediatrics 120(6):1297–1303. doi:10.1542/peds.2006-2959 View ArticlePubMedGoogle Scholar
- Marciniak B, Fayoux P, Hebrard A, Krivosic-Horber R, Engelhardt T, Bissonnette B (2009) Airway management in children: ultrasonography assessment of tracheal intubation in real time? Anesth Analg 108(2):461–465. doi:10.1213/ane.0b013e31819240f5 View ArticlePubMedGoogle Scholar
- Milling TJ, Jones M, Khan T, Tad-y D, Melniker LA, Bove J, Yarmush J, SchianodiCola J (2007) Transtracheal 2-d ultrasound for identification of esophageal intubation. J Emerg Med 32(4):409–414. doi:10.1016/j.jemermed.2006.08.022 View ArticlePubMedGoogle Scholar
- Park SC, Ryu JH, Yeom SR, Jeong JW, Cho SJ (2009) Confirmation of endotracheal intubation by combined ultrasonographic methods in the Emergency Department. Emerg Med Aust EMA 21(4):293–297. doi:10.1111/j.1742-6723.2009.01199.x View ArticleGoogle Scholar
- Pfeiffer P, Rudolph SS, Borglum J, Isbye DL (2011) Temporal comparison of ultrasound vs. auscultation and capnography in verification of endotracheal tube placement. Acta Anaesthesiol Scand 55(10):1190–1195. doi:10.1111/j.1399-6576.2011.02501.x View ArticlePubMedGoogle Scholar
- Weaver B, Lyon M, Blaivas M (2006) Confirmation of endotracheal tube placement after intubation using the ultrasound sliding lung sign. Acad Emerg Med Off J Soc Acad Emerg Med 13(3):239–244. doi:10.1197/j.aem.2005.08.014 View ArticleGoogle Scholar
- Werner SL, Smith CE, Goldstein JR, Jones RA, Cydulka RK (2007) Pilot study to evaluate the accuracy of ultrasonography in confirming endotracheal tube placement. Ann Emerg Med 49(1):75–80. doi:10.1016/j.annemergmed.2006.07.004 View ArticlePubMedGoogle Scholar
- Abbasi S, Farsi D, Zare MA, Hajimohammadi M, Rezai M, Hafezimoghadam P (2015) Direct ultrasound methods: a confirmatory technique for proper endotracheal intubation in the emergency department. Eur J Emerg Med Off J Eur Soc Emerg Med 22(1):10–16. doi:10.1097/MEJ.0000000000000108 View ArticleGoogle Scholar
- Hoffmann B, Gullett JP, Hill HF, Fuller D, Westergaard MC, Hosek WT, Smith JA (2014) Bedside ultrasound of the neck confirms endotracheal tube position in emergency intubations. Ultraschall Med 35(5):451–458. doi:10.1055/s-0034-1366014 View ArticlePubMedGoogle Scholar
- Das SK, Choupoo NS, Haldar R, Lahkar A (2014) Transtracheal ultrasound for verification of endotracheal tube placement: a systematic review and meta-analysis. Can J Anaesth. doi:10.1007/s12630-014-0301-z PubMedGoogle Scholar
- Chou EH, Dickman E, Tsou PY, Tessaro M, Tsai YM, Ma MH, Lee CC, Marshall J (2015) Ultrasonography for confirmation of endotracheal tube placement: a systematic review and meta-analysis. Resuscitation 90:97–103. doi:10.1016/j.resuscitation.2015.02.013 View ArticlePubMedGoogle Scholar
- Cook DA, Dupras DM (2004) A practical guide to developing effective web-based learning. J Gen Intern Med 19(6):698–707. doi:10.1111/j.1525-1497.2004.30029.x PubMed CentralView ArticlePubMedGoogle Scholar
- Blehar DJ, Barton B, Gaspari RJ (2015) Learning curves in emergency ultrasound education. Acad Emerg Med Off J Soc Acad Emerg Med 22(5):574–582. doi:10.1111/acem.12653 View ArticleGoogle Scholar
- Muslu B, Sert H, Kaya A, Demircioglu RI, Gozdemir M, Usta B, Boynukalin KS (2011) Use of sonography for rapid identification of esophageal and tracheal intubations in adult patients. J Ultrasound Med Off J Am Inst Ultrasound Med 30(5):671–676Google Scholar
- Goksu E, Sayrac V, Oktay C, Kartal M, Akcimen M (2010) How stylet use can effect confirmation of endotracheal tube position using ultrasound. Am J Emerg Med 28(1):32–36. doi:10.1016/j.ajem.2008.09.024 View ArticlePubMedGoogle Scholar
- Ma G, Davis DP, Schmitt J, Vilke GM, Chan TC, Hayden SR (2007) The sensitivity and specificity of transcricothyroid ultrasonography to confirm endotracheal tube placement in a cadaver model. J Emerg Med 32(4):405–407. doi:10.1016/j.jemermed.2006.08.023 View ArticlePubMedGoogle Scholar
- Uya A, Spear D, Patel K, Okada P, Sheeran P, McCreight A (2012) Can novice sonographers accurately locate an endotracheal tube with a saline-filled cuff in a cadaver model? A pilot study. Acad Emerg Med Off J Soc Acad Emerg Med 19(3):361–364. doi:10.1111/j.1553-2712.2012.01306.x View ArticleGoogle Scholar
- Stuntz R, Kochert E, Kehrl T, Schrading W (2014) The effect of sonologist experience on the ability to determine endotracheal tube location using transtracheal ultrasound. Am J Emerg Med 32(3):267–269. doi:10.1016/j.ajem.2013.11.032 View ArticlePubMedGoogle Scholar