Open Access

Rules for the road: an evidence-based approach to understanding diagnostic test performance of point-of-care ultrasound for pediatric abdominal emergencies

Critical Ultrasound Journal20101:20

https://doi.org/10.1007/s13089-010-0020-9

Received: 2 December 2009

Accepted: 2 February 2010

Published: 4 March 2010

I like to think there are many similarities between ultrasound machines and automobiles. Just as cars come in all shapes, sizes, functions (sports-car, minivans, pickup trucks, etc.), and price ranges, so do ultrasound machines—from the newer handheld ones now entering the market, laptop-sized ultrasound machines (which has allowed point-of-care ultrasound to rapidly diffuse into many areas of medicine), and to console-type ultrasound machines used by radiology departments [1]. Learning (and teaching someone) how to use an ultrasound machine is not so different from learning (and teaching someone) how to drive a car. Cars with automatic transmission are easier to learn to drive than those with manual transmission; driving in a large parking lot, or a rural area with no other cars on the road is easier to learn than driving in places like New York City or London. Similarly, some ultrasound applications are easier to learn than others: from fairly simple—looking at the bladder to assess urine volume prior to collection; a little more complex—a Focused Assessment with Sonography in Trauma (FAST) examination; to the very difficult—systematically finding and demonstrating a normal appendix.

We follow rules for the road in driving and when they are not followed we know on a daily basis in our emergency departments that catastrophes can occur; ultrasound in a clinician’s hands is no different—especially in those just learning to scan. Point-of-care ultrasound is a diagnostic test like any other and understanding the diagnostic test performance characteristics of ultrasound will help us stay safe on the “road to diagnosis” and here is where evidence-based medicine, or more specifically “evidence-based critical ultrasound [2]” can help us. Most discussion of errors in medicine have been related to preventing treatment-related harm, with little attention given to preventing diagnostic errors and misdiagnosis-related harm which are more common than harm to patients from medication-related errors [3]. When we look at the sensitivities and specificities for ultrasound to diagnose various abdominal conditions afflicting children such as free fluid in the abdomen in pediatric trauma [4, 5], pyloric stenosis [68], intussusception [9, 10], and appendicitis [11, 12], in general, the specificities are higher (usually >90%) than the sensitivities (Table 1). How does it help us to know that a diagnostic test tends to have higher specificity than sensitivity?
Table 1

Test characteristics of ultrasound for pediatric abdominal applications

Test

Sensitivity (%)

Specificity (%)

Likelihood ratio for a positive US

Likelihood ratio for a negative US

Pediatric FAST Exam [5]

82 (65–93)

95 (91–97)

15.6 (8.37–29.18)

0.19 (0.09–0.4)

Pyloric Stenosis [6]

98 (92–99.5)

99 (90–100)

82.35 (5.23–1295)

0.02 (0.004–0.097)

Intussusception [10]

83 (44–99)

97 (85–100)

27.5 (3.86–195.81)

0.17 (0.03–1.03)

Appendicitis [11]

44 (31–58)

93 (86–97)

6.5 (2.84–15.02)

0.53 (0.45–0.77)

PoC US for Appendicitis [12]a

65 (52–76)

90 (85–100)

6.4 (3.09–13.25)

0.4 (0.27–0.56)

aMixed population of adults and children

In David Sackett’s evidence-based medicine book [13], there is a handy mnemonic: A high Specificity test, a Positive result effectively “rules IN” disease (SpPIN). For tests with a high Sensitivity, a Negative result effectively “rules OUT” disease (SnNOUT). In general, ultrasound with its higher specificities relative to sensitivities is more accurate for “ruling in” disease, than “ruling out” disease (with very few exceptions). Thus, ultrasound is more reliable in identifying pathology (e.g., free fluid on a FAST exam, a shadowing appendicolith, the classic target of intussuception, etc.) than in completely ruling out the presence of disease or abnormality (e.g., absence of free fluid on FAST exam, normal muscle wall thickness of a vomiting infant’s pylorus, etc.).

However, sensitivity and specificity are traditional measures of diagnostic test utility that can be difficult to interpret, particularly when the values are not very high or not very low. Furthermore, they do not directly address the probability of disease in an individual patient after a diagnostic test has been applied. Likelihood ratios (LRs) are a measure of the predictive power of a test and may be more intuitive to understand than sensitivity and specificity. They are defined as the likelihood that a given test result would be found in a patient with a target disorder, relative to the likelihood of the same test result occurring in a patient without the target disorder. LRs can be derived for any diagnostic test, whether it is a clinical sign or symptom, laboratory test, or diagnostic imaging modality like ultrasound (presented in Table 1). LRs can be applied to a pre-test probability of disease in order to generate a post-test probability of disease. The direction and magnitude of change in probability from pre-test to post-test are determined by a test’s performance characteristics, which are incorporated by the likelihood ratio. Diagnostic tests with LRs >10 or <0.1 significantly alter pre-test probabilities, LRs >5 or <0.2 have moderate effects on pre-test probabilities, while LRs closer to 1 have little effect. Looking at Table 1, likelihood ratios for positive ultrasound results tend to increase pre-test probabilities more than it decreases pre-test probabilities with negative ultrasound results. If a resulting post-test probability of disease is high enough to make the diagnosis certain, it facilitates the clinician’s decision to treat. However, if the post-test probability is not high enough to confirm the diagnosis, a clinician may pursue further diagnostic testing. Detailed discussions of likelihood ratios are available elsewhere [13, 14].

Thus, the prime rule for the road when using point-of-care ultrasound: do not rely on ultrasound alone to “rule out” disease. With respect to likelihood ratios for negative ultrasound results, the post-test probability will probably not be low enough to confidently exclude disease. This will keep you from sending home the intussusception, pyloric stenosis or appendicitis solely based on a “negative” ultrasound. The worst thing that will happen is a confirmatory test or someone will tell you that the child you admitted did not have “x”, “y” or “z”. If you have an inconclusive or “negative” ultrasound (e.g., you did not see free fluid on a FAST exam, target sign of intussusception, or a big fat >6 mm appendicitis that hurts the patient on sono-palpation, etc.), take a step back, reassess the clinical picture, and get that confirmatory study or continue to observe your pediatric patient. An unaddressed area of research is the effect of serial ultrasound exams on overall diagnostic accuracy (either by same operator or by a different one) as an advantage of point-of-care ultrasound is that it is easily repeatable with abnormal findings highly reproducible. Most likely, with an abnormal or positive ultrasound result—you have probably clinched the diagnosis with a sufficiently high post-test probability that allows you to move forward with further evaluation if necessary, treatment and then onto your next emergency pediatric patient.

It is recognized that we need to minimize the amount of ionizing radiation that children receive from imaging studies [15], and with ongoing research, we can be hopeful that point-of-care ultrasound may have a role in this. In the coming years, we will likely see research studies that will investigate not just the accuracy of clinician-performed point-of-care ultrasound to diagnose various pediatric abdominal emergencies and other pathologies, but also to identify where preventable diagnostic errors may arise and to develop strategies to avoid them—whether it involves standardized scanning techniques, improved ultrasound machine functionality, or operator training factors. Just as road-traffic safety is an important discipline that has led to use of air-bags, car seats for infants, and seat-belt laws, efforts to study diagnostic errors related to point-of-care ultrasound use can help us avoid misdiagnosis-related harm in our patients. Bottom line: in general, think of point-of-care ultrasound as a “rule-in” diagnostic test and not a tool to “rule-out” disease.

Declarations

Conflict of interest

None.

Authors’ Affiliations

(1)
Departments of Emergency Medicine and Pediatrics, Mount Sinai School of Medicine of New York University

References

  1. Ishrak O (2009) Ultrasound—a revolution in the making. Crit Ultrasound J 1(1):27–32View ArticleGoogle Scholar
  2. Elbarbary M (2009) Evidence-based critical ultrasound: a mission possible. Crit Ultrasound J 1(1):3–4View ArticleGoogle Scholar
  3. Newman-Toker DE, Pronovost PJ (2009) Diagnostic errors—the next frontier for patient safety. JAMA 301(10):1060–1062PubMedView ArticleGoogle Scholar
  4. Holmes JF, Brant WE, Bond WF et al (2001) ED ultrasonography in the evaluation of hypotensive and normotensive children with blunt abdominal trauma. J Pediatr Surg 36:968–973PubMedView ArticleGoogle Scholar
  5. Holmes JF, Gladman JF, Chang CH (2007) Performance of abdominal ultrasonography in pediatric blunt trauma patients: a meta-analysis. J Pediatr Surg 42(9):1588–1594PubMedView ArticleGoogle Scholar
  6. Godbole P, Sprigg A, Dickson JA, Lin PC (1996) Ultrasound compared with clinical examination in infantile hypertrophic pyloric stenosis. Arch Dis Child 75(4):335–337PubMedPubMed CentralView ArticleGoogle Scholar
  7. Malcom GE 3rd, Raio CC, Del Rios M et al (2009) Feasibility of emergency physician diagnosis of hypertrophic pyloric stenosis using point-of-care ultrasound: a multi-center case series. J Emerg Med 37(3):283–286PubMedView ArticleGoogle Scholar
  8. McVay MR, Copeland DR, McMahon LE et al (2009) Surgeon-performed ultrasound for diagnosis of pyloric stenosis is accurate, reproducible and clinically valuable. J Pediatr Surg 44(1):169–171PubMedView ArticleGoogle Scholar
  9. Kairam N, Kaiafis C, Shih R (2009) Diagnosis of pediatric intussusceptions by an emergency physician-performed bedside ultrasound: a case report. Pediatr Emerg Care 25(3):177–180PubMedView ArticleGoogle Scholar
  10. Eshed I, Gorenstein A, Serour F et al (2004) Intussusception in children: can we rely on screening sonography performed by junior residents? Pediatr Radiol 34(2):134–137PubMedView ArticleGoogle Scholar
  11. Garcia Pena BM, Mandl KD, Kraus SJ et al (1999) Ultrasonography and limited computed tomography in the diagnosis and management of appendicitis in children. JAMA 282(11):1041–1046PubMedView ArticleGoogle Scholar
  12. Fox JC, Solley M, Anderson CL et al (2008) Prospective evaluation of emergency physician performed bedside ultrasound to detect acute appendicitis. Eur J Emerg Med 15(2):80–85PubMedView ArticleGoogle Scholar
  13. Sackett DL, Straus SE, Richardson WS et al (2000) Evidence-based medicine: how to practice and teach EBM, 2nd edn. Churchill Livingstone, UK, p 74Google Scholar
  14. Guyatt G, Rennie D, Meade MO et al (2008) Users’ guides to the medical literature: a manual for evidence-based clinical practice, 2nd edn. McGraw-Hill Companies, Inc., USAGoogle Scholar
  15. Hall EJ, Brenner DJ (2008) Cancer risks from diagnostic radiology. Br J Radiol 81(965):362–378PubMedView ArticleGoogle Scholar

Copyright

© Springer-Verlag 2010