Open Access

Estimating the accuracy of optic nerve sheath diameter measurement using a pocket-sized, handheld ultrasound on a simulation model

Critical Ultrasound Journal20168:18

https://doi.org/10.1186/s13089-016-0053-9

Received: 10 September 2016

Accepted: 3 November 2016

Published: 10 November 2016

Abstract

Background

Ultrasound measurement of optic nerve sheath diameter (ONSD) appears to be a promising, rapid, non-invasive bedside tool for identification of elevated intra-cranial pressure. With improvements in ultrasound technology, machines are becoming smaller; however, it is unclear if these ultra-portable handheld units have the resolution to make these measurements precisely. In this study, we estimate the accuracy of ONSD measurement in a pocket-sized ultrasound unit.

Methods

Utilizing a locally developed, previously validated model of the eye, ONSD was measured by two expert observers, three times with two machines and on five models with different optic nerve sheath sizes. A pocket ultrasound (Vscan, GE Healthcare) and a standard portable ultrasound (M-Turbo, SonoSite) were used to measure the models. Data was analyzed by Bland–Altman plot and intra-class correlation coefficient (ICC).

Results

The ICC between raters for the SonoSite was 0.878, and for the Vscan was 0.826. The between-machine agreement ICC was 0.752. Bland–Altman agreement analysis between the two ultrasound methods showed an even spread across the range of sheath sizes, and that the Vscan tended to read on average 0.33 mm higher than the SonoSite for each measurement, with a standard deviation of 0.65 mm.

Conclusions

Accurate ONSD measurement may be possible utilizing pocket-sized, handheld ultrasound devices despite their small screen size, lower resolution, and lower probe frequencies. Further study in human subjects is warranted for all newer handheld ultrasound models as they become available on the market.

Keywords

Point-of-care ultrasoundOptic nerve sheath diameterUltrasoundIntra-cranial pressureHandheld ultrasound

Background

Elevated intra-cranial pressure (ICP) is a frequent and grave complication of neurological injury secondary to conditions, such as traumatic brain injury, hydrocephalus, hepatic encephalopathy, and central nervous system infection. Close ICP monitoring is fundamental to the early detection and appropriate management [1]. Direct ventricular or intra-parenchymal monitoring is considered the gold standard for measuring ICP; however, the procedure is invasive, carries the risk of infection and bleeding, and is resource intensive [2, 3]. Other modalities for estimating ICP, such as lumbar puncture, computed tomography, or magnetic resonance imaging, have their own limitations, and can be impractical or dangerous in a variety of patient care settings [4]. A variety of non-invasive techniques utilizing ultrasound have been investigated including transcranial Doppler (TCD) ultrasonography and the pulsatility index calculated from its flow velocity waveform, and transorbital ultrasound measurement of optic nerve sheath diameter (ONSD). Both show promise in the emergency medicine and critical care literature for a variety of adult and pediatric patient populations [514]. Ultrasound is readily available, rapid, and non-invasive, making it a potentially excellent tool for monitoring the neurocritically ill.

With improvements in ultrasound technology, machines are becoming ultra-portable allowing for flexible point-of-care examinations, including at the scene of injury, in austere environments and during transport. Recent literature suggests that ONSD measurements by ultrasound show low inter- and intra-observer variability [6, 8, 9, 15, 16]. However, to diagnose increased ICP, ONSD measurements require precise measurement of a 3–6 mm structure to the nearest 0.1 mm accuracy [8, 17] and it is unclear if these ultra-portable handheld units have the resolution to make these precise measurements.

In this study, we estimate the accuracy of ONSD measurement in a pocket-sized ultrasound unit (Vscan, GE Healthcare) utilizing a locally developed model of the eye and optic nerve sheath, and compare it to previously validated portable unit (M-Turbo, SonoSite) as a reference standard.

Methods

Model

All measurements were made using our previously developed and validated optic nerve sheath models [16, 18] with varying sheath diameters of known size (simulated from 3-D printed discs), and conducted by expert point-of-care ultrasonographers (LG and GH).

Ultrasound technique

We used the standard ultrasound technique for measuring ONSD on our models as previously described [16, 18]. Both a handheld unit (Vscan with Dual Probe utilizing the linear transducer, GE healthcare, Little Chalfont, United Kingdom) and a conventional transducer using a 13–6 MHz linear array (L25x transducer with a SonoSite M-Turbo Ultrasound Machine, SonoSite Inc, Bothell, WA) were used. Sonographers measured each model with both machines (Fig. 1). The models were presented randomly, and the operators were blinded to the color and actual size of each disc. Trials were blocked by machine, where each operator made all of their SonoSite measurements consecutively, before making their Vscan readings.
Fig. 1

Ultrasound images of ONSD measurements utilizing the SonoSite (a) and Vscan (b) ultrasound units

Sample size

To compare the level of agreement between the two technologies, we powered our study by assuming a minimum Pearson correlation of 0.6, since less would likely preclude a useful conversion algorithm. Using the Fisher Z transformation with a significance level of 0.05 and a two-sided alternate hypothesis, we calculated that we required 24 measurements to be done with each ultrasound to achieve a 90% power. Therefore, we constructed five different models to be measured in triplicate by two expert operators with both machines for a total of 30 measurements per machine. The disc sizes measured 5.6 (light blue), 5.5 (pink), 6.9 (dark blue), 4.9 (black), and 3.8 (dark green) mm. Disc sizes were chosen to approximate the range of ONSD in patients with normal or elevated ICP found in vivo [6, 8].

Statistical analysis

Inter-rater reliabilities between operators and between machines were analyzed by intra-class correlation coefficient. Inter-rater reliability between machines (“inter-machine reliability”) was also examined by the Bland–Altman agreement analysis [19]. Finally, both a simple Pearson correlation and quadratic regression were used to compare measurements between machines.

Results

The ICC between raters for the SonoSite was 0.878, and that for the Vscan was 0.826. The between-machine agreement ICC was 0.752.

The Bland–Altman agreement analysis between the SonoSite and the Vscan is shown in Figs. 2 and 3. The Vscan measurements tended to be 0.33 mm higher than the SonoSite on average, with a standard deviation of their difference of 0.64 mm.
Fig. 2

Bland–Altman plot demonstrating the variability of SonoSite (black) and Vscan (red) measurements over the range of ONSD sizes

Fig. 3

Bland–Altman plot demonstrating the variability of SonoSite–Vscan measurements over the range of ONSD sizes

Simple Pearson correlation between machines demonstrated a correlation coefficient of 0.803. Using a quadratic regression, the correlation can be improved and a quadratic equation generated to predict the SonoSite measurement from the Vscan one (Fig. 4). The equation is:
Fig. 4

Quadratic regression model comparing VScan to SonoSite measurements

$$y \, = \, 8.54 \, - \, 2.86 \, x \, + \, 0.405 \, x^{2}$$
where x is the Vscan measurement and y represented the predicted measurement if using the SonoSite machine. After adjusting the Vscan data with this formula, the new mean difference is 0 ± 0.53 mm.

Discussion

It is currently unknown if pocket ultrasound devices have the resolution to make ONSD measurements as accurately as their larger counterparts. We attempted to assess the accuracy of ONSD measurement in an ultra-portable ultrasound (Vscan) and compare it to a conventional sized portable unit (SonoSite). This was achieved by utilizing our previously validated model, which evaluated intra- and inter-observer variability [16, 18]. We report that the ICC between raters was 0.826 for the Vscan and 0.878 for the SonoSite and the between-machine agreement ICC was 0.752.

Our high inter-rater agreement for both ultrasound devices is equivalent and larger than we have previously reported (ICC = 0.643) using our ONSD model with the same SonoSite device [16]. These improvements are likely due to our exclusive reliance on expert operators for our data this time, as our results are comparable to those found in the previous studies examining inter-rater reliability of experts in small populations in vivo [7, 15, 20, 21]. Although achieving competency in ONSD ultrasonography is not very arduous [22], errors in probe positioning and caliper placement can lead to incorrect measurement with novice operators.

The inter-machine agreement ICC between the SonoSite and the Vscan was 0.752. This is only slightly lower than the ICC between different observers for either machine implying an “excellent” agreement, and that the variation in measurements between the machines was only slightly greater than the variation in measurements between two observers using an identical machine. This suggests that a skilled operator may be able to use either machine to make ONSD measurements with a similar degree of accuracy as two operators using the same machine.

In the Bland–Altman analysis comparing level of agreement between Vscan and SonoSite measurements, there was an even spread across the range of disc sizes suggesting no systematic bias. It did find that the Vscan tended to over-estimate ONSD by a small amount (0.33 ± 0.65 mm). We hypothesize that this may be due to the screen size and resolution of the Vscan as compared to the SonoSite. Since the screen on the Vscan device was smaller than that of the SonoSite, attempts to position, the calipers on the edge of the optic nerve sheath may have lead to an overestimation. Importantly, this strong agreement between ONSD measurements by both devices allows us to create a quadratic conversion formula between the two measures. After adjusting the Vscan results using this formula, the mean difference shrinks to 0 ± 0.53 mm. A conversion formula could potentially be used to translate previously established normal reference ranges from the SonoSite into those more useful for the Vscan. This conversion formula requires further validation in a prospective study.

The advantage of our ONSD model is: it creates a controlled environment where the known disc size was consistent across different measurements and readers. This allows for an isolated comparison of measurement accuracy inherent to each technique and observer. On the surface, comparing the resolution of two ultrasound units to measure a fixed structure may seem like a simple task. The Vscan linear probe has a frequency range of 8–3.4 MHz which is significantly lower than the SonoSite’s 13–6 MHz probe. However, one must consider more than just sheer resolution of the unit, as ultrasound imaging is an operator dependent task that involves optic nerve sheath interpretation, caliper measurement, and ease of software and probe usage. Hence, the primary outcome of agreement between the two measurements is not only important, but also the secondary outcomes of inter-rater and within-subject reliability.

Pocket ultrasound units are not currently approved in Canada or the United States for the measurement of ONSD in human subjects as the existing units do not have the required presets to make these measurements safely. For this reason, we elected to perform this initial pilot study utilizing our ONSD model. The recommended settings for ONSD ultrasound in human subjects requires a thermal index (TI) ≤1 and mechanical index (MI) ≤0.23 [23] which are not achievable with the current Vscan firmware. We, therefore, do not encourage the “off-label” usage of these machines in live human subjects until the appropriate presets are made available.

Despite the importance of our findings, there are limitations to our study. First, all of the data comparing the relative accuracies of the ultrasound devices were obtained in a controlled setting on a simulation model. This was done to limit external variables and limit random error to more precisely study machine and observer effects between devices. However, it is difficult to extrapolate our findings to in vivo as the model does not necessarily account for effects, such as patient variation in anatomy or movement, fluctuations in ICP, and time constraints. Our model may itself introduce some random error of its own, such as imaging the 3D printed disc on an angle causing the disc to produce a larger shadow than usual, or that the shadow generated by the disc is not the same size as the disc itself. We also used a relatively small group of “experts”, as ultrasound evaluation of ONSD is still largely considered experimental in our centre, and there are relatively few experts to recruit. In addition, by having the sonographers both measure using the SonoSite first followed by the Vscan, we may have unintentionally introduced some bias into the measurements, however, looking at the Bland–Altman plot (Fig. 2), the similar variability around the mean for both machines would suggest if present this was likely inconsequential. Finally, our model simulates what is known as the “black stripe method” of ONSD measurement—meaning that the operator measures the edges of the internal diameter of the ONS. However, new quality criteria recently introduced [24] suggest that instead the external hypo-echogenic layers surrounding the optic nerve should be measured. To date, we have been unable to simulate this appearance in our model. This does not invalidate the comparison between ultrasound units are we are still measuring the same structure, but the authors wish to caution the reader that in vivo images may appear different and the ability of the units to clearly differentiate the borders of the ONS may be different in vivo as well.

Conclusions

This represents the first ONSD study utilizing a pocket-sized, handheld ultrasound device. Despite its small size, lower screen resolution, and significantly lower probe frequency, the ONSD measurements appear accurate and consistent when compared with the conventional portable units. Further study is needed to determine if pocket ultrasound can be accurately used in vivo to measure ONSD.

Abbreviations

ONSD: 

Optic nerve sheath diameter

ICP: 

Intra-cranial pressure

TI: 

Thermal index

MI: 

Mechanical index

ICC: 

Intra-class correlation coefficient

Declarations

Authors’ contributions

All authors were involved in study design, data analysis, and manuscript composition. LG, GJ, BU, and GH were involved in data collection. All authors read and approved the final manuscript.

Acknowledgements

This research was supported by a General Operating Grant from the Health Science Centre Foundation and a GFT Research Grant from the Department of Surgery, University of Manitoba, both in Winnipeg, Manitoba, Canada.

Competing interests

GE Healthcare provided a loaner Vscan unit for this study but had no involvement in the study design, data collection or interpretation, and manuscript composition.

Ethics approval and consent to participate

This study was deemed not to require research ethics board review by the University of Manitoba Health Research Ethics Board.

Funding

This research was supported by a General Operating Grant from the Health Science Centre Foundation and a GFT Research Grant from the Department of Surgery, University of Manitoba, both in Winnipeg, Manitoba, Canada.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Undergraduate Medical Education, University of Manitoba
(2)
Departments of Surgery, University of Manitoba
(3)
Clinician Investigator Program, University of Manitoba
(4)
Department of Internal Medicine, Section of Critical Care, University of Manitoba
(5)
Pediatrics and Child Health, University of Manitoba
(6)
Department of Critical Care, Keck Medical School, USC

References

  1. Pickard JD, Czosnyka M (1993) Management of raised intracranial pressure. J Neurol Neurosurg Psychiatry 56:845–858View ArticlePubMedPubMed CentralGoogle Scholar
  2. Bernuau J, Durand F (2006) Intracranial pressure monitoring in patients with acute liver failure: a questionable invasive surveillance. Hepatology 44:502–504. doi:10.1002/hep.21243 View ArticlePubMedGoogle Scholar
  3. Brain Trauma Foundation, American Association of Neurological Surgeons, Congress of Neurological Surgeons (2007) Guidelines for the management of severe traumatic brain injury. J Neurotrauma 24(Suppl 1):S1–S106Google Scholar
  4. Chmrc CFRN (2016) Optic nerve sheath diameter ultrasound and the diagnosis of increased intracranial pressure. Crit Care Nurs Clin N Am 28:95–99. doi:10.1016/j.cnc.2015.10.005 View ArticleGoogle Scholar
  5. Dubourg J, Javouhey E, Geeraerts T et al (2011) Ultrasonography of optic nerve sheath diameter for detection of raised intracranial pressure: a systematic review and meta-analysis. Intensiv Care Med 37:1059–1068. doi:10.1007/s00134-011-2224-2 View ArticleGoogle Scholar
  6. Bäuerle J, Lochner P, Kaps M, Nedelmann M (2010) Intra- and interobsever reliability of sonographic assessment of the optic nerve sheath diameter in healthy adults. J Neuroimaging 22:42–45. doi:10.1111/j.1552-6569.2010.00546.x View ArticlePubMedGoogle Scholar
  7. Moretti R, Pizzi B, Cassini F, Vivaldi N (2009) Reliability of optic nerve ultrasound for the evaluation of patients with spontaneous intracranial hemorrhage. Neurocrit Care 11:406–410. doi:10.1007/s12028-009-9250-8 View ArticlePubMedGoogle Scholar
  8. Goeres P, Zeiler FA, Unger B et al (2015) Ultrasound assessment of optic nerve sheath diameter in healthy volunteers. J Crit Care 31:168–171. doi:10.1016/j.jcrc.2015.10.009 View ArticlePubMedGoogle Scholar
  9. Ballantyne SA, O’Neill G, Hamilton R, Hollman AS (2002) Observer variation in the sonographic measurement of optic nerve sheath diameter in normal adults. Eur J Ultrasound 15:145–149View ArticlePubMedGoogle Scholar
  10. Bellner J, Romner B, Reinstrup P, Kristiansson KA, Ryding E, Brandt L (2004) Transcranial Doppler sonography pulsatility index (PI) reflects intracranial pressure (ICP). Surg Neurol 62:45–51 (discussion 51) View ArticlePubMedGoogle Scholar
  11. Brandi G, Béchir M, Sailer S, Haberthür C, Stocker R, Stover JF (2010) Transcranial color-coded duplex sonography allows to assess cerebral perfusion pressure noninvasively following severe traumatic brain injury. Acta Neurochir 152:965–972View ArticlePubMedGoogle Scholar
  12. Homburg AM, Jakobsen M, Enevoldsen E (1993) Transcranial Doppler recordings in raised intracranial pressure. Acta Neurol Scand 87:488–493View ArticlePubMedGoogle Scholar
  13. Bouzat P, Francony G, Declety P et al (2011) Transcranial Doppler to screen on admission patients with mild to moderate traumatic brain injury. Neurosurgery 68:1603–1609View ArticlePubMedGoogle Scholar
  14. Gura M, Elmaci I, Sari R, Coskun N (2011) Correlation of pulsatility index with intracranial pressure in traumatic brain injury. Turk Neurosurg 21:210–215PubMedGoogle Scholar
  15. Bäuerle J, Schuchardt F, Schroeder L et al (2013) Reproducibility and accuracy of optic nerve sheath diameter assessment using ultrasound compared to magnetic resonance imaging. BMC Neurol 13:187. doi:10.1186/1471-2377-13-187 View ArticlePubMedPubMed CentralGoogle Scholar
  16. Zeiler FA, Unger B, Zhu Q et al (2014) A unique model for ONSD part II: inter/intra-operator variability. Can J Neurol Sci 41:430–435View ArticlePubMedGoogle Scholar
  17. Asghar A, Hashmi M, Hussain A (2015) Optic nerve sheath diameter evaluated by transorbital sonography in healthy volunteers from Pakistan. Anaesth Pain Intensiv Care 19:282–286Google Scholar
  18. Zeiler FA, Unger B, Kramer AH et al (2013) A unique model for ultrasound assessment of optic nerve sheath diameter. Can J Neurol Sci 40:225–229View ArticlePubMedGoogle Scholar
  19. Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1:307–310Google Scholar
  20. Bäuerle J, Lochner P, Kaps M, Nedelmann M (2012) Intra- and interobserver reliability of sonographic assessment of the optic nerve sheath diameter in healthy adults. J Neuroimaging 22:42–45. doi:10.1111/j.1552-6569.2010.00546.x View ArticlePubMedGoogle Scholar
  21. Steinborn M, Fiegler J, Ruedisser K et al (2011) Measurement of the optic nerve sheath diameter in children: comparison between transbulbar sonography and magnetic resonance imaging. Ultraschall Med. doi:10.1055/s-0031-1273491 Google Scholar
  22. Potgieter DW, Kippin A, Ngu F, McKean C (2011) Can accurate ultrasonographic measurement of the optic nerve sheath diameter (a non-invasive measure of intracranial pressure) be taught to novice operators in a single training session? Anaesth Intensiv Care 39:95–100Google Scholar
  23. US Food and Drug Administration center for devices and Radiological Health (2008) Guidance for industry and FDA staff-information for manufacturers seeking marketing clearance of diagnostic ultrasound systems and transducers. US Department of Health and Human Services. Available at http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm070856.htm. Accessed 7 Nov 2016
  24. Sargsyan AE, Blaivas M, Geeraerts T, Karakitsos D (2014) “Ocular ultrasound in the intensive care unit” in critical care ultrasound. In: Lumb P, Karakitsos D (ed). Elsevier, AmsterdamGoogle Scholar

Copyright

© The Author(s) 2016