OBJECTIVE PERFORMANCE TESTING AND QUALITY ASSURANCE OF MEDICAL ULTRASOUND EQUIPMENTINTRODUCTIONTEST OBJECTSEQUIPMENT AND EXPERIMENTSEquipmentSoftwareExperimentsIMAGING QUALITYStandard equipment settingsMonitorTransmission output levelPreprocessing controlsTime-gain-compensationTransducer frequencyPostprocessing (LUT)Slice thickness/elevation focusContrast resolution and echo level dynamic rangeImage contrast sensitivitySystem sensitivity/penetration depthLateral sensitivity profile (linear array)Dead zone estimationSpatial resolution (2D in-plane point-spread-function, IP-PSF)Geometric conformityRESULTSDISCUSSION AND CONCLUSIONSREFERENCESAPPENDIX AAPPENDIX Bdoi:10.1016/j.ultrasmedbio.2006.09.006● Original ContributionOBJECTIVE PERFORMANCE TESTING AND QUALITY ASSURANCE OFMEDICAL ULTRASOUND EQUIPMENTJOHAN M. THIJSSEN,GERT WEIJERS, and CHRIS L. DE KORTEClinical Physics Laboratory, University Children’s Hospital, Radboud University Nijmegen Medical Center,Nijmegen, The Netherlands(Received 24 May 2006; revised 4 September 2006; in final form 19 September 2006)Abstract—There is an urgent need for a measurement protocol and software analysis for objective testing of theimaging performance of medical ultrasound equipment from a user’s point of view. Methods for testing ofimaging performance were developed. Simple test objects were used, which have a long life expectancy. First, theelevational focus (slice thickness) of the transducer was estimated and the in-plane transmit focus was positionedat the same depth. Next, the postprocessing look-up-table (LUT) was measured and linearized. The testsperformed were echo level dynamic range (dB), contrast resolution (i.e., gamma of display, number of graylevels/dB) and sensitivity, overall system sensitivity, lateral sensitivity profile, dead zone, spatial resolution andgeometric conformity of display. The concept of a computational observer was used to define the lesionsignal-to-noise ratio, SNRL(or Mahalanobis distance), as a measure for contrast sensitivity. All the measure-ments were made using digitized images and quantified by objective means, i.e., by image analysis. The wholeperformance measurement protocol, as well as the quantitative measurements, have been implemented insoftware. An extensive data-base browser was implemented from which analysis of the images can be started andreports generated. These reports contain all the information about the measurements, such as graphs, images andnumbers. The approach of calibrating the gamma by using a linearized LUT was validated by processingsimultaneously acquired rf data. The contrast resolution and echo level of the rf data had to be compressed bya factor of two and amplified by a gain factor corresponding to 12 dB. This resulted in contrast curves that werepractically identical to those obtained from DICOM image data. The effects of changing the transducer centerfrequency on the spatial resolution and contrast sensitivity were estimated to illustrate the practical usefulnessof the developed approach of quality assurance by measuring objective performance characteristics. Thedeveloped methods might be considered as a minimum set of objective quality assurance measures. This set mightbe used to predict clinical performance of medical ultrasound equipment, taking into account the performanceat a unique point in space i.e., the coinciding depths of the elevation and in-plane (azimuth) foci. Furthermore,it should be investigated whether the approach might be used to compare objectively various brands ofequipment and to evaluate the performance specifications given by the manufacturer. Last but not least, thedeveloped approach can be used to monitor, in a hospital environment, the medical ultrasound equipment duringits life cycle. The software package may be viewed and downloaded at the website http://www.qa4us.eu. (E-mail:[email protected]) © 2007 World Federation for Ultrasound in Medicine & Biology.Key Words: Computational observer, Contrast resolution, Echography, Image quality, Medical ultrasound,Objective assessment, Performance testing, Quality assurance, Spatial resolution, QA4US®.INTRODUCTIONThe first efforts to develop performance methods formedical ultrasound equipment by various national andinternational committees date back 30 y [American In-stitute of Ultrasound in Medicine (AIUM 1974), theAmerican Association of Physicists in Medicine (AAPM;Carson and Zagzebski 1977) and the International Elec-trotechnical Commission (IEC; Hill 1977)]. Progress hasbeen slow for various reasons. Evolution of equipmentfeatures and performance is still very rapid and someambiguity seems to be present within these committeesin defining a clear endpoint; should it be a technicalstandard or a quality assessment from the user’s point ofview? Another limitation of the methods involved inquality standards, so far, has been the involvement ofsubjective assessments in case of image quality. Thislimitation also holds for a recent AAPM report (GoodsittAddress correspondence to: Professor Johan M. Thijssen, PhD,833-Clinical Physics Laboratory, University Children’s Hospital, Rad-boud University Nijmegen Medical Center, P.O. Box 9101, 6500 HBNijmegen, The Netherlands. E-mail: [email protected] in Med. & Biol., Vol. 33, No. 3, pp. 460– 471, 2007Copyright © 2007 World Federation for Ultrasound in Medicine & BiologyPrinted in the USA. All rights reserved0301-5629/07/$–see front matter460et al. 1998), although it has evidently been based on theendpoint of quality of use, i.e., imaging performance aswell as for the paper by Dudley et al. (2001); see alsoGibson et al. (2001). Although the recent paper byBrowne et al. (2004) is based on objective assessment, aserious limitation is the use of gray level rather thanrelative echo level (in dB), as was proposed by Thijssenet al. (2002) and van Wijk and Thijssen (2002), which isincorporated also in new IEC standards being developed.Therefore, the dB scale is used in this paper for theestimation of most of the quality measures: overall dy-namic range, contrast resolution, contrast sensitivity,spatial resolution and overall system sensitivity.In this article, the approach is followed that, ratherthan by a subjective assessment, the performance mea-surements are carried out on echographic images, whichare digitally acquired in a computer and recalibrated byusing the postprocessing look-up table (LUT) and bymeasuring the gray level of