Accuracy of aortic annular measurements obtained from three-dimensional echocardiography, CT and MRI: human in vitro and in vivo studies
- Wendy Tsang1,
- Michael G Bateman2,
- Lynn Weinert1,
- Gian Pellegrini3,
- Victor Mor-Avi1,
- Lissa Sugeng4,
- Hubert Yeung3,
- Amit R Patel1,
- Alexander J Hill3,
- Paul A Iaizzo2,
- Roberto M Lang1
- 1University of Chicago Medical Center, Chicago, Illinois, USA
- 2Department of Surgery, Visible Heart Lab, University of Minnesota, Minneapolis, Minnesota, USA
- 3Medtronic, Inc., Minneapolis, Minnesota, USA
- 4Yale University, New Haven, Connecticut, USA
- Correspondence to Dr Roberto M Lang, Department of Cardiology, University of Chicago, D5841 South Maryland Avenue, MC 5084, Chicago, Illinois 60637, USA;
Contributors All authors contributed equally to the design, implementation and writing of the paper.
- Accepted 15 May 2012
Objectives To determine the accuracy of calcium-containing rings measurements imaged by three-dimensional echocardiography (3DE), multi-slice CT (MSCT) and cardiac magnetic resonance (CMR) under ideal conditions against the true ring dimensions. To compare the accuracy of aortic annulus (AoA) measurements in ex vivo human hearts using 3DE, MSCT and CMR. To determine the accuracy of AoA measurements in an in vivo human model.
Design 3DE, MSCT and CMR imaging were performed on 30 calcium-containing rings and 28 explanted human hearts. Additionally, 15 human subjects with clinical indication for MSCT underwent 3DE. Two experts in each modality measured the images.
Main outcome measures Bias and intraclass correlation coefficient for accuracy of imaging measurements when compared with actual ring dimensions. Bias, intraclass correlation coefficient and variability were obtained: (1) when comparing explanted human heart AoA measurements from the two remaining imaging modalities with the most accurate one as determined from the ring measurements and (2) in in vivo human AoA measurements. Analysis was repeated on explanted heart subgroups divided by aortic valve Agatston score.
Results Against the known ring dimensions, CMR had the highest accuracy and the lowest variability. MSCT measurements had high accuracy but wider variability and 3DE had the lowest accuracy with the largest variability. When 3DE and MSCT were compared with CMR, 3DE underestimated and MSCT overestimated AoA dimensions, but inter-measurement variability of 3DE and MSCT were similar. When divided by Agatston score, both 3DE and MSCT measurements were larger and showed greater variability with increasing calcium burden. The in vivo study showed that the correlation between 3DE and MSCT measurements was high; however, 3DE measurements were smaller than those measured with MSCT.
Conclusions In the in vitro model, CMR measurements were the most accurate for assessing the actual dimensions suggesting that further investigations on its role in AoA measurement in TAVR are needed. However from the in vivo model, MSCT and 3DE are reasonable alternatives with the understanding that they can slightly overestimate and underestimate annular dimensions, respectively.
- Three-dimensional echocardiography
- cardiac magnetic resonance
- aortic annulus
- imaging and diagnostics
- valvular disease
- diagnostic imaging
- systolic function
- CT scanning
- myocardial viability
- myocardial perfusion
- atrial fibrillation
- cardiac function
- cardiac remodelling
- coronary physiology
- coronary haemodynamics
- cardiac surgery
- echocardiography (transoesophageal)
- contrast media
- exercise testing
Degenerative aortic stenosis is the most common valve disease in the USA.1 ,2 Traditionally, surgical aortic valve replacement was considered the ‘gold standard’ in treating this disease and improving prognosis. However, approximately 30% of patients with severe, symptomatic aortic stenosis are considered inoperable due to comorbid conditions. Transcatheter aortic valve replacement (TAVR) is a new, less invasive treatment for aortic stenosis offered to this patient group. Since TAVR was first performed in 2002, more than 20 000 implantations have been completed to date.3 ,4
While TAVR is a less invasive procedure than surgical valve replacement, proper performance relies heavily on accurate quantification of the aortic annulus (AoA) dimensions. Without this information, difficulties may be encountered during prosthesis implantation due to incorrect sizing. As well, postprocedural complications related to incorrect sizing, such as aortic regurgitation, are associated with poor prognosis.5 ,6 To date, fluoroscopy and two-dimensional (2D) transthoracic and transoesophageal echocardiography (TEE) have been primarily used to plan and guide TAVR procedures. However, these imaging modalities are limited in that they display three-dimensional (3D) structures in two dimensions. In contrast, real-time 3D echocardiography (3DE), multi-slice CT (MSCT) and cardiac magnetic resonance (CMR) are all able to provide 3D or multi-slice images of the aortic root. Accordingly, these imaging modalities have become more commonly used for TAVR procedure planning and guidance. However, thorough comparative studies between these imaging modalities are scarce and/or the results often rely on inter-modality correlations of measurements rather than accuracy against a true gold standard.7–15
The primary goals of this study were: (1) to determine the accuracy of calcium-containing rings measurements imaged by 3DE, MSCT and CMR under ideal conditions against the true ring dimensions; (2) to compare the accuracy of AoA measurements in ex vivo human hearts using 3DE, MSCT and CMR; and (3) to compare the accuracy of AoA measurements in living humans imaged using standard 3DE and MSCT protocols.
To compare measurements against a gold standard, we first employed an in vitro approach in which we simulated calcified aortic valves by implanting constructed calcium rings in aortas. We hypothesised that this model would be useful to determine the most accurate imaging modality for performing ring measurements in the presence of calcium under ideal imaging conditions. We also imaged ex vivo human hearts fixed in gel to examine AoA dimension differences between imaging modalities. The imaging modality that was found to be most accurate when compared with actual ring dimensions was used as the gold standard for AoA dimension comparison. Having determined the most accurate imaging modality under ideal imaging conditions, we then imaged the AoA in patients to determine inter-modality differences.
In vitro studies
We used 30 specially constructed calcium rings that were precision-machined elliptical annuli of solid water phantom material (Gammex 457; Gammex, Middleton, Wisconsin, USA) and implanted into ex vivo aortas. They ranged from 19 to 37 mm major outer diameters with eccentricity of 0.57 (ie, major to minor outer diameter ratio of 1.22). The width and thickness of the annuli were held constant at 5 mm. Three channels (3 mm wide and 2 mm deep), arranged along the ellipse of the annuli, were filled with calcium hydroxyapatite (Ca10(PO4)6(OH)2) and held in place with cyanoacrylate (Loctite, Henkel Corporation, Rocky Hill, Connecticut, USA) and UV adhesives (Dymax; Dymax Corporation, Torrington, Connecticut, USA, figure 1A). The manufactured rings were first measured when dry before implantation into the tissues, and then after explantation while still wet to ensure that the rings retained their measurement sizes. No differences in the sizes of the rings under either condition were noted.
For AoA measurements, we imaged the aortas of 28 human cadaveric hearts with varying degrees of aortic valve calcification. The Research Ethics Boards of the University of Chicago and the University of Minnesota approved this study.
The fresh heart specimens were obtained from: (1) organ donors whose hearts were not deemed viable for transplantation and were donated for research (via LifeSource Upper Midwest Organ Procurement Organisation) or (2) from bodies donated to the University of Minnesota's Anatomy Bequest Program. This study was reviewed and approved by these organisations prior to the completion of the imaging protocols. After excision, the fresh/unfixed specimens were cleaned and perfusion-fixed in 10% buffered formalin, by attaching the cannulated great vessels to a system, which provided a pressure head of approximately 50 mm Hg. This methodology fixes a given heart in end-diastole.16 Subsequently, the hearts, saturated with formalin, were placed in 12”×7”×6” sealable polymer containers and submerged in 0.7% agar gel at approximately 45°C. The hearts were then palpated to ensure that all possible air was ejected from the chambers and major vessels to reduce any air–tissue susceptibility artefacts, before being suspended in an anatomically correct position (figure 1B). Once cooled to room temperature (∼23°C), all supports used to suspend the hearts were removed and any remaining air volume within the container was displaced with gel.
Image acquisition and formats
All in vitro imaging was performed on each heart within a 24-h time frame using the same imaging methods for the rings and the fixed human hearts. 3DE was performed using an iE33 imaging system (Philips Healthcare, Andover, Massachusetts, USA), equipped with a fully sampled matrix transthoracic echocardiography (TTE) transducer (S5-1). The top of the specimen-containing box was removed and the probe was placed on the gel surface. Slight pressure was applied to the gel when necessary to improve image quality, while minimising disruption to the orientation of the specimen.
Initially, gain settings were optimised using a narrow-angled imaging mode, which allows acquisition of a 3DE pyramidal volume of approximately 30×60°. Zoomed 3DE images of the entire ring were then acquired with frame rates ranging between 5 and 28 Hz (mean 17±5 Hz). Acquisition of 3D datasets was repeated three times to optimise image quality for analysis. All 3DE images were reviewed offline using commercial software (3DQ, QLAB; Philips Healthcare, Andover, Massachusetts, USA) by a trained cardiologist. Three orthogonal views of the ring were used to extract a planar en-face view. This image, as well as a planar image of the AoA from each heart, was used for measurements (figure 1F–H).
CMR was performed using a 3T system (Philips Healthcare, Best, The Netherlands) with a SENSE head coil with eight elements, which could fit around the box containing the gel-fixed heart. Spin-echo sequences were used to identify the ring or aorta. T1-weighted 3D ultrafast gradient echo images of the entire fabricated rings and aorta were acquired and exported in DICOM format to an independent workstation and reviewed by an experienced CMR reader using OsiriX software (V.3.8.1; Pixmeo Sarl, Geneva, Switzerland). Images featuring a planar en-face view of a given ring and one image of each heart's AoA were exported for measurements (figure 1L–N).
MSCT scans were performed using a 256-slice Brilliance iCT (Philips Healthcare, Cleveland, Ohio, USA) with prospective gating using a simulated ECG signal and with collimation 96×0.625 mm, rotation time 270 ms, tube voltage 120 kV and tube current 360 MA. The image slice thickness was 0.9 mm as was the reconstruction thicknesses. Datasets were then transferred to a remote workstation (Extended Brilliance Workspace; Philips) where an experienced CT reader reviewed the stacked images and exported two images for measurement: one image featuring a planar view of the ring, and one image featuring a planar view of a given AoA (figure 1I–K). The AoA was defined as the plane at the point of leaflet insertion.
In vitro study design
To determine the most accurate imaging modality against a true gold standard, six independent observers, two experts for each modality (3DE, CMR and CT), performed measurements on the ring images in their respective imaging modality. To minimise the variability introduced with the selection of the imaging planes, the interpreters were provided with fixed images, which could not be manipulated. For each imaging modality, two orthogonal views used to ensure that true, centred, en-face views of the rings as well as the AoA were obtained. A schematic demonstrating the long- and short-axes measurements were provided as a guide for the interpreters (figure 1C–E). For each imaging modality, six measurements were performed on each ring image: area, long- and short-axes measurements of the internal and external dimension of a given ring (figure 1F, G, I, J, L, M).
Ring image quality was assessed using a three-grade scale based on the visualisation of the inner and outer ring edges: ‘0’ if two or more inner or outer ring edges were poorly visualised; ‘1’ if only one ring edge was poor visualised; and ‘2’ if there were no visualisation limitations.
For the in vitro human heart images, AoA measurements from the remaining two imaging modalities were compared with the most accurate modality as determined from the first part of the study. Again, the same six independent observers performed measurements on the AoA images in their respective specialties. For each imaging modality, long- and short-axes measurements were obtained (figure 1H, K, N).
To assess the impact of aortic valve calcification, the hearts were divided into three groups based on their aortic valve Agatston score. The Agatston score was calculated from the non-contrast CT images using OsiriX software with a detection threshold of 90 HU. Calcification was attributed to the aortic valve if it was clearly visible within the valve cusps. Supravalvular calcifications and calcification of the coronary arteries including the ostia were not included. The aortic valve Agatston score were used to divide the hearts into three groups: (1) no calcium with a zero score; (2) mild-to-moderate calcium with a score between 1 and 500 HU; and (3) severe calcium with a score >500 HU.
In vivo study
Image acquisition and formats
MSCT and 3DE used in the in vivo part of the study were performed within a 24-h time frame on 16 patients (60±13 years, six male subjects) referred for clinically indicated MSCT. Transthoracic 3DE images were acquired from an apical window using the iE33 imaging system (Philips, Andover, Massachusetts, USA) with a matrix array transducer (X3-1). Initially, gain settings were optimised using a narrow-angled imaging mode, which allows acquisition of a 3DE pyramidal volume of approximately 30×60°. Zoomed 3DE images from the left ventricular outflow tract to the ascending aorta were then acquired. Care was taken to ensure that the AoA was placed in the middle of the sector. Full-volume acquisition was performed using ECG gating over four consecutive cardiac cycles. Images were reviewed immediately to determine whether the aortic root was well visualised. Identical to the process described above, a single en-face view of the AoA was exported for measurement.
MSCT scans were performed using standard clinical protocols on a 16-slice multidetector scanner (Toshiba, Otawara, Japan). Non-ionic iodinated contrast agent (Ultravist-370, Schering, Berlin, Germany) was injected into the antecubital vein (140 ml, 3.5 ml/s) and followed by a 50-ml saline bolus. Image acquisition was triggered by the appearance of contrast in the aortic root. Imaging parameters included 250 ms gantry rotation time with 5 mm per rotation, and tube voltage of 120 kV with currents of 300 mA. Scan data were then reconstructed at 0.5 mm slice thickness and 0.5 mm in-slice resolution using retrospective ECG-gating from early systole (0% of the RR interval) to late diastole (90% of the RR interval) at 10% steps. β-Blockers were not given during the CT acquisition protocol.
In vivo study design
AoA measurements from MSCT and 3DE were compared. Four independent observers performed measurements on the AoA images in their respective specialties. For each imaging modality, long- and short-axes measurements were obtained.
For the ring analysis, inner ring long- and short-axes measurements were grouped together and the same was done for the outer ring long- and short-axes measurements. For the in vitro AoA measurements, the long- and short-axes were grouped together. Bland–Altman analysis was used to compare measurements obtained with 3DE, MSCT and CMR to the actual ring dimensions and between imaging modalities for the in vitro human AoA measurements. The gold standard ring area was calculated from the known ring dimensions using the formula: area=(π*length*width)/4. To estimate measurement errors between imaging modalities, intraclass correlation coefficients were calculated for comparisons among 3DE, MSCT and CMR measurements against actual ring dimensions. Intraobserver variability was not assessed, since the 3DE images could not be fixed for repeated measurements. The reproducibility of in vitro AoA measurements was evaluated using intraclass correlation and the coefficient of variation, calculated as the absolute difference between the corresponding pairs of measurements in per cent of their mean. The Bland–Altman and intraclass correlation analyses were repeated for the three subgroups determined by the aortic valve Agatston score.
After performing MSCT and 3DE in living humans, the reproducibility of AoA measurements was evaluated using intraclass correlation and the coefficient of variation, calculated as the absolute difference between the corresponding pairs of measurements in per cent of their mean. Comparison between the imaging modalities was performed with Bland–Altman analysis and intraclass correlation coefficient.
Imaging of all 30 implanted rings was possible with MSCT, CMR and 3DE. For MSCT and CMR, image quality was graded ‘2’ in all 30 rings. With 3DE, 17 (57%) ring images received a score of ‘2’, four (13%) received a score of ‘1’ and nine (30%) received a score of ‘0’. Poor visualisation scores were due to acoustic shadowing caused by the calcified portions of the ring.
When assessing the accuracy of the measurements obtained with 3DE, CMR and MSCT compared with the known ring dimensions and area, MSCT and CMR were found to be highly accurate with high correlation coefficients (table 1) and near zero biases. However, the limits of agreement were slightly wider for MSCT compared with CMR. This pattern remained regardless of whether the inner or outer ring dimensions or area were measured. In contrast, 3DE had slightly lower correlation coefficients, and larger bias with wider limits of agreement (table 1).
In vitro AoA measurement analysis
Imaging of the cadaveric AoA was feasible with MSCT, CMR and 3DE in all the 28 hearts examined. Table 2 summarises the baseline characteristics of the hearts measured. Given that CMR was determined in the ring experiment to have the highest accuracies and best reproducibility, this modality was used as the reference standard for subsequent comparisons of AoA measurements. When comparing AoA measurements derived from MSCT images against this reference standard, MSCT measurements were noted to slightly overestimate the AoA dimensions with relatively narrow limits of agreement (table 2). In contrast, AoA measured from 3DE images resulted in a slight underestimation of annular dimensions with wider limits of agreement (table 2).
When the inter-measurement reliability was examined, CMR had the highest intraclass correlation values with the narrowest percentage variability (table 2). Repeated measurements showed that compared with CMR, 3DE and MSCT had lower correlation coefficients and wider variability. However, they had similar intraclass correlation values and percentage variability when compared with each other (table 2).
When the AoA measurements were divided according to their aortic valve Agatston score, 3DE and MSCT intraclass correlation decreased with increasing valve calcification (table 2). CMR had the highest ICC values across the groups. When the 3DE and MSCT measurements were compared with CMR, increasing valve calcium scores were associated with an increase in measured dimensions.
In vivo AoA measurement analysis
Imaging of the AoA was feasible with MSCT and 3DE in all 15 human subjects (figure 1O, P). 3DE resulted in smaller measurements compared with MSCT (table 3). A high intraclass correlation was observed between 3DE and MSCT. When inter-measurement reliability was examined, MSCT had a higher intraclass correlation value than 3DE (table 3). While the mean percentage variability was higher for MSCT compared with 3DE, 3DE had wider standard deviations.
TAVR is a recently developed technique that is offered as an alternative to surgical aortic valve replacement to patients with severe aortic stenosis who are considered to be a high surgical risk.3 ,11 ,14 ,17 Accurate AoA measurement is critical for patient selection and successful valve implantation. Currently, 2D TTE and TEE have been the primary methods for determining AoA measurements for TAVR. Previous studies comparing these two imaging modalities have found that AoA measurements using TEE were typically larger than transthoracic measurements and the correlation with surgical valve sizing was good but not excellent as the measured annular sizes were not completely predictive of implanted valve size chosen intraoperatively by surgical valve sizers.9 ,11 ,18 In comparison, studies using MSCT report that AoA areas were typically larger compared with 2D TTE, 2D TEE or 3D TEE.7 ,10 ,11 ,13 ,15 This again is in contrast to CMR where sagittal AoA measurements have been reported to have similar dimensions when compared with 2D TEE, whereas coronal AoA CMR measurements are significantly different.19 ,20 Some of the differences reported with 2D TTE/TEE can be attributed to the underappreciation of the elliptical AoA shape. However, the major limitation of these studies is that they lacked an absolute gold standard reference, such as the one achieved in this study by the use of phantom imaging to determine the true accuracy of each modality.
In the present study, we used a unique in vitro model stimulating a calcified AoA to assess the true accuracy of 3DE, CMR and MSCT based measurements of a calcified AoA. This in vitro model allows multiple sources of measurement variability to be controlled to help improve the understanding of the differences in accuracy and the factors involved in variability between imaging modalities.
With 3DE, poorer image quality, caused by shadowing of the calcium ring, was noted in 30% of the ring images. Similar to clinical practice observations, imaging calcified aortic annuli or in our case fabricated calcium rings using CMR or MSCT did not result in artefacts, which can considerably affect measurement accuracy. In contrast, the poorer ring image quality caused by shadowing noted with 3DE resulted in lower accuracy, when compared with measurements obtained from MSCT or CMR images. These finding were consistent with both ring diameter measurements as well as area measurements. Measurements from MSCT and CMR were more accurate and highly reproducible. Given that calcium ring measurements obtained with CMR displayed the highest accuracy and the lowest variability, this modality was chosen as the imaging reference standard for assessing AoA in this study.
When doing so, we found that 3DE measurements underestimated while MSCT overestimated AoA measurements. However, the intrameasurement variability for 3DE and MSCT was similar. These findings are consistent with published studies comparing 2D TTE, 2D TEE and MSCT AoA dimensions in TAVR planning and are likely related to beam-hardening.7 ,11
When we examined our results according to the amount of aortic valve calcification, we noted that AoA measurement variability on 3DE and MSCT but not CMR augmented with increasing amounts of calcium. When 3DE and MSCT AoA measurements were compared with CMR, as the magnitude of calcium increased, the measured dimensions increased as reflected by increasing positive biases. This suggests that the presence of significant amounts of calcium led observers to make larger measurements.
From our in vivo model, we found that there was a good correlation between 3DE and MSCT measurements; however, 3DE measurements were slightly smaller than those obtained by MSCT and had wider variability.
It has previously been argued that MSCT AoA measurements are not equivalent to those from 2D TEE or 2D TTE because the imaging planes from which these measurements are obtained are more accurately identified on MSCT due to the ability to manipulate these images in 3D.21 However, with appropriate acquisition protocols and new postprocessing software available to manipulate CMR and 3DE images, accurate measuring planes can now be obtained with these two modalities. We have found in our in vitro model that CMR and 3DE have similar if not smaller AoA intrameasurement variability and can be performed without the radiation or iodine exposure required by MSCT imaging. Overall, the potential role of CMR in evaluating the aortic root and annulus in TAVR patients has been underappreciated. This study has demonstrated that for our in vitro model, even in the presence of calcium, CMR was the most accurate imaging modality with the highest level of reproducibility and should therefore be considered a reasonable option for AoA dimension assessments in TAVR patients. However, interpreters should be aware of the limitations on analysis of CMR images due to the imaging protocol used, patient breath-holding abilities and arrhythmias.
Current clinical guidelines for the imaging-based assessment of the AoA in TAVR patients state that TEE is the preferred tool for the assessment of LV outflow tract morphology and aortic root dimensions prior to device implantation.14 As determined in our AoA substudy, transthoracic 3DE is a reasonable option along with MSCT to obtain AoA measurements. Of interest, a recent study found that using MSCT to determine AoA size would have changed TAVR strategy in 40%–42% of patients because of the 1–2 mm greater size measurements obtained with MSCT compared with 2D TTE or TEE.11 However, implantation based on 2D TEE measurements in this same study was successful in 33 of the 34 treated patients.11 When we examine these findings in the context of our results, MSCT is likely overestimating the true annular dimensions due to beam-hardening. Nevertheless, despite the underestimation of measurements obtained from echocardiography, ultimately there was no difference in outcomes, perhaps because the guidelines for determining aortic device size were developed using echocardiographic measurements.21 Overall, image quality should be considered in determining reliability of measurements from different imaging modalities.
Our in vitro model has the inherent limitation that it lacks cardiac pulsations or respiratory motion during image acquisition. These are usually the major sources of artefacts in cardiac imaging that contribute to the variability encountered in measurements across imaging modalities. In planning this study, we sought to image the AoA phantom under ideal conditions in order to minimise the sources of variability and determine the highest possible accuracy of each imaging modality. Thus, while the 3DE and MSCT imaging protocols are similar to those used clinically, the CMR protocol used in our study is not feasible in a beating heart. As well, the ring size tolerances from manufacturing should not be considered as a confounding factor in the image measurement variability as they were below the imaging resolution of the utilised methodologies. Another limitation is the lack of contrast use on the MSCT images, which would be standard in clinical practice. This did not affect our results as MSCT had a near zero bias with excellent reproducibility. Last, while schematics of the rings were provided to the interpreters to define the long- and short-axes measurements, the interpreters may still not have identified the true centre resulting in overestimation of the rings and AoA dimensions. In addition to this, greater variability in the AoA measurements could have been caused by heart anatomy distortion during fixation in the gel.
In summary, we found that for our in vitro model, CMR was the most accurate method for assessing the implanted calcium ring phantoms. When CMR was used as the reference standard in human hearts, MSCT and 3DE were found to be reasonable alternatives, particularly when recognising that these modalities can slightly overestimate and underestimate annular dimensions, respectively. As well, increasing calcium burden leads to greater measurement variability and an absolute increase in measurements. From our in vivo model, agreement between MSCT and 3DE was good although 3DE had slightly smaller measurements and wider variability.
We would like to thank the organ donors and their families for their generous gifts of these specimens for research. We would also like to thank LifeSource for their support in recovering and transporting these specimens to the Visible Heart laboratory.
Funding Dr Tsang is funded through a CIHR research fellowship grant. This study was funded in part by a research grant from Medtronic, Minnesota, USA.
Competing interests None.
Ethics approval Ethics approval was provided by the University of Chicago and the University of Minnesota.
Provenance and peer review Not commissioned; externally peer reviewed.