An approach to comparing accuracies of two FLAIR MR sequences in the detection of multiple sclerosis lesions in the brain in the absence of gold standard.
Publication/Presentation Date
6-1-2010
Abstract
RATIONALE AND OBJECTIVES: The purpose of this study was to present a new methodology to compare accuracies of two imaging fluid attenuated inversion recovery (FLAIR) magnetic resonance sequences in detection of multiple sclerosis (MS) lesions in the brain in the absence of ground truth, and to determine whether the two sequences, which differed only in echo time (TE), have the same accuracy.
MATERIALS AND METHODS: We acquired FLAIR images at TE(1) = 90 ms and TE(2) = 155 ms from 46 patients with MS (24-69 years old, mean 45.8, 15 males) and 11 healthy volunteers (23-54 years old, mean 37.1, 6 males). Seven experienced neuroradiologists segmented lesions manually on randomly presented corresponding TE(1) and TE(2) images. For every image pair, a "surrogate ground truth" for each TE was generated by applying probability thresholds, ranging from 0.3 to 0.5, to the weighted average of experts' segmentations. Jackknife alternative free-response receiver operating characteristic analysis was used to compare experts' performance on TE(1) and TE(2) images, using successively the TE(1)- and TE(2)-based ground truths.
RESULTS: Supratentorially, there were significant differences in relative accuracy between the two sequences, ranging from 8.4% to 12.1%. In addition, we found a higher ratio of false positives to true positives for the TE(2) sequence using the TE(2) ground truth, compared to the TE(1) equivalent. Infratentorially, differences in the relative accuracy did not reach statistical significance.
CONCLUSION: The presented methodology may be useful in assessing the value of new clinical imaging protocols or techniques in the context of replacing existing ones, when the absolute ground truth is not available, and in determining changes in disease progression in follow-up studies. Our results suggest that the sequence with shorter TE should be preferred because it generates relatively fewer false positives. The finding is consistent with results of previous computer simulation studies.
Volume
17
Issue
6
First Page
686
Last Page
695
ISSN
1878-4046
Published In/Presented At
Bilello, M., Suri, N., Krejza, J., Woo, J. H., Bagley, L. J., Mamourian, A. C., Vossough, A., Chen, J. Y., Millian, B. R., Mulderink, T., Markowitz, C. E., & Melhem, E. R. (2010). An approach to comparing accuracies of two FLAIR MR sequences in the detection of multiple sclerosis lesions in the brain in the absence of gold standard. Academic radiology, 17(6), 686–695. https://doi.org/10.1016/j.acra.2010.01.019
Disciplines
Business Administration, Management, and Operations | Health and Medical Administration | Management Sciences and Quantitative Methods
PubMedID
20457413
Department(s)
Administration and Leadership
Document Type
Article