Assessment of a quantitative metric for 4D CT artifact evaluation by observer consensus

Sarah J. Castillo, Richard Castillo, Peter Balter, Tinsu Pan, Geoffrey Ibbott, Brian Hobbs, Ying Yuan, Thomas Guerrero

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

The benefits of four-dimensional computed tomography (4D CT) are limited by the presence of artifacts that remain difficult to quantify. A correlation-based metric previously proposed for ciné 4D CT artifact identification was further validated as an independent artifact evaluator by using a novel qualitative assessment featuring a group of observers reaching a consensus decision on artifact location and magnitude. The consensus group evaluated ten ciné 4D CT scans for artifacts over each breathing phase of coronal lung views assuming one artifact per couch location. Each artifact was assigned a magnitude score of 1-5, 1 indicating lowest severity and 5 indicating highest severity. Consensus group results served as the ground truth for assessment of the correlation metric. The ten patients were split into two cohorts; cohort 1 generated an artifact identification threshold derived from receiver operating characteristic analysis using the Youden Index, while cohort 2 generated sensitivity and specificity values from application of the artifact threshold. The Pearson correlation coefficient was calculated between the correlation metric values and the consensus group scores for both cohorts. The average sensitivity and specificity values found with application of the artifact threshold were 0.703 and 0.476, respectively. The correlation coefficients of artifact magnitudes for cohort 1 and 2 were 0.80 and 0.61, respectively, (p < 0.001 for both); these correlation coefficients included a few scans with only two of the five possible magnitude scores. Artifact incidence was associated with breathing phase (p < 0.002), with presentation less likely near maximum exhale. Overall, the correlation metric allowed accurate and automated artifact identification. The consensus group evaluation resulted in efficient qualitative scoring, reduced interobserver variation, and provided consistent identification of artifact location and magnitudes.

Original languageEnglish (US)
Pages (from-to)190-201
Number of pages12
JournalJournal of Applied Clinical Medical Physics
Volume15
Issue number3
StatePublished - 2014
Externally publishedYes

Fingerprint

Four-Dimensional Computed Tomography
Artifacts
artifacts
evaluation
Computerized tomography
Tomography
correlation coefficients
breathing
thresholds
Respiration
couches
Sensitivity and Specificity
Observer Variation
ground truth
scoring
sensitivity

Keywords

  • 4D CT
  • Artifacts
  • Correlation
  • Visual assessment

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Radiation
  • Instrumentation

Cite this

Castillo, S. J., Castillo, R., Balter, P., Pan, T., Ibbott, G., Hobbs, B., ... Guerrero, T. (2014). Assessment of a quantitative metric for 4D CT artifact evaluation by observer consensus. Journal of Applied Clinical Medical Physics, 15(3), 190-201.

Assessment of a quantitative metric for 4D CT artifact evaluation by observer consensus. / Castillo, Sarah J.; Castillo, Richard; Balter, Peter; Pan, Tinsu; Ibbott, Geoffrey; Hobbs, Brian; Yuan, Ying; Guerrero, Thomas.

In: Journal of Applied Clinical Medical Physics, Vol. 15, No. 3, 2014, p. 190-201.

Research output: Contribution to journalArticle

Castillo, SJ, Castillo, R, Balter, P, Pan, T, Ibbott, G, Hobbs, B, Yuan, Y & Guerrero, T 2014, 'Assessment of a quantitative metric for 4D CT artifact evaluation by observer consensus', Journal of Applied Clinical Medical Physics, vol. 15, no. 3, pp. 190-201.
Castillo SJ, Castillo R, Balter P, Pan T, Ibbott G, Hobbs B et al. Assessment of a quantitative metric for 4D CT artifact evaluation by observer consensus. Journal of Applied Clinical Medical Physics. 2014;15(3):190-201.
Castillo, Sarah J. ; Castillo, Richard ; Balter, Peter ; Pan, Tinsu ; Ibbott, Geoffrey ; Hobbs, Brian ; Yuan, Ying ; Guerrero, Thomas. / Assessment of a quantitative metric for 4D CT artifact evaluation by observer consensus. In: Journal of Applied Clinical Medical Physics. 2014 ; Vol. 15, No. 3. pp. 190-201.
@article{277ac74e26114b92a01e63a41b589d1e,
title = "Assessment of a quantitative metric for 4D CT artifact evaluation by observer consensus",
abstract = "The benefits of four-dimensional computed tomography (4D CT) are limited by the presence of artifacts that remain difficult to quantify. A correlation-based metric previously proposed for cin{\'e} 4D CT artifact identification was further validated as an independent artifact evaluator by using a novel qualitative assessment featuring a group of observers reaching a consensus decision on artifact location and magnitude. The consensus group evaluated ten cin{\'e} 4D CT scans for artifacts over each breathing phase of coronal lung views assuming one artifact per couch location. Each artifact was assigned a magnitude score of 1-5, 1 indicating lowest severity and 5 indicating highest severity. Consensus group results served as the ground truth for assessment of the correlation metric. The ten patients were split into two cohorts; cohort 1 generated an artifact identification threshold derived from receiver operating characteristic analysis using the Youden Index, while cohort 2 generated sensitivity and specificity values from application of the artifact threshold. The Pearson correlation coefficient was calculated between the correlation metric values and the consensus group scores for both cohorts. The average sensitivity and specificity values found with application of the artifact threshold were 0.703 and 0.476, respectively. The correlation coefficients of artifact magnitudes for cohort 1 and 2 were 0.80 and 0.61, respectively, (p < 0.001 for both); these correlation coefficients included a few scans with only two of the five possible magnitude scores. Artifact incidence was associated with breathing phase (p < 0.002), with presentation less likely near maximum exhale. Overall, the correlation metric allowed accurate and automated artifact identification. The consensus group evaluation resulted in efficient qualitative scoring, reduced interobserver variation, and provided consistent identification of artifact location and magnitudes.",
keywords = "4D CT, Artifacts, Correlation, Visual assessment",
author = "Castillo, {Sarah J.} and Richard Castillo and Peter Balter and Tinsu Pan and Geoffrey Ibbott and Brian Hobbs and Ying Yuan and Thomas Guerrero",
year = "2014",
language = "English (US)",
volume = "15",
pages = "190--201",
journal = "Journal of applied clinical medical physics / American College of Medical Physics",
issn = "1526-9914",
publisher = "American Institute of Physics Publising LLC",
number = "3",

}

TY - JOUR

T1 - Assessment of a quantitative metric for 4D CT artifact evaluation by observer consensus

AU - Castillo, Sarah J.

AU - Castillo, Richard

AU - Balter, Peter

AU - Pan, Tinsu

AU - Ibbott, Geoffrey

AU - Hobbs, Brian

AU - Yuan, Ying

AU - Guerrero, Thomas

PY - 2014

Y1 - 2014

N2 - The benefits of four-dimensional computed tomography (4D CT) are limited by the presence of artifacts that remain difficult to quantify. A correlation-based metric previously proposed for ciné 4D CT artifact identification was further validated as an independent artifact evaluator by using a novel qualitative assessment featuring a group of observers reaching a consensus decision on artifact location and magnitude. The consensus group evaluated ten ciné 4D CT scans for artifacts over each breathing phase of coronal lung views assuming one artifact per couch location. Each artifact was assigned a magnitude score of 1-5, 1 indicating lowest severity and 5 indicating highest severity. Consensus group results served as the ground truth for assessment of the correlation metric. The ten patients were split into two cohorts; cohort 1 generated an artifact identification threshold derived from receiver operating characteristic analysis using the Youden Index, while cohort 2 generated sensitivity and specificity values from application of the artifact threshold. The Pearson correlation coefficient was calculated between the correlation metric values and the consensus group scores for both cohorts. The average sensitivity and specificity values found with application of the artifact threshold were 0.703 and 0.476, respectively. The correlation coefficients of artifact magnitudes for cohort 1 and 2 were 0.80 and 0.61, respectively, (p < 0.001 for both); these correlation coefficients included a few scans with only two of the five possible magnitude scores. Artifact incidence was associated with breathing phase (p < 0.002), with presentation less likely near maximum exhale. Overall, the correlation metric allowed accurate and automated artifact identification. The consensus group evaluation resulted in efficient qualitative scoring, reduced interobserver variation, and provided consistent identification of artifact location and magnitudes.

AB - The benefits of four-dimensional computed tomography (4D CT) are limited by the presence of artifacts that remain difficult to quantify. A correlation-based metric previously proposed for ciné 4D CT artifact identification was further validated as an independent artifact evaluator by using a novel qualitative assessment featuring a group of observers reaching a consensus decision on artifact location and magnitude. The consensus group evaluated ten ciné 4D CT scans for artifacts over each breathing phase of coronal lung views assuming one artifact per couch location. Each artifact was assigned a magnitude score of 1-5, 1 indicating lowest severity and 5 indicating highest severity. Consensus group results served as the ground truth for assessment of the correlation metric. The ten patients were split into two cohorts; cohort 1 generated an artifact identification threshold derived from receiver operating characteristic analysis using the Youden Index, while cohort 2 generated sensitivity and specificity values from application of the artifact threshold. The Pearson correlation coefficient was calculated between the correlation metric values and the consensus group scores for both cohorts. The average sensitivity and specificity values found with application of the artifact threshold were 0.703 and 0.476, respectively. The correlation coefficients of artifact magnitudes for cohort 1 and 2 were 0.80 and 0.61, respectively, (p < 0.001 for both); these correlation coefficients included a few scans with only two of the five possible magnitude scores. Artifact incidence was associated with breathing phase (p < 0.002), with presentation less likely near maximum exhale. Overall, the correlation metric allowed accurate and automated artifact identification. The consensus group evaluation resulted in efficient qualitative scoring, reduced interobserver variation, and provided consistent identification of artifact location and magnitudes.

KW - 4D CT

KW - Artifacts

KW - Correlation

KW - Visual assessment

UR - http://www.scopus.com/inward/record.url?scp=84900006128&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84900006128&partnerID=8YFLogxK

M3 - Article

VL - 15

SP - 190

EP - 201

JO - Journal of applied clinical medical physics / American College of Medical Physics

JF - Journal of applied clinical medical physics / American College of Medical Physics

SN - 1526-9914

IS - 3

ER -