A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets

Richard Castillo, Edward Castillo, Rudy Guerra, Valen E. Johnson, Travis McPhail, Amit K. Garg, Thomas Guerrero

Research output: Contribution to journalArticle

321 Citations (Scopus)

Abstract

Expert landmark correspondences are widely reported for evaluating deformable image registration (DIR) spatial accuracy. In this report, we present a framework for objective evaluation of DIR spatial accuracy using large sets of expert-determined landmark point pairs. Large samples (>1100) of pulmonary landmark point pairs were manually generated for five cases. Estimates of inter- and intra-observer variation were determined from repeated registration. Comparative evaluation of DIR spatial accuracy was performed for two algorithms, a gradient-based optical flow algorithm and a landmark-based moving least-squares algorithm. The uncertainty of spatial error estimates was found to be inversely proportional to the square root of the number of landmark point pairs and directly proportional to the standard deviation of the spatial errors. Using the statistical properties of this data, we performed sample size calculations to estimate the average spatial accuracy of each algorithm with 95% confidence intervals within a 0.5 mm range. For the optical flow and moving least-squares algorithms, the required sample sizes were 1050 and 36, respectively. Comparative evaluation based on fewer than the required validation landmarks results in misrepresentation of the relative spatial accuracy. This study demonstrates that landmark pairs can be used to assess DIR spatial accuracy within a narrow uncertainty range.

Original languageEnglish (US)
Pages (from-to)1849-1870
Number of pages22
JournalPhysics in Medicine and Biology
Volume54
Issue number7
DOIs
StatePublished - 2009
Externally publishedYes

Fingerprint

Least-Squares Analysis
Sample Size
Uncertainty
Observer Variation
Confidence Intervals
Lung

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Radiological and Ultrasound Technology

Cite this

Castillo, R., Castillo, E., Guerra, R., Johnson, V. E., McPhail, T., Garg, A. K., & Guerrero, T. (2009). A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Physics in Medicine and Biology, 54(7), 1849-1870. https://doi.org/10.1088/0031-9155/54/7/001

A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. / Castillo, Richard; Castillo, Edward; Guerra, Rudy; Johnson, Valen E.; McPhail, Travis; Garg, Amit K.; Guerrero, Thomas.

In: Physics in Medicine and Biology, Vol. 54, No. 7, 2009, p. 1849-1870.

Research output: Contribution to journalArticle

Castillo, R, Castillo, E, Guerra, R, Johnson, VE, McPhail, T, Garg, AK & Guerrero, T 2009, 'A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets', Physics in Medicine and Biology, vol. 54, no. 7, pp. 1849-1870. https://doi.org/10.1088/0031-9155/54/7/001
Castillo, Richard ; Castillo, Edward ; Guerra, Rudy ; Johnson, Valen E. ; McPhail, Travis ; Garg, Amit K. ; Guerrero, Thomas. / A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. In: Physics in Medicine and Biology. 2009 ; Vol. 54, No. 7. pp. 1849-1870.
@article{10d4e9414e4440a18f4c32fd3e25eb54,
title = "A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets",
abstract = "Expert landmark correspondences are widely reported for evaluating deformable image registration (DIR) spatial accuracy. In this report, we present a framework for objective evaluation of DIR spatial accuracy using large sets of expert-determined landmark point pairs. Large samples (>1100) of pulmonary landmark point pairs were manually generated for five cases. Estimates of inter- and intra-observer variation were determined from repeated registration. Comparative evaluation of DIR spatial accuracy was performed for two algorithms, a gradient-based optical flow algorithm and a landmark-based moving least-squares algorithm. The uncertainty of spatial error estimates was found to be inversely proportional to the square root of the number of landmark point pairs and directly proportional to the standard deviation of the spatial errors. Using the statistical properties of this data, we performed sample size calculations to estimate the average spatial accuracy of each algorithm with 95{\%} confidence intervals within a 0.5 mm range. For the optical flow and moving least-squares algorithms, the required sample sizes were 1050 and 36, respectively. Comparative evaluation based on fewer than the required validation landmarks results in misrepresentation of the relative spatial accuracy. This study demonstrates that landmark pairs can be used to assess DIR spatial accuracy within a narrow uncertainty range.",
author = "Richard Castillo and Edward Castillo and Rudy Guerra and Johnson, {Valen E.} and Travis McPhail and Garg, {Amit K.} and Thomas Guerrero",
year = "2009",
doi = "10.1088/0031-9155/54/7/001",
language = "English (US)",
volume = "54",
pages = "1849--1870",
journal = "Physics in Medicine and Biology",
issn = "0031-9155",
publisher = "IOP Publishing Ltd.",
number = "7",

}

TY - JOUR

T1 - A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets

AU - Castillo, Richard

AU - Castillo, Edward

AU - Guerra, Rudy

AU - Johnson, Valen E.

AU - McPhail, Travis

AU - Garg, Amit K.

AU - Guerrero, Thomas

PY - 2009

Y1 - 2009

N2 - Expert landmark correspondences are widely reported for evaluating deformable image registration (DIR) spatial accuracy. In this report, we present a framework for objective evaluation of DIR spatial accuracy using large sets of expert-determined landmark point pairs. Large samples (>1100) of pulmonary landmark point pairs were manually generated for five cases. Estimates of inter- and intra-observer variation were determined from repeated registration. Comparative evaluation of DIR spatial accuracy was performed for two algorithms, a gradient-based optical flow algorithm and a landmark-based moving least-squares algorithm. The uncertainty of spatial error estimates was found to be inversely proportional to the square root of the number of landmark point pairs and directly proportional to the standard deviation of the spatial errors. Using the statistical properties of this data, we performed sample size calculations to estimate the average spatial accuracy of each algorithm with 95% confidence intervals within a 0.5 mm range. For the optical flow and moving least-squares algorithms, the required sample sizes were 1050 and 36, respectively. Comparative evaluation based on fewer than the required validation landmarks results in misrepresentation of the relative spatial accuracy. This study demonstrates that landmark pairs can be used to assess DIR spatial accuracy within a narrow uncertainty range.

AB - Expert landmark correspondences are widely reported for evaluating deformable image registration (DIR) spatial accuracy. In this report, we present a framework for objective evaluation of DIR spatial accuracy using large sets of expert-determined landmark point pairs. Large samples (>1100) of pulmonary landmark point pairs were manually generated for five cases. Estimates of inter- and intra-observer variation were determined from repeated registration. Comparative evaluation of DIR spatial accuracy was performed for two algorithms, a gradient-based optical flow algorithm and a landmark-based moving least-squares algorithm. The uncertainty of spatial error estimates was found to be inversely proportional to the square root of the number of landmark point pairs and directly proportional to the standard deviation of the spatial errors. Using the statistical properties of this data, we performed sample size calculations to estimate the average spatial accuracy of each algorithm with 95% confidence intervals within a 0.5 mm range. For the optical flow and moving least-squares algorithms, the required sample sizes were 1050 and 36, respectively. Comparative evaluation based on fewer than the required validation landmarks results in misrepresentation of the relative spatial accuracy. This study demonstrates that landmark pairs can be used to assess DIR spatial accuracy within a narrow uncertainty range.

UR - http://www.scopus.com/inward/record.url?scp=63549096561&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=63549096561&partnerID=8YFLogxK

U2 - 10.1088/0031-9155/54/7/001

DO - 10.1088/0031-9155/54/7/001

M3 - Article

VL - 54

SP - 1849

EP - 1870

JO - Physics in Medicine and Biology

JF - Physics in Medicine and Biology

SN - 0031-9155

IS - 7

ER -