Virtual reality simulator training for laparoscopic colectomy

What metrics have construct validity?

Skandan Shanmugan, Fabien Leblanc, Anthony J. Senagore, C. Neal Ellis, Sharon L. Stein, Sadaf Khan, Conor P. Delaney, Bradley J. Champagne

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

Background: Virtual reality simulation for laparoscopic colectomy has been used for training of surgical residents and has been considered as a model for technical skills assessment of board-eligible colorectal surgeons. However, construct validity (the ability to distinguish between skill levels) must be confirmed before widespread implementation. Objective: This study was designed to specifically determine which metrics for laparoscopic sigmoid colectomy have evidence of construct validity. Design: General surgeons that had performed fewer than 30 laparoscopic colon resections and laparoscopic colorectal experts (>200 laparoscopic colon resections) performed laparoscopic sigmoid colectomy on the LA P Mentor model. All participants received a 15-minute instructional warm-up and had never used the simulator before the study. Performance was then compared between each group for 21 metrics (procedural, 14; intraoperative errors, 7) to determine specifically which measurements demonstrate construct validity. Performance was compared with the Mann-Whitney U-test (p <0.05 was significant). Results: Fifty-three surgeons; 29 general surgeons, and 24 colorectal surgeons enrolled in the study. The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 of 14 procedural metrics by distinguishing levels of surgical experience (p <0.05). The most discriminatory procedural metrics (p <0.01) favoring experts were reduced instrument path length, accuracy of the peritoneal/medial mobilization, and dissection of the inferior mesenteric artery. Intraoperative errors were not discriminatory for most metrics and favored general surgeons for colonic wall injury (general surgeons, 0.7; colorectal surgeons, 3.5; p = 0.045). Limitations: Individual variability within the general surgeon and colorectal surgeon groups was not accounted for. Conclusions: The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 procedure-specific metrics. However, using virtual reality simulator metrics to detect intraoperative errors did not discriminate between groups. If the virtual reality simulator continues to be used for the technical assessment of trainees and board-eligible surgeons, the evaluation of performance should be limited to procedural metrics.

Original languageEnglish (US)
Pages (from-to)210-214
Number of pages5
JournalDiseases of the Colon and Rectum
Volume57
Issue number2
DOIs
StatePublished - Feb 2014
Externally publishedYes

Fingerprint

Colectomy
Sigmoid Colon
Colon
Surgeons
Inferior Mesenteric Artery
Mentors
Nonparametric Statistics
Dissection

Keywords

  • Construct validity
  • Education
  • Laparoscopic colectomy
  • Virtual reality simulation

ASJC Scopus subject areas

  • Gastroenterology

Cite this

Shanmugan, S., Leblanc, F., Senagore, A. J., Ellis, C. N., Stein, S. L., Khan, S., ... Champagne, B. J. (2014). Virtual reality simulator training for laparoscopic colectomy: What metrics have construct validity? Diseases of the Colon and Rectum, 57(2), 210-214. https://doi.org/10.1097/DCR.0000000000000031

Virtual reality simulator training for laparoscopic colectomy : What metrics have construct validity? / Shanmugan, Skandan; Leblanc, Fabien; Senagore, Anthony J.; Ellis, C. Neal; Stein, Sharon L.; Khan, Sadaf; Delaney, Conor P.; Champagne, Bradley J.

In: Diseases of the Colon and Rectum, Vol. 57, No. 2, 02.2014, p. 210-214.

Research output: Contribution to journalArticle

Shanmugan, S, Leblanc, F, Senagore, AJ, Ellis, CN, Stein, SL, Khan, S, Delaney, CP & Champagne, BJ 2014, 'Virtual reality simulator training for laparoscopic colectomy: What metrics have construct validity?', Diseases of the Colon and Rectum, vol. 57, no. 2, pp. 210-214. https://doi.org/10.1097/DCR.0000000000000031
Shanmugan, Skandan ; Leblanc, Fabien ; Senagore, Anthony J. ; Ellis, C. Neal ; Stein, Sharon L. ; Khan, Sadaf ; Delaney, Conor P. ; Champagne, Bradley J. / Virtual reality simulator training for laparoscopic colectomy : What metrics have construct validity?. In: Diseases of the Colon and Rectum. 2014 ; Vol. 57, No. 2. pp. 210-214.
@article{027fc41e79ff4221be6e13bca553cdcb,
title = "Virtual reality simulator training for laparoscopic colectomy: What metrics have construct validity?",
abstract = "Background: Virtual reality simulation for laparoscopic colectomy has been used for training of surgical residents and has been considered as a model for technical skills assessment of board-eligible colorectal surgeons. However, construct validity (the ability to distinguish between skill levels) must be confirmed before widespread implementation. Objective: This study was designed to specifically determine which metrics for laparoscopic sigmoid colectomy have evidence of construct validity. Design: General surgeons that had performed fewer than 30 laparoscopic colon resections and laparoscopic colorectal experts (>200 laparoscopic colon resections) performed laparoscopic sigmoid colectomy on the LA P Mentor model. All participants received a 15-minute instructional warm-up and had never used the simulator before the study. Performance was then compared between each group for 21 metrics (procedural, 14; intraoperative errors, 7) to determine specifically which measurements demonstrate construct validity. Performance was compared with the Mann-Whitney U-test (p <0.05 was significant). Results: Fifty-three surgeons; 29 general surgeons, and 24 colorectal surgeons enrolled in the study. The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 of 14 procedural metrics by distinguishing levels of surgical experience (p <0.05). The most discriminatory procedural metrics (p <0.01) favoring experts were reduced instrument path length, accuracy of the peritoneal/medial mobilization, and dissection of the inferior mesenteric artery. Intraoperative errors were not discriminatory for most metrics and favored general surgeons for colonic wall injury (general surgeons, 0.7; colorectal surgeons, 3.5; p = 0.045). Limitations: Individual variability within the general surgeon and colorectal surgeon groups was not accounted for. Conclusions: The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 procedure-specific metrics. However, using virtual reality simulator metrics to detect intraoperative errors did not discriminate between groups. If the virtual reality simulator continues to be used for the technical assessment of trainees and board-eligible surgeons, the evaluation of performance should be limited to procedural metrics.",
keywords = "Construct validity, Education, Laparoscopic colectomy, Virtual reality simulation",
author = "Skandan Shanmugan and Fabien Leblanc and Senagore, {Anthony J.} and Ellis, {C. Neal} and Stein, {Sharon L.} and Sadaf Khan and Delaney, {Conor P.} and Champagne, {Bradley J.}",
year = "2014",
month = "2",
doi = "10.1097/DCR.0000000000000031",
language = "English (US)",
volume = "57",
pages = "210--214",
journal = "Diseases of the Colon and Rectum",
issn = "0012-3706",
publisher = "Lippincott Williams and Wilkins",
number = "2",

}

TY - JOUR

T1 - Virtual reality simulator training for laparoscopic colectomy

T2 - What metrics have construct validity?

AU - Shanmugan, Skandan

AU - Leblanc, Fabien

AU - Senagore, Anthony J.

AU - Ellis, C. Neal

AU - Stein, Sharon L.

AU - Khan, Sadaf

AU - Delaney, Conor P.

AU - Champagne, Bradley J.

PY - 2014/2

Y1 - 2014/2

N2 - Background: Virtual reality simulation for laparoscopic colectomy has been used for training of surgical residents and has been considered as a model for technical skills assessment of board-eligible colorectal surgeons. However, construct validity (the ability to distinguish between skill levels) must be confirmed before widespread implementation. Objective: This study was designed to specifically determine which metrics for laparoscopic sigmoid colectomy have evidence of construct validity. Design: General surgeons that had performed fewer than 30 laparoscopic colon resections and laparoscopic colorectal experts (>200 laparoscopic colon resections) performed laparoscopic sigmoid colectomy on the LA P Mentor model. All participants received a 15-minute instructional warm-up and had never used the simulator before the study. Performance was then compared between each group for 21 metrics (procedural, 14; intraoperative errors, 7) to determine specifically which measurements demonstrate construct validity. Performance was compared with the Mann-Whitney U-test (p <0.05 was significant). Results: Fifty-three surgeons; 29 general surgeons, and 24 colorectal surgeons enrolled in the study. The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 of 14 procedural metrics by distinguishing levels of surgical experience (p <0.05). The most discriminatory procedural metrics (p <0.01) favoring experts were reduced instrument path length, accuracy of the peritoneal/medial mobilization, and dissection of the inferior mesenteric artery. Intraoperative errors were not discriminatory for most metrics and favored general surgeons for colonic wall injury (general surgeons, 0.7; colorectal surgeons, 3.5; p = 0.045). Limitations: Individual variability within the general surgeon and colorectal surgeon groups was not accounted for. Conclusions: The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 procedure-specific metrics. However, using virtual reality simulator metrics to detect intraoperative errors did not discriminate between groups. If the virtual reality simulator continues to be used for the technical assessment of trainees and board-eligible surgeons, the evaluation of performance should be limited to procedural metrics.

AB - Background: Virtual reality simulation for laparoscopic colectomy has been used for training of surgical residents and has been considered as a model for technical skills assessment of board-eligible colorectal surgeons. However, construct validity (the ability to distinguish between skill levels) must be confirmed before widespread implementation. Objective: This study was designed to specifically determine which metrics for laparoscopic sigmoid colectomy have evidence of construct validity. Design: General surgeons that had performed fewer than 30 laparoscopic colon resections and laparoscopic colorectal experts (>200 laparoscopic colon resections) performed laparoscopic sigmoid colectomy on the LA P Mentor model. All participants received a 15-minute instructional warm-up and had never used the simulator before the study. Performance was then compared between each group for 21 metrics (procedural, 14; intraoperative errors, 7) to determine specifically which measurements demonstrate construct validity. Performance was compared with the Mann-Whitney U-test (p <0.05 was significant). Results: Fifty-three surgeons; 29 general surgeons, and 24 colorectal surgeons enrolled in the study. The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 of 14 procedural metrics by distinguishing levels of surgical experience (p <0.05). The most discriminatory procedural metrics (p <0.01) favoring experts were reduced instrument path length, accuracy of the peritoneal/medial mobilization, and dissection of the inferior mesenteric artery. Intraoperative errors were not discriminatory for most metrics and favored general surgeons for colonic wall injury (general surgeons, 0.7; colorectal surgeons, 3.5; p = 0.045). Limitations: Individual variability within the general surgeon and colorectal surgeon groups was not accounted for. Conclusions: The virtual reality simulators for laparoscopic sigmoid colectomy demonstrated construct validity for 8 procedure-specific metrics. However, using virtual reality simulator metrics to detect intraoperative errors did not discriminate between groups. If the virtual reality simulator continues to be used for the technical assessment of trainees and board-eligible surgeons, the evaluation of performance should be limited to procedural metrics.

KW - Construct validity

KW - Education

KW - Laparoscopic colectomy

KW - Virtual reality simulation

UR - http://www.scopus.com/inward/record.url?scp=84893760036&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84893760036&partnerID=8YFLogxK

U2 - 10.1097/DCR.0000000000000031

DO - 10.1097/DCR.0000000000000031

M3 - Article

VL - 57

SP - 210

EP - 214

JO - Diseases of the Colon and Rectum

JF - Diseases of the Colon and Rectum

SN - 0012-3706

IS - 2

ER -