Effects of comprehensive educational reforms on academic success in a diverse student body

Steven Lieberman, Michael Ainsworth, Gregory K. Asimakis, Lauree Thomas, Lisa D. Cain, Melodee G. Mancuso, Jeffrey P. Rabek, Ni Zhang, Ann W. Frye

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

Context Calls for medical curriculum reform and increased student diversity in the USA have seen mixed success: performance outcomes following curriculum revisions have been inconsistent and national matriculation of under-represented minority (URM) students has not met aspirations. Published innovations in curricula, academic support and pipeline programmes usually describe isolated interventions that fail to affect curriculum-level outcomes.Methods United States Medical Licensing Examination (USMLE) Step 1 performance and graduation rates were analysed for three classes of medical students before (matriculated 1995-1997, n = 517) and after (matriculated 2003-2005, n = 597) implementing broad-based reforms in our education system. The changes in pipeline recruitment and preparation programmes, instructional methods, assessment systems, academic support and board preparation were based on sound educational principles and best practices.Results Post-reform classes were diverse with respect to ethnicity (25.8% URM students), gender (51.8% female), and Medical College Admissions Test (MCAT) score (range 20-40; 24.1% scored ≤ 25). Mean ± standard deviation MCAT scores were minimally changed (from 27.2 ± 4.7 to 27.8 ± 3.6). The Step 1 failure rate decreased by 69.3% and mean score increased by 14.0 points (effect size: d = 0.67) overall. Improvements were greater among women (failure rate decreased by 78.9%, mean score increased by 15.6 points; d = 0.76) and URM students (failure rate decreased by 76.5%, mean score increased by 14.6 points; d = 0.74), especially African-American students (failure rate decreased by 93.6%, mean score increased by 20.8 points; d = 1.12). Step 1 scores increased across the entire MCAT range. Four- and 5-year graduation rates increased by 7.1% and 5.8%, respectively.Conclusions The effect sizes in these performance improvements surpassed those previously reported for isolated interventions in curriculum and student support. This success is likely to have resulted from the broad-based, mutually reinforcing nature of reforms in multiple components of the education system. The results suggest that a narrow reductionist view of educational programme reform is less likely to result in improved educational outcomes than a system perspective that addresses the coordinated functioning of multiple aspects of the academic enterprise.

Original languageEnglish (US)
Pages (from-to)1232-1240
Number of pages9
JournalMedical Education
Volume44
Issue number12
DOIs
StatePublished - Dec 2010

Fingerprint

student body
academic success
educational reform
College Admission Test
Curriculum
Students
reform
curriculum
student
minority
education system
Education
curriculum revision
performance
Licensure
Medical Students
Practice Guidelines
African Americans
educational program
best practice

ASJC Scopus subject areas

  • Medicine(all)
  • Education

Cite this

Effects of comprehensive educational reforms on academic success in a diverse student body. / Lieberman, Steven; Ainsworth, Michael; Asimakis, Gregory K.; Thomas, Lauree; Cain, Lisa D.; Mancuso, Melodee G.; Rabek, Jeffrey P.; Zhang, Ni; Frye, Ann W.

In: Medical Education, Vol. 44, No. 12, 12.2010, p. 1232-1240.

Research output: Contribution to journalArticle

Lieberman, S, Ainsworth, M, Asimakis, GK, Thomas, L, Cain, LD, Mancuso, MG, Rabek, JP, Zhang, N & Frye, AW 2010, 'Effects of comprehensive educational reforms on academic success in a diverse student body', Medical Education, vol. 44, no. 12, pp. 1232-1240. https://doi.org/10.1111/j.1365-2923.2010.03770.x
Lieberman, Steven ; Ainsworth, Michael ; Asimakis, Gregory K. ; Thomas, Lauree ; Cain, Lisa D. ; Mancuso, Melodee G. ; Rabek, Jeffrey P. ; Zhang, Ni ; Frye, Ann W. / Effects of comprehensive educational reforms on academic success in a diverse student body. In: Medical Education. 2010 ; Vol. 44, No. 12. pp. 1232-1240.
@article{2ab75d20205f4e85b92669daa64c4e08,
title = "Effects of comprehensive educational reforms on academic success in a diverse student body",
abstract = "Context Calls for medical curriculum reform and increased student diversity in the USA have seen mixed success: performance outcomes following curriculum revisions have been inconsistent and national matriculation of under-represented minority (URM) students has not met aspirations. Published innovations in curricula, academic support and pipeline programmes usually describe isolated interventions that fail to affect curriculum-level outcomes.Methods United States Medical Licensing Examination (USMLE) Step 1 performance and graduation rates were analysed for three classes of medical students before (matriculated 1995-1997, n = 517) and after (matriculated 2003-2005, n = 597) implementing broad-based reforms in our education system. The changes in pipeline recruitment and preparation programmes, instructional methods, assessment systems, academic support and board preparation were based on sound educational principles and best practices.Results Post-reform classes were diverse with respect to ethnicity (25.8{\%} URM students), gender (51.8{\%} female), and Medical College Admissions Test (MCAT) score (range 20-40; 24.1{\%} scored ≤ 25). Mean ± standard deviation MCAT scores were minimally changed (from 27.2 ± 4.7 to 27.8 ± 3.6). The Step 1 failure rate decreased by 69.3{\%} and mean score increased by 14.0 points (effect size: d = 0.67) overall. Improvements were greater among women (failure rate decreased by 78.9{\%}, mean score increased by 15.6 points; d = 0.76) and URM students (failure rate decreased by 76.5{\%}, mean score increased by 14.6 points; d = 0.74), especially African-American students (failure rate decreased by 93.6{\%}, mean score increased by 20.8 points; d = 1.12). Step 1 scores increased across the entire MCAT range. Four- and 5-year graduation rates increased by 7.1{\%} and 5.8{\%}, respectively.Conclusions The effect sizes in these performance improvements surpassed those previously reported for isolated interventions in curriculum and student support. This success is likely to have resulted from the broad-based, mutually reinforcing nature of reforms in multiple components of the education system. The results suggest that a narrow reductionist view of educational programme reform is less likely to result in improved educational outcomes than a system perspective that addresses the coordinated functioning of multiple aspects of the academic enterprise.",
author = "Steven Lieberman and Michael Ainsworth and Asimakis, {Gregory K.} and Lauree Thomas and Cain, {Lisa D.} and Mancuso, {Melodee G.} and Rabek, {Jeffrey P.} and Ni Zhang and Frye, {Ann W.}",
year = "2010",
month = "12",
doi = "10.1111/j.1365-2923.2010.03770.x",
language = "English (US)",
volume = "44",
pages = "1232--1240",
journal = "Medical Education",
issn = "0308-0110",
publisher = "Wiley-Blackwell",
number = "12",

}

TY - JOUR

T1 - Effects of comprehensive educational reforms on academic success in a diverse student body

AU - Lieberman, Steven

AU - Ainsworth, Michael

AU - Asimakis, Gregory K.

AU - Thomas, Lauree

AU - Cain, Lisa D.

AU - Mancuso, Melodee G.

AU - Rabek, Jeffrey P.

AU - Zhang, Ni

AU - Frye, Ann W.

PY - 2010/12

Y1 - 2010/12

N2 - Context Calls for medical curriculum reform and increased student diversity in the USA have seen mixed success: performance outcomes following curriculum revisions have been inconsistent and national matriculation of under-represented minority (URM) students has not met aspirations. Published innovations in curricula, academic support and pipeline programmes usually describe isolated interventions that fail to affect curriculum-level outcomes.Methods United States Medical Licensing Examination (USMLE) Step 1 performance and graduation rates were analysed for three classes of medical students before (matriculated 1995-1997, n = 517) and after (matriculated 2003-2005, n = 597) implementing broad-based reforms in our education system. The changes in pipeline recruitment and preparation programmes, instructional methods, assessment systems, academic support and board preparation were based on sound educational principles and best practices.Results Post-reform classes were diverse with respect to ethnicity (25.8% URM students), gender (51.8% female), and Medical College Admissions Test (MCAT) score (range 20-40; 24.1% scored ≤ 25). Mean ± standard deviation MCAT scores were minimally changed (from 27.2 ± 4.7 to 27.8 ± 3.6). The Step 1 failure rate decreased by 69.3% and mean score increased by 14.0 points (effect size: d = 0.67) overall. Improvements were greater among women (failure rate decreased by 78.9%, mean score increased by 15.6 points; d = 0.76) and URM students (failure rate decreased by 76.5%, mean score increased by 14.6 points; d = 0.74), especially African-American students (failure rate decreased by 93.6%, mean score increased by 20.8 points; d = 1.12). Step 1 scores increased across the entire MCAT range. Four- and 5-year graduation rates increased by 7.1% and 5.8%, respectively.Conclusions The effect sizes in these performance improvements surpassed those previously reported for isolated interventions in curriculum and student support. This success is likely to have resulted from the broad-based, mutually reinforcing nature of reforms in multiple components of the education system. The results suggest that a narrow reductionist view of educational programme reform is less likely to result in improved educational outcomes than a system perspective that addresses the coordinated functioning of multiple aspects of the academic enterprise.

AB - Context Calls for medical curriculum reform and increased student diversity in the USA have seen mixed success: performance outcomes following curriculum revisions have been inconsistent and national matriculation of under-represented minority (URM) students has not met aspirations. Published innovations in curricula, academic support and pipeline programmes usually describe isolated interventions that fail to affect curriculum-level outcomes.Methods United States Medical Licensing Examination (USMLE) Step 1 performance and graduation rates were analysed for three classes of medical students before (matriculated 1995-1997, n = 517) and after (matriculated 2003-2005, n = 597) implementing broad-based reforms in our education system. The changes in pipeline recruitment and preparation programmes, instructional methods, assessment systems, academic support and board preparation were based on sound educational principles and best practices.Results Post-reform classes were diverse with respect to ethnicity (25.8% URM students), gender (51.8% female), and Medical College Admissions Test (MCAT) score (range 20-40; 24.1% scored ≤ 25). Mean ± standard deviation MCAT scores were minimally changed (from 27.2 ± 4.7 to 27.8 ± 3.6). The Step 1 failure rate decreased by 69.3% and mean score increased by 14.0 points (effect size: d = 0.67) overall. Improvements were greater among women (failure rate decreased by 78.9%, mean score increased by 15.6 points; d = 0.76) and URM students (failure rate decreased by 76.5%, mean score increased by 14.6 points; d = 0.74), especially African-American students (failure rate decreased by 93.6%, mean score increased by 20.8 points; d = 1.12). Step 1 scores increased across the entire MCAT range. Four- and 5-year graduation rates increased by 7.1% and 5.8%, respectively.Conclusions The effect sizes in these performance improvements surpassed those previously reported for isolated interventions in curriculum and student support. This success is likely to have resulted from the broad-based, mutually reinforcing nature of reforms in multiple components of the education system. The results suggest that a narrow reductionist view of educational programme reform is less likely to result in improved educational outcomes than a system perspective that addresses the coordinated functioning of multiple aspects of the academic enterprise.

UR - http://www.scopus.com/inward/record.url?scp=78649564261&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78649564261&partnerID=8YFLogxK

U2 - 10.1111/j.1365-2923.2010.03770.x

DO - 10.1111/j.1365-2923.2010.03770.x

M3 - Article

C2 - 21070343

AN - SCOPUS:78649564261

VL - 44

SP - 1232

EP - 1240

JO - Medical Education

JF - Medical Education

SN - 0308-0110

IS - 12

ER -