This is one school’s experience with NBME assessments. Results or outcomes may vary and depend on factors beyond NBME’s control, such as student participation, individual study habits, faculty support and engagement, and/or variations in curriculum, among other things.

The University of North Carolina School of Medicine (UNC) recognized the challenge of reliably predicting student performance on standardized exams and identifying students at risk of underperforming. To address this, UNC combined NBME® Subject Exams data to develop models that help flag students who may be at risk, enabling early intervention and support well before they take higher-stakes assessments. 

UNC provided clerkship-level students with vouchers for NBME Self-Assessments. The resulting scores, along with prior standardized test results and the timing between Self-Assessments and Subject Exams, were used to create a multivariable linear regression model. While predictive value varied by clerkship, this approach offered a practical means to identify students who might benefit from additional support. 

The initial models, though limited by changes such as the transition of USMLE® Step 1 to pass/fail, demonstrate the potential of analytics in supporting student success. The six Clinical Science Subject Examinations — Internal Medicine, Surgery, Pediatrics, Obstetrics-Gynecology, Psychiatry, and Family Medicine — are administered during the clerkship year. Self-Assessments from the Clinical Mastery Series are taken two to four weeks prior to each Subject Exam, providing valuable data for both students and faculty. 

Faculty at UNC note that NBME Self-Assessments offer an accessible way for students to gauge their readiness, while modeling by the school enables early identification and support for those at risk. This proactive approach fosters student success and readiness for high-stakes assessments.

“Having the ability to model student performance before taking a high stakes assessment helped us identify students at high risk of failure, allowing us to intervene early and improve student success.”

 Christina Shenvi, MD, PhD, MBA; Former Director, Office of Academic Excellence, UNC School of Medicine

“NBME products, like Self-Assessments provide us with an easy, accessible way to help our students understand that they are prepared for their exams.” 

Gina Horne, NBME Chief Proctor, UNC School of Medicine

NBME PORTFOLIO PARTS USED

NBME® Subject Exams

The six Clinical Science Subject Examinations are administered during the clerkship year: Internal Medicine, Surgery, Pediatrics, Obstetrics-Gynecology, Psychiatry, and Family Medicine.  

NBME Self-Assessments

Vouchers were provided to students to take self-assessments from the Clinical Mastery Series from 2 – 4 weeks before taking each of the six Clinical Science Subject Examinations  

SUPPORTING DATA 

Alexander S M, Dallaghan GL, Birch M, Smith KL, Howard N, Shenvi CL: What makes a near-peer learning and tutoring program effective in undergraduate medical education: a qualitative analysis. Med Sci Educ. 2022, 32:1495-502. 10.1007/s40670-022-01680-0 

Alexander S M, Shenvi C L, Nichols K R, et al.: (June 22, 2023) Multivariate Modeling of Student Performance on NBME Subject Exams. Cureus 15(6): e40809. doi:10.7759/cureus.40809  

Swan Sein A, Daniel M, Fleming A, et al.: Identifying and supporting students to prevent USMLE Step 1 failures when testing follows clerkships: insights from 9 schools. Acad Med. 2020, 95:1338-45. 10.1097/ACM.0000000000003272 

Wang L, Laird-Fick HS, Parker CJ, Solomon D: Using Markov chain model to evaluate medical students’ trajectory on progress tests and predict USMLE step 1 scores—a retrospective cohort study in one medical school. BMC Med Educ. 2021, 21:200. 10.1186/s12909-021-02633-8 

Interested in receiving the latest from NBME?