Identification and Analysis of Item Environment Effects on Organic Chemistry Multiple-Choice Exams
-
Olga Michels [1] ; David G. Schreurs [2] ; Sachin Nedungadi [3] ; Thomas C. Pentecost [4] ; Patricia J. Kreke [6] ; Jeffrey R. Raker [5] ; Kristen L. Murphy [2]
-
[1] Luther College
Luther College
Township of Decorah, Estados Unidos
-
[2] University of Wisconsin–Milwaukee
University of Wisconsin–Milwaukee
City of Milwaukee, Estados Unidos
-
[3] University of Nebraska at Omaha
University of Nebraska at Omaha
City of Omaha, Estados Unidos
-
[4] Grand Valley State University
Grand Valley State University
Charter Township of Allendale, Estados Unidos
-
[5] University of South Florida
University of South Florida
Estados Unidos
-
[6] Mount St. Mary’s University, United States
-
- Localización: Journal of chemical education, ISSN 0021-9584, Vol. 102, Nº 1, 2025, págs. 43-52
- Idioma: inglés
- Texto completo no disponible (Saber más ...)
-
Resumen
When delivering high-stakes assessments, multiple forms with differing item order are often used to mitigate cheating. However, an often-overlooked side-effect of this practice is differences in item order may impact student performance. The result of this can be a misrepresentation of student understanding and provide an advantage to one group of students based on the form they were assigned. This work explores the detection of these form-dependent items and classifies detected items based on the likely reason for changes in item performance. Analyses were completed using a combination of classical test theory and item response theory methods while analyzing first-term ACS organic chemistry exam items. The form dependence of most items could be explained by content priming, preceding item difficulty, or a combination of both. A new layer of item order effects was identified through this analysis which we term “perceived item difficulty”. Perceived item difficulty was further explored through a qualitative study.
