Effect of a Representative Sample of Internally Calibrated Mental Effort and Polytomously Scored Data on Representing Cognitive Efficiency

• Sachin Nedungadi ^[1] ; Olga Rinco Michels ^[2] ; Patricia J. Kreke ^[4] ; Jeffrey R. Raker ^[3] ; Kristen L. Murphy ^[5]
  1. [1] University of Nebraska at Omaha

     University of Nebraska at Omaha

     City of Omaha, Estados Unidos

     
  2. [2] Luther College

     Luther College

     Township of Decorah, Estados Unidos

     
  3. [3] University of South Florida

     University of South Florida

     Estados Unidos

     
  4. [4] Mount St. Mary’s University, United States
  5. [5] University of Wisconsin─Milwaukee, United States

  Mostrar afiliaciones +
• Localización: Journal of chemical education, ISSN 0021-9584, Vol. 99, Nº 3, 2022, págs. 1326-1335
• Idioma: inglés
• Texto completo no disponible (Saber más ...)
• Resumen

  • Practice examinations developed at the ACS Examinations Institute ask students to self-report mental effort when answering items. This self-reported mental effort together with performance can be represented in the form of a cognitive efficiency graph for each student giving information on the utilization of cognitive resources and content knowledge. When performance and mental effort used in the cognitive efficiency graph for a student is based on dichotomously scored data and raw reported mental effort, limitations arise. For example, it might be difficult to see a greater differentiation among students, and instructors might be interpreting student cognitive efficiency in certain content areas incorrectly. This study utilizes organic chemistry practice exam data to report an internal calibration method where a student’s mental effort is calibrated using their average reported mental effort across all items. Additionally, this study describes the benefits of using partial-credit assignment to provide greater differentiation in student performance. Results suggest that a more accurate cognitive efficiency graph can be obtained leading to a better assessment of student utilization of cognitive resources for different content areas. These results can have implications for how similar data on other ACS exams or similar data being used by researchers and practitioners can be analyzed to better represent cognitive efficiency.