Resumen de What We Don’t Test: What an Analysis of Unreleased ACS Exam Items Reveals about Content Coverage in General Chemistry Assessments
Jessica J. Reed, Sachel M. Villafañe, Jeffrey R. Raker, Thomas A. Holme, Kristen L. Murphy
General chemistry courses are often the foundation for the study of other science disciplines and upper-level chemistry concepts. Students who take introductory chemistry courses are more often from health and science-related fields than chemistry. As such, the content taught and assessed in general chemistry courses is envisioned as building blocks for concepts in other science courses and across many science disciplines. American Chemical Society (ACS) exams are developed by committees of expert chemists and serve as representative artifacts of content valued by the chemistry community. Before an exam is released, items developed by an examination committee undergo trial testing, and items deemed too easy, too hard, or not discriminating based on item statistics are removed. Analysis of content coverage from items on multiple released general chemistry ACS exams from the past decade and their associated unreleased trial tests revealed content areas where few exam items were written. Content coverage was analyzed by aligning exam items to the ACS Anchoring Concepts Content Map for General Chemistry. By comparing released and unreleased trial items, we found that many of the content coverage gaps on released exams are the result of a lack of items developed about these concepts. These conceptual holes warrant a discussion of whether what is being assessed is what is desired to be assessed and how curricular improvement and assessment reform efforts address content coverage.
