Reliability, validity, and fairness: a content analysis of assessment development publications in major engineering education journals

• Autores: Kerrie A. Douglas, Anastasia Rynearson, Senay Purzer, Johannes Strobel
• Localización: The International journal of engineering education, ISSN-e 0949-149X, Vol. 32, no. 5 (Parte A), 2016, 1971 págs.
• Idioma: inglés
• Texto completo no disponible (Saber más ...)
• Resumen

  • After more than a decade of efforts to enhance the quality of engineering education research, including assessmentdevelopment, it is timely to explore what types of validity evidence are frequently reported in assessment research articles.According to theStandards for Educational and Psychological Testing, the foundation of quality assessment rests onevidence of reliability, validity, and fairness. The purpose of this study was to explore what aspects of reliability, validity,and fairness evidence are provided in assessment instrument development publications in major engineering educationjournals since 2005. Using quantitative content analysis, the authors reviewed twenty-nine articles published in four majorengineering education journals between 2005 and 2015. A coding scheme, based on Messick’s Unified Theory of Validityand theStandards for Educational and Psychological Testingwas developed to code the aspects of reliability, validity, andfairness provided in each article. Frequencies for each code are reported. Engineering education articles on instrumentdevelopment most frequently reported evidence related to aspects of internal reliability, content-related validity, andsubstantive aspects of validity. However, studies of generalizability, consequences, and fairness were largely void. Inaddition, reliability was most frequently studied through internal reliability coefficients, while other forms, such as test-retest were less frequently reported.