Use of multilevel models to consider non-cognitive information in evaluation: Personality, Omissions and item-writing Flaws

• Autores: Susana Sanz Velasco
• Directores de la Tesis: Carmen García García (dir. tes.), Ricardo Olmos Albacete (dir. tes.)
• Lectura: En la Universidad Autónoma de Madrid ( España ) en 2023
• Idioma: inglés
• Número de páginas: 223
• Títulos paralelos:
  • Uso de modelos multinivel para considerar información no cognitiva en la evaluación: personalidad, omisiones y fallos de redacción
• Tribunal Calificador de la Tesis: Patricia Recio Saboya (presid.), Francisco José Abad García (secret.), Esther Ulitzsch (voc.)
• Calificación: Sobresaliente Cum Laude
• Materias:
  • Psicología
    • Evaluación y diagnostico en psicología
      • Teoría de la medición
      • Estadística
• Enlaces
  • Tesis en acceso abierto en: Biblos-e Archivo
• Resumen

  • The present dissertation focuses on understanding and analyzing the relationships that exist among various item characteristics, the personality of the examinees, and their test taking behavior, with the aim of enhancing evaluation procedures in various contexts. The significance of this research lies in its potential to shed light on factors that can influence the accuracy and fairness of evaluations, as well as in identifying potentia l predictors for individuals' future academic and professional outcomes. Regarding item characteristics, a fundamental aspect is how they are formulated and whether they include item writing flaws . This could lead to examinees obtaining lower scores than their actual level of knowledge would reflect. This issue is further compounded when considering factors such as personality and test taking behavior. For example, certain personality traits can influence how examinees approach tests, affecting their performance and leading them to omit responses or answer inconsistently. The us e of multilevel cross ed random effects models allows us to incorporate all this information within the same model, ensuring that all possible sources of variability in the data are considered. Furthermore, to avoid unfair or biased evaluations, it is essential to take into account these non cognitive factors. To achieve this, considering the information about examinees' behavior, particularly their tendency to omit, not only as n uisanc e but as a substantive factor, can provide insights into how examinees approach evaluation tasks. Therefore, the relationship between examinees' personality and their tendency to omit is investigated, with the goal of understanding what underlies this omis sion and h ow it may affect their future outcomes. Lastly, when this information provided by missing values is not treated as a substantive variable but as a factor that may bias the estimates, an appropriate treatment of missing values becomes necessary. However, analyzing the data with multilevel cross ed random effects models presents challenges in this regard. Classic M ultiple I mputation ( can be problematic due to the complexity of possible effects present and the non linear interaction between variables. To overcome this obstacle, a fully Bayesian ap proach is proposed, where the compatibility of the substantive model and the imputation model is ensured, taking into account non linear effects and interactions among different covariates, thereby allowing the inclusion of a more flexible and complex model