A Trustworthy Automated Short-Answer Scoring System Using a New Dataset and Hybrid Transfer Learning Method

• Martinus Maslim ^[1] ; Hei-Chia Wang ^[1] ; Cendra Devayana Putra ^[1] ; Yulius Denny Prabowo ^[2]
  1. [1] National Cheng Kung University

     National Cheng Kung University

     Taiwán

     
  2. [2] Bina Nusantara University
• Localización: IJIMAI, ISSN-e 1989-1660, Vol. 8, Nº. 5, 2024, págs. 37-45
• Idioma: inglés
• Enlaces
  • Texto completo
• Resumen

  • To measure the quality of student learning, teachers must conduct evaluations. One of the most efficient modes of evaluation is the short answer question. However, there can be inconsistencies in teacher-performed manual evaluations due to an excessive number of students, time demands, fatigue, etc. Consequently, teachers require a trustworthy system capable of autonomously and accurately evaluating student answers. Using hybrid transfer learning and student answer dataset, we aim to create a reliable automated short answer scoring system called Hybrid Transfer Learning for Automated Short Answer Scoring (HTL-ASAS). HTL-ASAS combines multiple tokenizers from a pretrained model with the bidirectional encoder representations from transformers. Based on our evaluation of the training model, we determined that HTL-ASAS has a higher evaluation accuracy than models used in previous studies. The accuracy of HTL-ASAS for datasets containing responses to questions pertaining to introductory information technology courses reaches 99.6%. With an accuracy close to one hundred percent, the developed model can undoubtedly serve as the foundation for a trustworthy ASAS system.