Resumen de Automating the knowledge assessment workflow for large student groups: A development experience

Andrija Bošnjaković, Jelica Protic, Dragan Bojić, Igor Tartalja

• The paper presents our long-term experience at the University of Belgrade—School of Electrical Engineering in thedevelopment and use of the software system calledTest, designed to automate the knowledge assessment workflow forlarge student groups in situations that usually require the use of pencil-and-paper testing. The test assembly is doneinteractively and/or automatically from a database of problems (questions, coding assignments, etc.) and previous tests. Inorder to enable objective and uniform knowledge tests, a rich set of parameterized problem selection criteria is madeavailable to teachers. Preparation, scanning, and analysis of bubble sheets are highly automated and designed to work withplain paper and regular scanners. TheTestsystem integrates with relevant learning management systems and the school’sinformation system. Quantitative measurements have indicated improvements in both test quality of individual tests anduniformity of test quality across multiple tests, compared to the tests assembled manually according to intuitively appliedcriteria. The system is used for the introductory programming courses at the School. It has been developed through anumber of diploma theses. Overall experience showed that the designed system provides an appropriate automation tool, auseful educational platform, and a valuable base for further research.