What students ask matters: LLM interaction depth, task quality, and immediate recall in higher education

• Autores: Vangelis Tsiligkiris
• Localización: International Journal of Educational Technology in Higher Education, ISSN 2365-9440, Nº. 23, 2026
• Idioma: inglés
• Enlaces
  • Texto Completo Ejemplar
• Resumen

  • Large Language Models (LLMs) are rapidly transforming higher education, yet evidence on how interaction patterns affect learning remains limited. This study examines whether explanation-seeking dialogue with an LLM is associated with task quality and immediate recall in a controlled learning session. Twenty-two postgraduate students completed a pre-test, an LLM-assisted neuroeconomics case task, and an immediate post-test. Fine-grained interaction logs captured per-turn telemetry, enabling extraction of Depth (proportion of "why/how/explain" prompts), Volume, and Pacing features. Results showed large immediate pre–post score gains (Cohen’s dz = 2.12, p < .001). In the task-quality regression model, Depth was positively associated with task quality beyond baseline and Volume (β = 6.27, p = .006), indicating that a one-standard-deviation increase in explanation-seeking prompts was associated with approximately six additional marks on a 0–100 scale. Depth was not associated with immediate recall (β =  − 0.014, p = .728); instead, gain scores were strongly associated with baseline knowledge, consistent with reduced headroom for improvement among students starting from a higher pre-test score (β =  − 0.161, p < .001). The findings indicate a dissociation between performance quality and short-term recall in LLM-supported study. This aligns with cognitive-psychology evidence that elaboration improves comprehension while retrieval practice consolidates retention. Pedagogically, the dissociation suggests that depth-oriented dialogue may need to be paired with deliberate memory-strengthening activities such as self-testing or spaced retrieval if comprehension gains are to translate into recall, although the present design does not test such activities directly. Methodologically, it contributes a replicable, privacy-preserving instrumentation pipeline linking conversational telemetry to learning outcomes. Limitations include single-group design, small sample (n = 22), immediate testing only, and keyword-based depth proxies. Future work should randomise assistance styles, incorporate delayed retention tests, and refine depth measurement through semantic coding.