Fraunhofer-Gesellschaft

Publica

Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

The timeliness deviation: A novel approach to evaluate educational recommender systems for closed-courses

 
: Krauss, Christopher; Merceron, Agathe; Arbanowski, Stefan

:

Association for Computing Machinery -ACM-:
9th International Conference on Learning Analytics & Knowledge, LAK 2019. Proceedings : Learning analytics to promote inclusin and success, Tempe, AZ, USA, March 04 - 08, 2019
New York: ACM, 2019
ISBN: 978-1-4503-6256-6
pp.195-204
International Conference on Learning Analytics and Knowledge (LAK) <9, 2019, Tempe/Ariz.>
Bundesministerium für Bildung und Forschung BMBF (Deutschland)
01PD14002D
Smart Learning - Medieneinsatz in der Online Weiterbildung
Bundesministerium für Bildung und Forschung BMBF (Deutschland)
01PD17002D
Smart Learning - Medieneinsatz in der Online Weiterbildung
English
Conference Paper
Fraunhofer FOKUS ()

Abstract
The decision on what item to learn next in a course can be supported by a recommender system (RS), which aims at making the learning process more efficient and effective. However, learners and learning activities frequently change over time. The question is: how are timely appropriate recommendations of learning resources actually evaluated and how can they be compared?
Researchers have found that, in addition to a standardized dataset definition, there is also a lack of standardized definitions of evaluation procedures for RS in the area of Technology Enhanced Learning. This paper argues that, in a closed-course setting, a time-dependent split into the training set and test set is more appropriate than the usual cross-validation to evaluate the Top-N recommended learning resources at various points in time. Moreover, a new measure is introduced to determine the timeliness deviation between the point in time of an item recommendation and the point in time of the actual access by the user. Different recommender algorithms, including two novel ones, are evaluated with the time-dependent evaluation framework and the results, as well as the appropriateness of the framework, are discussed.

: http://publica.fraunhofer.de/documents/N-537619.html