Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Rethinking the evaluation methodology of authorship verification methods

: Halvani, Oren; Graner, Lukas


Bellot, P.:
Experimental IR Meets Multilinguality, Multimodality, and Interaction : 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, September 10-14, 2018, Proceedings
Cham: Springer International Publishing, 2018 (Lecture Notes in Computer Science 11018)
ISBN: 978-3-319-98931-0 (Print)
ISBN: 978-3-319-98932-7 (Online)
International Conference of the Cross-Language Evaluation Forum for European Languages (CLEF) <9, 2018, Avignon>
Bundesministerium für Bildung und Forschung BMBF
Fraunhofer SIT ()

Authorship verification (AV) concerns itself with the task to judge, if two or more documents have been written by the same person. Even though an increase of research activities in the last years can be observed, it can also be clearly seen that AV suffers of well-defined evaluation standards. Based on a comprehensive literature review of more than 50 research works including conference papers, journals, bachelor's/master's theses and doctoral dissertations, we could not identify consistent evaluation procedures that adequately reflect the reliability of AV methods. To counteract this, we propose an alternative evaluation methodology based on the construction of reliable corpora in combination with a more suitable performance measure. In an experimental setup our approach reveals the weakness of a number of existing and successful AV methods, in particular, when it comes to accept as many documents of the true author, while at the same time reject as many documents of other authors, as possible.