• English
  • Deutsch
  • Log In
    or
  • Research Outputs
  • Projects
  • Researchers
  • Institutes
  • Statistics
Repository logo
Fraunhofer-Gesellschaft
  1. Home
  2. Fraunhofer-Gesellschaft
  3. Konferenzschrift
  4. Proxy Indicators for the Quality of Open-domain Dialogues
 
  • Details
  • Full
Options
2021
Conference Paper
Titel

Proxy Indicators for the Quality of Open-domain Dialogues

Abstract
The automatic evaluation of open-domain dialogues remains a largely unsolved challenge. Despite the abundance of work done in the field, human judges have to evaluate dialogues quality. As a consequence, performing such evaluations at scale is usually expensive. This work investigates using a deep-learning model trained on the General Language Understanding Evaluation (GLUE) benchmark to serve as a quality indication of open-domain dialogues. The aim is to use the various GLUE tasks as different perspectives on judging the quality of conversation, thus reducing the need for additional training data or responses that serve as quality references. Due to this nature, the method can infer various quality metrics and can derive a component-based overall score. We achieve statistically signif icant correlation coefficients of up to 0.7.
Author(s)
Nedelchev, Rostislav
Lehmann, Jens
Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Usbeck, Ricardo
Hauptwerk
Conference on Empirical Methods in Natural Language Processing, EMNLP 2021. Proceedings
Project(s)
SPEAKER
JOSEPH
Cleopatra
ML2R
ScaDS.AI
TAILOR
Funder
Bundesministerium für Wirtschaft und Energie BMWi (Deutschland)
Fraunhofer-Gesellschaft FhG
European Commission EC
Bundesministerium für Bildung und Forschung BMBF (Deutschland)
Bundesministerium für Bildung und Forschung BMBF (Deutschland)
European Commission EC
Konferenz
Conference on Empirical Methods in Natural Language Processing (EMNLP) 2021
Thumbnail Image
Externer Link
Externer Link
Language
English
google-scholar
Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
  • Cookie settings
  • Imprint
  • Privacy policy
  • Api
  • Send Feedback
© 2022