Fraunhofer-Gesellschaft

Publica

Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Towards Guidelines for Assessing Qualities of Machine Learning Systems

 
: Siebert, Julien; Jöckel, Lisa; Heidrich, Jens; Nakamichi, Koji; Ohashi, Kyoko; Namba, Isao; Yamamoto, Rieko; Aoyama, Mikio

:

Shepperd, Martin (Ed.):
Quality of Information and Communications Technology. 13th International Conference, QUATIC 2020. Proceedings : Faro, Portugal, September 9-11, 2020, online conference
Cham: Springer Nature, 2020 (Communications in computer and information science 1266)
ISBN: 978-3-030-58792-5 (Print)
ISBN: 978-3-030-58793-2 (Online)
S.17-31
International Conference on the Quality of Information and Communications Technology (QUATIC) <13, 2020, Online>
Englisch
Konferenzbeitrag
Fraunhofer IESE ()
Machine learning; Software quality; Quality evaluation

Abstract
Nowadays, systems containing components based on machine learning (ML) methods are becoming more widespread. In order to ensure the intended behavior of a software system, there are standards that define necessary quality aspects of the system and its components (such as ISO/IEC 25010). Due to the different nature of ML, we have to adjust quality aspects or add additional ones (such as trustworthiness) and be very precise about which aspect is really relevant for which object of interest (such as completeness of training data), and how to objectively assess adherence to quality requirements. In this article, we present the construction of a quality model (i.e., evaluation objects, quality aspects, and metrics) for an ML system based on an industrial use case. This quality model enables practitioners to specify and assess quality requirements for such kinds of ML systems objectively. In the future, we want to learn how the term quality differs between different types of ML systems and come up with general guidelines for specifying and assessing qualities of ML systems.

: http://publica.fraunhofer.de/dokumente/N-603281.html