1 - 10 of 44
-
PublicationDeutsche Normungsroadmap Künstliche Intelligenz(DIN, 2022-12-09)
-
PublicationConstruction of a quality model for machine learning systems( 2022)Nowadays, systems containing components based on machine learning (ML) methods are becoming more widespread. In order to ensure the intended behavior of a software system, there are standards that define necessary qualities of the system and its components (such as ISO/IEC 25010). Due to the different nature of ML, we have to re-interpret existing qualities for ML systems or add new ones (such as trustworthiness). We have to be very precise about which quality property is relevant for which entity of interest (such as completeness of training data or correctness of trained model), and how to objectively evaluate adherence to quality requirements. In this article, we present how to systematically construct quality models for ML systems based on an industrial use case. This quality model enables practitioners to specify and assess qualities for ML systems objectively. In addition to the overall construction process described, the main outcomes include a meta-model for specifying quality models for ML systems, reference elements regarding relevant views, entities, quality properties, and measures for ML systems based on existing research, an example instantiation of a quality model for a concrete industrial use case, and lessons learned from applying the construction process. We found that it is crucial to follow a systematic process in order to come up with measurable quality properties that can be evaluated in practice. In the future, we want to learn how the term quality differs between different types of ML systems and come up with reference quality models for evaluating qualities of ML systems.
-
PublicationRequirements-Driven Method to Determine Quality Characteristics and Measurements for Machine Learning Software and Its Evaluation( 2020)As the applications of machine learning algorithms in various fields are widely demanded, the development of machine learning software systems (MLS) is rapidly increasing. The quality of MLS is different from that of conventional software systems, in the sense that it depends on the amount and distribution of training data in a model learning and input data during operation. This is a major challenge in quality assurance of MLS development for the enterprise. In this paper, we propose a requirements-driven method to determine the quality characteristics of the MLS. Major contributions of this paper include: (1) Extending the quality characteristics of ISO 25010, which defines the conventional software quality, to those unique to MLS; this paper also defines its measuring method. (2) A method to identify requirements, i.e., issues to be determined in the requirements definition, in order to derive the quality characteristics and measurement methods for MLS, since the quality characteristics and the measurement method depend on the goals of the system under development. In order to evaluate the proposed method, we carried out an empirical study of the quality characteristics and measurement methods related to functional correctness and the maturity of the MLS for the enterprise. Based on the study, we compare the quality characteristics and measurement methods derived by the proposed method with those suggested by developers, and demonstrate the effectiveness of the proposed method.
-
PublicationTowards Guidelines for Assessing Qualities of Machine Learning Systems( 2020)Nowadays, systems containing components based on machine learning (ML) methods are becoming more widespread. In order to ensure the intended behavior of a software system, there are standards that define necessary quality aspects of the system and its components (such as ISO/IEC 25010). Due to the different nature of ML, we have to adjust quality aspects or add additional ones (such as trustworthiness) and be very precise about which aspect is really relevant for which object of interest (such as completeness of training data), and how to objectively assess adherence to quality requirements. In this article, we present the construction of a quality model (i.e., evaluation objects, quality aspects, and metrics) for an ML system based on an industrial use case. This quality model enables practitioners to specify and assess quality requirements for such kinds of ML systems objectively. In the future, we want to learn how the term quality differs between different types of ML systems and come up with general guidelines for specifying and assessing qualities of ML systems.
-
PublicationData Science: Technologies for Better Software( 2019)Data science is mandatory in today's business to capitalize on achievements and assets. This specifically holds for modern software development, where data science facilitates analyzing product, process, and usage and thus managing evolution and performance. With the convergence of embedded and IT domains, such as the Internet of Things (IoT) and automotive systems, software systems are becoming more complex. Complexity has two faces. On one hand it means more functionality and fluid delivery models, thus offering markets more value, such as the ability to deliver a single-customer focus. Complexity, however, also means the growth of technical debt, which slows productivity and lowers quality. As software engineering generates ever larger and more varied data sets, such as feature usage, code analysis, test coverage, error logs, and maintenance data, companies face the challenge of unlocking the value of that data.
-
PublicationFuture skills: Ansätze zur Vermittlung von Data Literacy in der Hochschulbildung(Edition Stifterverband, 2018)Die Fähigkeit, planvoll mit Daten umzugehen und sie im jeweiligen Kontext bewusst einsetzen und hinterfragen zu können wird im Zuge der digitalen Transformationen von zunehmender Wichtigkeit und stellt eine zentrale Kompetenz in allen Sektoren und Disziplinen dar. Auf der einen Seite werden Data Scientists benötigt, die in der Lage sind, speziell mit großen heterogenen Daten umzugehen und die Technologie rund um den Big-Data-Lifecycle beherrschen, um schnell Entscheidungen basierend auf Daten und daraus abgeleiteten Informationen ermöglichen zu können. Auf der anderen Seite werden in der Breite in allen Sektoren und Disziplinen Personen benötigt, welche die Fähigkeit besitzen, Daten auf kritische Art und Weise zu sammeln, zu managen, zu bewerten und anzuwenden. Diese Fähigkeiten werden unter dem Begriff Data Literacy zusammengefasst. Im Auftrag der Arbeitsgruppe Curriculum 4.0 des Hochschulforums Digitalisierung führten das Fraunhofer-Institut für Experimentelles Software-Engineering IESE und die Gesellschaft für Informatik eine Studie durch, um umsetzbares Wissen für Hochschulen und Fächer für die Curriculum-Entwicklung im Hinblick auf Data Literacy zusammenzustellen. Der Fokus lag dabei auf europäischen und internationalen Best-Practice-Beispielen, welche Angebote zur bedarfsgerechten, disziplinübergreifenden Vermittlung von Wissen zur datengestützten Arbeit und Entscheidungsfindung aufgebaut haben. Im Vordergrund stand die Wissensvermittlung von Data Literacy in den Anwendungsdomänen und nicht die Ausbildung von Data Scientists. Im Rahmen der Studie wurden dazu Best-Practice-Beispiele recherchiert, essentielle Literaturquellen und existierende Studien analysiert, Interviews mit Fachexperten und eine Online-Umfrage durchgeführt sowie mögliche Handlungsempfehlungen in einem internationalen Experten-Workshop erarbeitet. Basierend auf den Studienergebnissen lassen sich eine Reihe von Herausforderungen und Maßnahmen zusammenfassen, die im vorliegenden Bericht im Detail beschrieben werden. Hierbei sei angemerkt, dass die Maßnahmen einen teils heterogenen Lösungsraum mit verschiedenen Optionen aufspannen, die sich in einem bestimmten Anwendungskontext bewährt haben. Auf dieser Basis lässt sich zwar noch nicht direkt ein homogenes Maßnahmenpaket ableiten, aber die Studie liefert in den essentiellen Bereichen Anregungen, konkrete Lösungsbausteine und Praxisbeispiele, die von Hochschulakteuren und der Politik, als Grundlage genutzt werden können, um Curricula im Hinblick auf Data Literacy erfolgreich zu gestalten bzw. die richtigen Anreize und Strukturen dafür zu schaffen. Damit liefern die Studienergebnisse einen wichtigen Beitrag, um die für die digitale Transformation essentiellen Kompetenzen in Data Literacy nachhaltig aufbauen zu können.
-
-
PublicationExploiting Big Data's Benefits( 2016)Knowing about big data's potential for exploiting new business ideas is a key capability for staying successful in the market. Potential analysis provides a systematic way to identify and close the gap between big data's possible benefits and the ability to turn that data into business value.
-
-
PublicationModel-based quality management of software development projects( 2014)Managing the quality of artifacts created during the development process is an integral part of software project management. Software quality models capture the knowledge and experience regarding the quality characteristics of interest, the measurement data that can help to reason about them, and the mechanisms to use for characterizing and assessing software quality. They are the foundation for managing software quality in projects in an evidence-based manner. Nowadays, coming up with suitable quality models for an organization is still a challenging endeavor. This chapter deals with the definition and usage of software quality models for managing software development projects and discusses different challenges and solutions in this area. The challenges are: (1) There is no universal model that can be applied in every environment because quality is heavily dependent on the application context. In practice and research, a variety of different quality models exists. Finding the "right" model requires a clear picture of the goals that should be obtained from using the model. (2) Quality models need to be tailored to company specifics and supported by corresponding tools. Existing standards (such as the ISO/IEC 25000 series) are often too generic and hard to fully implement in an organization. (3) Practitioners require a comprehensive set of techniques, methods, and tools for systematically specifying, adapting, and applying quality models in practice. (4) In order to create sustainable quality models, their contribution to the organizational goals must be clarified, and the models need to be integrated into the development and decision-making processes.
- 1 (current)
- 2
- 3
- 4
- 5