• English
  • Deutsch
  • Log In
    Password Login
    Research Outputs
    Fundings & Projects
    Researchers
    Institutes
    Statistics
Repository logo
Fraunhofer-Gesellschaft
  1. Home
  2. Fraunhofer-Gesellschaft
  3. Scopus
  4. Datastack: Unification of Heterogeneous Machine Learning Dataset Interfaces
 
  • Details
  • Full
Options
2022
Conference Paper
Title

Datastack: Unification of Heterogeneous Machine Learning Dataset Interfaces

Abstract
Machine learning (ML) dataset preprocessing, cleaning, and integration into ML pipelines is often a cum-bersome endeavor that is susceptible to bugs and leads to unstructured code from the start. While existing frameworks for dataset integration often come with an extensive dataset repository, extending these repositories to new datasets is nontrivial due to lack of dataset retrieval, processing and iterator separation. To simplify the process of dataset integration, we present Datastack, an open-source framework that minimizes these efforts by providing well-defined interfaces that seamlessly integrate into existing machine learning frameworks. Inspired by stream processing frameworks such as Flink or Storm, Datastack decouples dataset-specific peculiarities such as custom data formats from the framework by introducing byte streams on an interface level. Furthermore, Datastack delivers dataset preprocessing functionalities such as stacking, splitting, and merging to alleviate error-prone data processing pipelines.
Author(s)
Lübbering, Max  
Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS  
Pielka, Maren  
Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS  
Henk, Ilhamcengiz
Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS  
Sifa, Rafet  
Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS  
Mainwork
IEEE 38th International Conference on Data Engineering Workshops, ICDEW 2022. Proceedings  
Conference
International Conference on Data Engineering 2022  
International Workshop on Databases and Machine Learning 2022  
DOI
10.1109/ICDEW55742.2022.00014
Language
English
Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS  
Keyword(s)
  • dataset integration

  • dataset processing

  • stream processing

  • Cookie settings
  • Imprint
  • Privacy policy
  • Api
  • Contact
© 2024