An Approach for Data Pipeline with Distributed Query Engine for Industrial Applications

The data driven services in industrial automation systems are transforming the world of automation industry by optimizing industrial processes and providing Value Added Services (VASs) with the grace of Industry 4.0, Big Data and Artificial Intelligence (AI). A demand driven data pipeline is essential to connect different industrial data sources in a shop floor with different data storage systems for service provisioning. This paper analyzes an experimental approach and corresponding challenges to optimize computing resource allocation in industrial applications to construct such demand driven data pipeline to provide data driven services through an open source, flexible and extensible distributed query engine known as Presto, which can perform interactive analytical queries for different purposes such as condition monitoring, asset management or many others.