Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB

Filters

Reset filters

Settings
Now showing 1 - 10 of 19
No Thumbnail Available
Publication

Deep Learning-Based Action Detection for Continuous Quality Control in Interactive Assistance Systems

2023 , Besginow, Andreas , Büttner, Sebastian , Ukita, Norimichi , Röcker, Carsten

Interactive assistance systems have shown to be useful in various industrial settings, in particular those involving human labor like manual assembly of workpieces. Current systems support workers based on different technologies like projection-based augmented reality, hand or tool tracking or automated inspections using computer vision techniques. While these technologies help to increase product quality significantly, existing solutions are not able to monitor the entire process, which makes it difficult to detect process errors. In this paper, we present a deep-learning based approach for continuous on-the-fly quality control within an interactive assistance system. By using labeled video data of an assembly process, a model can be trained that automatically recognizes and distinguishes single actions and thus control the sequence of subsequent work processes. By integrating the system into the interactive assistance systems, users are made aware on any process errors. Besides presenting the concept and implementation of our deep-learning integration into the assistance system, we describe the created industrial assembly-oriented dataset and present the results from our technical evaluation that shows the potential of applying deep-learning methods into interactive assistance systems.

View
No Thumbnail Available
Publication

One-Hand Controller for Human-Drone Interaction - a Human-Centered Prototype Development

2020 , Büttner, Sebastian , Zaitoon, Rami , Heinz, Mario , Röcker, Carsten

Using remote control transmitters is a common way to control a drone. For the future, we envision drones that are intuitively controllable with new input devices. One possibility could be the use of one-hand controllers. Here, we present an exploration of using a 3-D mouse as a controller for human-drone interaction. We ran a pre-study that investigated the users' natural spatial mapping between controller and drone dimensions. Based on these results we developed our prototype that shows the feasibility of our concept. A series of flight tests were conducted and the mapping between controller and flight movements were iteratively improved. In this paper, we present our development process and the implementation of our prototype.

View
No Thumbnail Available
Publication

A multi-level localization system for intelligent user interfaces

2018 , Heinz, Mario , Büttner, Sebastian , Wegerich, Martin , Marek, Frank , Röcker, Carsten

The localization of employees in the industrial environment plays a major role in the development of future intelligent user interfaces and systems. Yet, localizing people also raises ethical, legal and social issues. While a precise localization is essential for context-aware systems and real-time optimization of processes, a permanently high localization accuracy creates opportunities for surveillance and therefore has a negative impact on workplace privacy. In this paper, we propose a new concept of a multi-level localization system which tries to find a way to meet both the technical requirements for a localization with a high accuracy as well as the interests of employees in terms of privacy. Depending on the users-location, different localization technologies are used, that restrict the accuracy to the least required level by design. Furthermore, we present a prototypical implementation of the concept that shows the feasibility of our multi-level localization concept. Using this system, intelligent systems become able to react on employees based on their location without permanently monitoring the precise user location.

View
No Thumbnail Available
Publication

Exploring design opportunities for intelligent worker assistance: A new approach using projetion-based AR and a novel hand-tracking algorithm

2017 , Büttner, Sebastian , Sand, Oliver , Röcker, Carsten

This paper presents a prototype of an intelligent assistive system for workers in stationary manual assembly using projection-based augmented reality (AR) and intelligent hand tracking. By using depth cameras, the system can track the hands of the user and makes the user aware of wrong picking actions or errors in the assembly process. The system automatically adapts the digital projection-based overlay according to the current work situation. The main research contribution of our work is the presentation of a novel hand-tracking algorithm. In addition, we present the results of an user study of the system that shows the challenges and opportunities of our system and the hand-tracking algorithm in particular. We assume that our results will inform the future design of assistive systems in manual assembly.

View
No Thumbnail Available
Publication

Teaching by Demonstrating - How Smart Assistive Systems can Learn from Users

2020 , Büttner, Sebastian , Peda, Andreas , Heinz, Mario , Röcker, Carsten

Projection-based assitive systems that guide users through assembly work are on their way to industrial application. Previous research work investigated how people can be supported with such systems. However, there has been little work on the question on how to generate and author sequential instructions for assitive systems. In this paper, we present a new concept and a prototypical implementation of an assitive system that can be taught by demonstrating an assembly process. By using a combination of RGB and depth cameras, we can generate an assembly instruction of Lego Duplo bricks based on the demonstration of a user. This generated manual can later on be used for assisting other users in the assembly process. By our prototype system, we show the technological feasibility of assistive systems that can learn from users.

View
No Thumbnail Available
Publication

Exploring Users' Eye Movements when Using Projection-Based Assembly Assistive Systems

2020 , Heinz, Mario , Büttner, Sebastian , Röcker, Carsten

Projection-based assistive systems have shown to be a promising technology to support workers during manual assembly processes in industrial manufacturing by projecting instructions into the working area. While existing studies have investigated various aspects of these systems, little research has been conducted regarding the way in which the user accesses the provided instructions. In this paper we analyze the eye movements of users during the repeated execution of an assembly task at a projection-based assistive system in order to gain insights into the utilization of the presented instructions. For this purpose, we analyzed eye tracking recordings from a user study with 15 participants to investigate the sequences in which the respective instructions are observed by the users. The results show a significantly lower number of nonlinear gaze sequences as well as a significantly higher number of steps without observing the instructions during the repeated use of the assistive system. In addition, there was a significantly lower task completion time during repeated use of the assistive system.

View
No Thumbnail Available
Publication

A checklist based approach for evaluating augmented reality displays in industrial applications

2018 , Paelke, Volker , Büttner, Sebastian , Mucha, Henrik , Röcker, Carsten

The selection of suitable display technologies for industrial augmented reality (AR) applications is becoming increasingly relevant as such applications move from the proof-of-concept to the application stage. To support project managers, designers and developers in the critical selection process we have developed a checklist of important aspects and related evaluation hints that helps to speed up and improve the selection process. The checklist presented in this paper was designed to be useful for both researchers and practitioners. It combines pertinent information from relevant standards like ISO 9241-210 with results from current research literature and experience from several AR projects in industrial contexts. It can be applied both in collaboration with AR experts, where it helps to prepare relevant information for the collaboration and thus streamlines the process, or stand-alone, as a guideline for the evaluation of different options by a design team.

View
No Thumbnail Available
Publication

Making Object Detection Available to Everyone - A Hardware Prototype for Semi-automatic Synthetic Data Generation

2020 , Besginow, Andreas , Büttner, Sebastian , Röcker, Carsten

The capabilities of object detection are well known, but many projects don't use them, despite potential benefit. Even though the use of object detection algorithms is facilitated through frameworks and publications, a big issue is the creation of the necessary training data. To tackle this issue, this work shows the design and evaluation of a prototype, which allows users to create synthetic datasets for object detection in images. The prototype is evaluated using YOLOv3 as the underlying detector and shows that the generated datasets are equally good in quality as manually created data. This encourages a wide adoption of object detection algorithms in different areas, since image creation and labeling is often the most time consuming step.

View
No Thumbnail Available
Publication

Handling Work Complexity with AR/Deep Learning

2019 , Dhiman, Hitesh , Büttner, Sebastian , Röcker, Carsten , Reisch, Raphael

Complexity is a fundamental part of product design and manufacturing today, owing to increased demands for customization and advances in digital design techniques. Assembling and repairing such an enormous variety of components means that workers are cognitively challenged, take longer to search for the relevant information and are prone to making mistakes. Although in recent years deep learning approaches to object recognition have seen rapid advances, the combined potential of deep learning and augmented reality in the industrial domain remains relatively under explored. In this paper we introduce AR-ProMO, a combined hardware/software solution that provides a generalizable assistance system for identifying mistakes during product assembly and repair.

View
No Thumbnail Available
Publication

Towards a framework for assistance systems to support work processes in smart factories

2017 , Fellmann, Michael , Robert, Sebastian , Büttner, Sebastian , Mucha, Henrik , Röcker, Carsten

Increasingly, production processes are enabled and controlled by Information Technology (IT), a development being also referred to as ""Industry 4.0"". IT thereby contributes to flexible and adaptive production processes, and in this sense factories become ""smart factories"". In line with this, IT also more and more supports human workers via various assistance systems. This support aims to both support workers to better execute their tasks and to reduce the effort and time required when working. However, due to the large spectrum of assistance systems, it is hard to acquire an overview and to select an adequate system for a smart factory based on meaningful criteria. We therefore synthesize a set of comparison criteria into a consistent framework and demonstrate the application of our framework by classifying three examples.

View