Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Challenges of designing hand recognition for a manual assembly assistance system

: Root, Martin; Jauch, Christian

Fulltext urn:nbn:de:0011-n-5525080 (857 KByte PDF)
MD5 Fingerprint: 742875ef105d93bda7a6dfd398ea32a8
Copyright Society of Photo-Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.
Created on: 30.7.2019

Stella, Ettore ; Society of Photo-Optical Instrumentation Engineers -SPIE-, Bellingham/Wash.; European Optical Society -EOS-:
Multimodal Sensing: Technologies and Applications : 24-27 June 2019, Munich, Germany
Bellingham, WA: SPIE, 2019 (Proceedings of SPIE 11059)
Paper 110590R, 10 pp.
Conference "Multimodal Sensing - Technologies and Applications" <2019, Munich>
Optical Metrology Conference <2019, Munich>
Conference Paper, Electronic Publication
Fraunhofer IPA ()
Assistenzsystem; Handbetätigung; manuelle Montage; maschinelles Lernen; Positionsbestimmung; Segmentierung; Hand; Lokalisation; Lokalisation

Manual assembly remains an important task in production with changing requirements for the worker due to i.e. mass personalized products. To keep up with these requirement changes, modern assistance systems are suitable to support the workers. The goals of these assistance systems are to detect errors in the process, guide the worker through new processes and document the process. To achieve this, the assistance system needs to follow each step of the worker reliably. This can be realized with a visual scene analysis based on machine learning. In the presented work, one 3D-sensor (active stereo vision with an additional RGB camera) is used for the scene analysis. The presented work evaluates existing methods for hand localization and hand pose recognition for the application in manual assembly assistance systems. A new procedure for hand localization based on current machine learning techniques is developed specifically for the scenario of recognizing hand joints (also called hand pose) in a manual assembly scenario. Additionally, different methods for hand pose recognition are compared in the application scenario. Based on the results of the developed hand localization method, current hand pose detection methods were evaluated (Dense-Regression and DeepPrior++). While the methods worked well in a scenario comparable to the dataset, they did not perform very well in a manual assembly scenario. Improvements can be made using pixel-wise segmentation or using specific datasets for training containing data from manual assembly scenarios.