Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Blu: What GUIs are made of

: Sermuga Pandian, Vinoth Pandian; Suleri, Sarah; Jarke, Matthias


Association for Computing Machinery -ACM-; Association for Computing Machinery -ACM-, Special Interest Group on Computer and Human Interaction -SIGCHI-:
IUI 2020, 25th International Conference on Intelligent User Interfaces. Proceedings : March 17-20, 2020, Cagliari Italy
New York: ACM, 2020
ISBN: 978-1-4503-7513-9
International Conference on Intelligent User Interfaces (IUI) <25, 2020, Cagliari>
Conference Paper
Fraunhofer FIT ()

UI designers look for inspirational examples from existing UI designs during the prototyping process. However, they have to reconstruct these example UI designs from scratch to edit content or apply styling. The existing solution attempts to make UI screens into editable vector graphics using image segmentation techniques. In this research, we aim to use deep learning and gestalt laws-based algorithms to convert UI screens to editable blueprints by identifying the constituent UI element categories, their location, dimension, text content, and layout hierarchy. In this paper, we present a proof-of-concept web application that uses the UI screens and annotations from the RICO dataset and generates an editable blueprint vector graphic, and a UI layout tree. With this research, we aim to support UX designers in reconstructing UI screens and communicating UI layout information to developers.