Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

An architecture to provide adaptive, synchronized and multimodal human computer interaction

: Blechschmitt, E.; Strödecke, C.


Association for Computing Machinery -ACM-, Special Interest Group on Multimedia -SIGMM-; Association for Computing Machinery -ACM-, Special Interest Group on Graphics -SIGGRAPH-; Association for Computing Machinery -ACM-, Special Interest Group on Data Communication -SIGCOMM-:
ACM Multimedia 2002. Proceedings : 10th International Conference on Multimedia, Juan-les-Pins, December 1 - 6, 2002
New York: ACM, 2002
ISBN: 1-58113-620-X
International Conference on Multimedia <10, 2002, Juan-les-Pins>
Conference Paper
Fraunhofer IGD ()
multimodal interaction; synchronization; mobile agent; dialog move

In this paper we present a solution to compose synchronized multimodal computer human interaction with a pattern-oriented approach. The focus of this paper is how to synchronize the different modalities of interaction. In our application scenario the system consists of mobile agents and the user interface is generated from a dialog description language, which is XML-encoded. The generated User Interface is executed by one or more UI-engines, which can use one or more modalities like a text chat system, a speech-based system or a graphical, window-oriented user interface. The user interface is separated and also structured by so-called dialog moves which are suitable to be adapted to different UI-engines. The system supports more than one UI-engine at the same time which results in a possibly multimodal interaction. The interaction is synchronized by observing the events produced by the dialog moves of each UI-engine. The system is evaluated in a project called MAP1 which deals with new human computer interaction methods in future mobile work environments.