Model-based dialogue evaluation for dialogue systems in vehicles

The research work in the EMMIS project (evaluation of Man-Machine Interface by Simulation techniques) has the main objective to specify and develop evaluation methods and tools for the human-machine interfaces of ATT (Advanced road and Transport Telematics) functions. The activity is based on the application and integration of simulation methods, namely driving simulators, rapid prototyping techniques and evaluation tools. Whereas a lot of useful tools exist for the domain of designing human-machine dialogues, there is a deficiency in supporting the evaluation of dialogue interfaces. The paper describes a methodology for the model-based evaluation of human-machine dialogues with regard to the specific conditions in vehicles. The methodology is based on an approach modelling the driver's task-related knowledge by means of production systems. Cognitive complexity measures of drivervehicle interactions, such as visual distraction, learning and execution time are derived. The integration o f design and evaluation is achieved by generating the production system models of the driver's tasks automatically when the tasks are executed with a prototyped dialogue system.