Using the MPEG query format for cross-modal identification

During the last years, the cross-modal search of visual and audio information has become more and more important. Using both domains, video and audio, turned out to be much more robust for the identification of video streams, than the visual part of the video stream alone. This paper describes a method for the audiovisual identification on remote databases using the MPQF. Additionally a service provider is deployed, which splits and aggregates the query and send them to two remote MPEG-7 databases (visual and audio) for identification. Among others a novel technique for the feature extraction on the service provider side is described, which is based on the MPQF. The interface between user and database is described in detail, examples are given and extensive results for the cross-modal search are presented.