Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Cross-modal identification of audiovisual streams directly from the compressed domain

: Gruhne, M.; Dunker, P.; Mikhalev, A.; Fedotov, I.; Andritsopoulos, F.


Institute of Electrical and Electronics Engineers -IEEE-:
IEEE 13th International Symposium on Consumer Electronics, ISCE 2009. Proceedings. Vol.2 : Kyoto, Japan, 25-28 May 2009
Piscataway, NJ: IEEE, 2009
ISBN: 1-4244-2975-7
ISBN: 978-1-4244-2975-2
International Symposium on Consumer Electronics (ISCE) <13, 2009, Kyoto>
Fraunhofer IDMT ()
audio visual system; data compression; decoding; feature extraction; information retrieval; media streaming

During the last years, a number of search and retrieval methods for audio and visual content were described in literature. Also cross-modal approaches started to emerge recently. All search methods are based on audiovisual fingerprints, which are extracted from the audiovisual data prior to the actual search. Since the most data are available in the compressed domain, they must be decompressed prior to the feature extraction. This paper describes an audiovisual search engine, which extracts their features directly from the compressed domain, without performing a decoding algorithm. The direct video feature extraction is based on motion vectors and the direct audio feature extraction is based on a polyphase matrix description. The paper depicts an overview of the search engine, which operates cross-modal on audio, and visual features and describes the evaluation of the direct feature extraction methods in detail.