Temporally tolerant video matching

A temporally tolerant notion of visual similarity for videos is defined in the domain of television advertisements. This notion encompasses robustness to channel and encoding noise, encoding rate, and reediting of the advertisements. Using a color-based visual feature as a basis for describing the frames of the video, a sub-shot segmentation of the video that is consistent across encoding rates and extensible to streaming media is produced. This segmentation is subsequently utilized in a similarity matrix based matching algorithm that effectively matches temporally reencoded and reedited videos. Experimental matching results for the situation of discrete video files are given. The method is developed to be fully extensible to any domain with a continuous media stream.