An Interactive Framework for Cross-modal Attribute-based Person Retrieval

Person re-identification systems generally rely on a query person image to find additional occurrences of this person across a camera network. In many real-world situations, however, no such query image is available and witness testimony is the only clue upon which to base a search. Cross-modal re-identification based on attribute queries can help in such cases but currently yields a low matching accuracy which is often not sufficient for practical applications. In this work we propose an interactive feedback-driven framework, which successfully bridges the modality gap and achieves a significant increase in accuracy by 47% in mean average precision (mAP) compared to the fully automatic cross-modal state-of-the-art. We further propose a cluster-based feedback method as part of the framework, which outperforms naïve user feedback by more than 9% mAP. Our results set a new state-of-the-art for fully automatic and feedback-driven cross-modal attribute-based re-identification on two public datasets.