CANDOR: Centroid-Adjusted Normalized Decision-Level Optimization and Recognition for Blended Emotions

Recognizing blended emotions in audiovisual data is important for various applications such as targeted advertising, social media analysis, and deepfake detection, yet many estimation models still assume that each clip expresses a single dominant emotion. The Blended Emotion Recognition Challenge (BlEmoRe) addresses this limitation with a multimodal benchmark where systems must predict both the presence of one or two emotions and their relative salience. In this paper, we present CANDOR, a set of innovations that address three structural obstacles often apparent in conventional estimation pipelines taking advantage of a fusion of pre-trained features: (i) pre-trained features encode actor identity far more strongly than emotion content, (ii) feature-level concatenation reduces per-model discriminative power, and (iii) the training data exhibits a regular per-actor class distribution that can serve as an inductive bias. Therefore, CANDOR introduces identity normalization via unsupervised centroid subtraction, decision-level fusion that preserves each model’s discriminative geometry, a KL distributional regularizer informed by the observed per-actor class distribution, and test-time distributional optimization that refines predictions via gradient descent. Additionally, using the pre-trained features, we investigate how different fusion strategies, such as attention mechanisms, gated methods, and sparse mixture-of-experts architectures, impact classification performance. We then create an ensemble system by combining the top-performing models using fold-corrected weighted voting for emotion presence detection and confidencegated aggregation for salience estimation; this fused ensemble serves as an additional baseline. On the test set of the BlEmoRe challenge, CANDOR achieves first place with an average score of 0.457 (ACCpresence = 0.641, ACCsalience = 0.272), improving over the fusion ensemble by 67% which achieves a test score of 0.274.