Indexed by:
Abstract:
Shared information refers to the common semantic information across multiple modalities, and complementary information refers to modality-specific information that complements other modalities. How to fully utilize this information is a key issue in the multimodal sentiment analysis. In this paper, we first propose a Slice Aggregation (SA) algorithm to address the issue of correlation over time. We use sliding windows to calculate the horizontal and vertical correlations, and then aggregate slices into a series of chunks, each represents a set of successive slices with consistent correlation. Second, we introduce a Dynamic Fusion (DF) strategy comprising two components: shared information fusion and complementary information fusion. The former utilizes a multilayer perceptron (MLP) to extract high-level shared representations, whereas the latter employs a cross-modal multi-head attention mechanism to fuse low-level complementary information. Finally, we propose an SA-DF framework where SA organizes raw slices into correlation-consistent chunks, and DF progressively fuses features across these chunks. The concatenated fused features are used for final sentiment prediction. The experiments on CMU-MOSI and CH-SIMS datasets show that the proposed SA-DF can achieve the best performance on sentiment analysis tasks when compared with the state-of-the-art baselines. © China Computer Federation (CCF) 2025.
Keyword:
Reprint 's Address:
Email:
Source :
CCF Transactions on Pervasive Computing and Interaction
ISSN: 2524-521X
Year: 2025
2 . 2 0 0
JCR@2023
Cited Count:
SCOPUS Cited Count:
ESI Highly Cited Papers on the List: 0 Unfold All
WanFang Cited Count:
Chinese Cited Count:
30 Days PV: 0
Affiliated Colleges: