• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
成果搜索

author:

Wang, J. (Wang, J..) [1] | Wang, S. (Wang, S..) [2] | Lin, M. (Lin, M..) [3] | Xu, Z. (Xu, Z..) [4] | Guo, W. (Guo, W..) [5]

Indexed by:

Scopus

Abstract:

Multimodal sentiment analysis is an actively growing research area that utilizes language, acoustic and visual signals to predict sentiment inclination. Compared to language, acoustic and visual features carry a more evident personal style which may degrade the model generalization capability. The issue will be exacerbated in a speaker-independent setting, where the model will encounter samples from unseen speakers during the testing stage. To mitigate personal style's impact, we propose a framework named SIMR for learning speaker-independent multimodal representation. This framework separates the nonverbal inputs into style encoding and content representation with the aid of informative cross-modal correlations. Besides, in terms of integrating cross-modal complementary information, the classical transformer-based approaches are inherently inclined to discover compatible cross-modal interactions but ignore incompatible ones. In contrast, we suggest simultaneously locating both through an enhanced cross-modal transformer module. Experimental results show that the proposed model achieves state-of-the-art performance on several datasets. © 2023 Elsevier Inc.

Keyword:

Multimodal fusion Multimodal representation learning Multimodal sentiment analysis Multi-view learning

Community:

  • [ 1 ] [Wang, J.]College of Computer and Data Science, Fuzhou University, Fuzhou, 350116, China
  • [ 2 ] [Wang, J.]College of Computer and Cyber Security, Fujian Normal University, Fuzhou, 350117, China
  • [ 3 ] [Wang, S.]College of Computer and Data Science, Fuzhou University, Fuzhou, 350116, China
  • [ 4 ] [Wang, S.]Key Laboratory of Network Computing and Intelligent Information Processing, Fuzhou University, Fuzhou, 350116, China
  • [ 5 ] [Lin, M.]College of Computer and Cyber Security, Fujian Normal University, Fuzhou, 350117, China
  • [ 6 ] [Lin, M.]Digital Fujian Institute of Big Data Security Technology, Fujian Normal University, Fuzhou, 350117, China
  • [ 7 ] [Xu, Z.]Business School, Sichuan University, Sichuan, Chengdu, 610064, China
  • [ 8 ] [Guo, W.]College of Computer and Data Science, Fuzhou University, Fuzhou, 350116, China
  • [ 9 ] [Guo, W.]Key Laboratory of Network Computing and Intelligent Information Processing, Fuzhou University, Fuzhou, 350116, China

Reprint 's Address:

  • [Guo, W.]College of Computer and Data Science, China

Show more details

Related Keywords:

Source :

Information Sciences

ISSN: 0020-0255

Year: 2023

Volume: 628

Page: 208-225

0 . 0

JCR@2023

0 . 0 0 0

JCR@2023

ESI HC Threshold:32

CAS Journal Grade:1

Cited Count:

WoS CC Cited Count: 0

SCOPUS Cited Count: 5

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 0

Affiliated Colleges:

Online/Total:137/10046163
Address:FZU Library(No.2 Xuyuan Road, Fuzhou, Fujian, PRC Post Code:350116) Contact Us:0591-22865326
Copyright:FZU Library Technical Support:Beijing Aegean Software Co., Ltd. 闽ICP备05005463号-1