• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
成果搜索

author:

Cheng, Jiawei (Cheng, Jiawei.) [1] | Zhu, Xiaofei (Zhu, Xiaofei.) [2] | Yang, Zhou (Yang, Zhou.) [3]

Indexed by:

EI Scopus

Abstract:

Multimodal emotion recognition in conversations aims to accurately detect emotions by integrating audio, text, and video modalities, playing an important role in various systems. Existing approaches utilize convolutional and recurrent networks to learn short-term emotional information from individual modalities, or employ graph and attention mechanisms to integrate long-term emotional information from multiple modalities. These methods effectively combine emotional information within the conversational content in the time domain.However, psychological research shows that emotional information are not only conveyed in the time domain but also in the frequency domain (e.g., pitch and speech rate). To capture emotions from a more comprehensive perspective, we propose TF-MERC, a framework that integrates both time and frequency domains.TF-MERC uses a multi-domain alignment module to learn modality information within the time or frequency domains. It then employs FATransformer to deeply integrate the multimodal associations between the time and frequency domains, providing a more comprehensive approach for emotion prediction.Experimental results show that TF-MERC outperforms state-of-the-art methods, achieving superior performance across multiple datasets. © 2025 ACM.

Keyword:

Behavioral research Emotion Recognition Frequency domain analysis Interactive computer graphics Interactive computer systems Psychology computing Speech recognition Time domain analysis

Community:

  • [ 1 ] [Cheng, Jiawei]Chongqing University of Technology, Chongqing, China
  • [ 2 ] [Zhu, Xiaofei]Chongqing University of Technology, Chongqing, China
  • [ 3 ] [Yang, Zhou]Fuzhou University, Fuzhou, China

Reprint 's Address:

Email:

Show more details

Version:

Related Keywords:

Related Article:

Source :

Year: 2025

Page: 126-134

Language: English

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 0

Affiliated Colleges:

Online/Total:1230/13834293
Address:FZU Library(No.2 Xuyuan Road, Fuzhou, Fujian, PRC Post Code:350116) Contact Us:0591-22865326
Copyright:FZU Library Technical Support:Beijing Aegean Software Co., Ltd. 闽ICP备05005463号-1