Indexed by:
Abstract:
To address the issues of low spatial resolution and susceptibility to noise in traditional single-modality brain-computer interface (BCI) technologies based on electroencephalography (EEG), an increasing number of studies have focused on BCI research that combines EEG signals with functional near-infrared spectroscopy (fNIRS) signals. However, integrating these two heterogeneous signals poses challenges. This paper proposes an innovative end-to-end signal fusion method based on deep learning and evidence theory for motor imagery (MI) classification. The spatiotemporal feature information of EEG signals is extracted using dual-scale temporal convolution and depth wise separable convolution, with a hybrid attention module introduced to enhance the network’s ability to perceive important features. For fNIRS signals, spatial convolution across all channels explores activation differences between different brain regions, while parallel temporal convolution and gated recurrent unit (GRU) capture richer temporal feature information. During the decision fusion stage, the decision outputs obtained from decoding each signal are first utilized to estimate uncertainty using Dirichlet distribution parameter estimation. Subsequently, Dempster-Shafer theory (DST) is employed for dual-layer reasoning, effectively merging evidence from the two basic belief assignment (BBA) methods and different modalities to obtain the decoding results. The proposed model is evaluated on the publicly available TU-Berlin-A dataset, achieving an average accuracy of 83.26%, which represents a 3.78 percentage points improvement compared to the state-of-the-art research. This provides new ideas and approaches for fusion studies based on EEG and fNIRS signals. © 2025 Chinese Institute of Electronics. All rights reserved.
Keyword:
Reprint 's Address:
Email:
Version:
Source :
Acta Electronica Sinica
ISSN: 0372-2112
Year: 2025
Issue: 3
Volume: 53
Page: 941-950
Cited Count:
SCOPUS Cited Count:
ESI Highly Cited Papers on the List: 0 Unfold All
WanFang Cited Count:
Chinese Cited Count:
30 Days PV: 3
Affiliated Colleges: