• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
成果搜索

author:

Zhang, Zhengfeng (Zhang, Zhengfeng.) [1] | Lai, Peichao (Lai, Peichao.) [2] | Wang, Ruiqing (Wang, Ruiqing.) [3] | Ye, Feiyang (Ye, Feiyang.) [4] | Wang, Yilei (Wang, Yilei.) [5] (Scholars:王一蕾)

Indexed by:

CPCI-S EI Scopus

Abstract:

Unsupervised sentence embedding methods based on contrastive learning have gained attention for effectively representing sentences in natural language processing. Retrieving additional samples via a nearest-neighbor approach can enhance the model's ability to learn relevant semantics and distinguish sentences. However, previous related research mainly focused on retrieving neighboring samples within a single batch range or global range, which makes the model possibly unable to capture effective semantic information or incurs excessive time cost. Furthermore, previous methods use retrieved neighbor samples as hard negatives. We argue that nearest neighbor samples contain relevant semantic information, and treating them as hard negatives risks losing valuable semantic knowledge. In this work, we introduce Neighbor Contrastive learning for unsupervised Sentence Embeddings(NCSE), which combines contrastive learning with the nearest-neighbor approach. Specifically, we create a candidate set to store sentence embeddings across multiple batches. Retrieving the candidate set can ensure sufficient samples, making it easier for the model to learn relevant semantics. Using retrieved nearest neighbor samples as positives and applying the self-attention mechanism to aggregate the sample and its neighbors encourages the model to learn relevant semantics from multiple neighbors. Experiments on the semantic text similarity task demonstrate our method's effectiveness in sentence embedding learning.

Keyword:

contrastive learning sentence embedding unsupervised learning

Community:

  • [ 1 ] [Zhang, Zhengfeng]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China
  • [ 2 ] [Lai, Peichao]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China
  • [ 3 ] [Wang, Ruiqing]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China
  • [ 4 ] [Ye, Feiyang]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China
  • [ 5 ] [Wang, Yilei]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China

Reprint 's Address:

  • 王一蕾

    [Wang, Yilei]Fuzhou Univ, Coll Comp & Data Sci, Fuzhou, Peoples R China

Show more details

Related Keywords:

Related Article:

Source :

2024 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, IJCNN 2024

ISSN: 2161-4393

Year: 2024

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 1

Online/Total:343/10370176
Address:FZU Library(No.2 Xuyuan Road, Fuzhou, Fujian, PRC Post Code:350116) Contact Us:0591-22865326
Copyright:FZU Library Technical Support:Beijing Aegean Software Co., Ltd. 闽ICP备05005463号-1