• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
成果搜索

author:

Zhang, Zhengfeng (Zhang, Zhengfeng.) [1] | Lai, Peichao (Lai, Peichao.) [2] | Wang, Ruiqing (Wang, Ruiqing.) [3] | Ye, Feiyang (Ye, Feiyang.) [4] | Wang, Yilei (Wang, Yilei.) [5]

Indexed by:

EI

Abstract:

Unsupervised sentence embedding methods based on contrastive learning have gained attention for effectively representing sentences in natural language processing. Retrieving additional samples via a nearest-neighbor approach can enhance the model's ability to learn relevant semantics and distinguish sentences. However, previous related research mainly focused on retrieving neighboring samples within a single batch range or global range, which makes the model possibly unable to capture effective semantic information or incurs excessive time cost. Furthermore, previous methods use retrieved neighbor samples as hard negatives. We argue that nearest neighbor samples contain relevant semantic information, and treating them as hard negatives risks losing valuable semantic knowledge. In this work, we introduce Neighbor Contrastive learning for unsupervised Sentence Embeddings(NCSE), which combines contrastive learning with the nearest-neighbor approach. Specifically, we create a candidate set to store sentence embeddings across multiple batches. Retrieving the candidate set can ensure sufficient samples, making it easier for the model to learn relevant semantics. Using retrieved nearest neighbor samples as positives and applying the self-attention mechanism to aggregate the sample and its neighbors encourages the model to learn relevant semantics from multiple neighbors. Experiments on the semantic text similarity task demonstrate our method's effectiveness in sentence embedding learning. © 2024 IEEE.

Keyword:

Adversarial machine learning Contrastive Learning Embeddings Federated learning Semantics Unsupervised learning

Community:

  • [ 1 ] [Zhang, Zhengfeng]Fuzhou University, College of Computer and Data Science, Fuzhou, China
  • [ 2 ] [Lai, Peichao]Fuzhou University, College of Computer and Data Science, Fuzhou, China
  • [ 3 ] [Wang, Ruiqing]Fuzhou University, College of Computer and Data Science, Fuzhou, China
  • [ 4 ] [Ye, Feiyang]Fuzhou University, College of Computer and Data Science, Fuzhou, China
  • [ 5 ] [Wang, Yilei]Fuzhou University, College of Computer and Data Science, Fuzhou, China

Reprint 's Address:

Email:

Show more details

Related Keywords:

Related Article:

Source :

Year: 2024

Language: English

Cited Count:

WoS CC Cited Count:

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 4

Affiliated Colleges:

Online/Total:168/10372261
Address:FZU Library(No.2 Xuyuan Road, Fuzhou, Fujian, PRC Post Code:350116) Contact Us:0591-22865326
Copyright:FZU Library Technical Support:Beijing Aegean Software Co., Ltd. 闽ICP备05005463号-1