• Complex
  • Title
  • Keyword
  • Abstract
  • Scholars
  • Journal
  • ISSN
  • Conference
成果搜索

author:

Wang, Zihan (Wang, Zihan.) [1] | Xie, Junwei (Xie, Junwei.) [2] | Yao, Zhiping (Yao, Zhiping.) [3] | Kuang, Xu (Kuang, Xu.) [4] | Gao, Qinquan (Gao, Qinquan.) [5] (Scholars:高钦泉) | Tong, Tong (Tong, Tong.) [6] (Scholars:童同)

Indexed by:

CPCI-S EI

Abstract:

The aim of Knowledge Distillation (KD) is to train lightweight student models through extra supervision from large teacher models. Most previous KD methods transfer feature information from teacher models to student models via connections of feature maps at the same layers. This paper proposes a novel multi-level knowledge distillation method, referred to as Normalized Feature Fusion Knowledge distillation(NFFKD). The proposed model learns different levels of knowledge to improve the network performance. We proposed to use the hierarchical mixed loss(HML) module to minimize the gap between the intermediate feature layers of the teacher and the student, and the teacher-student gap is reduced by normalizing the logits. Experimental results have demonstrated that the proposed NFFKD shows superiority over several state-of-the-art KD methods on public datasets under different settings.

Keyword:

convolutional neural network deep learning knowledge distillation knowledge transfer

Community:

  • [ 1 ] [Wang, Zihan]Fuzhou Univ, Coll Phys & Informat Engn, Fuzhou, Peoples R China
  • [ 2 ] [Yao, Zhiping]Fuzhou Univ, Coll Phys & Informat Engn, Fuzhou, Peoples R China
  • [ 3 ] [Kuang, Xu]Fuzhou Univ, Coll Phys & Informat Engn, Fuzhou, Peoples R China
  • [ 4 ] [Wang, Zihan]Fuzhou Univ, Fujian Key Lab Med Instrumentat & Pharmaceut Tech, Fuzhou, Peoples R China
  • [ 5 ] [Yao, Zhiping]Fuzhou Univ, Fujian Key Lab Med Instrumentat & Pharmaceut Tech, Fuzhou, Peoples R China
  • [ 6 ] [Kuang, Xu]Fuzhou Univ, Fujian Key Lab Med Instrumentat & Pharmaceut Tech, Fuzhou, Peoples R China
  • [ 7 ] [Gao, Qinquan]Fuzhou Univ, Fujian Key Lab Med Instrumentat & Pharmaceut Tech, Fuzhou, Peoples R China
  • [ 8 ] [Tong, Tong]Fuzhou Univ, Fujian Key Lab Med Instrumentat & Pharmaceut Tech, Fuzhou, Peoples R China
  • [ 9 ] [Xie, Junwei]Imperial Vis Technol, Fuzhou, Peoples R China
  • [ 10 ] [Gao, Qinquan]Imperial Vis Technol, Fuzhou, Peoples R China
  • [ 11 ] [Tong, Tong]Imperial Vis Technol, Fuzhou, Peoples R China

Reprint 's Address:

Show more details

Related Keywords:

Source :

2022 IEEE THE 5TH INTERNATIONAL CONFERENCE ON BIG DATA AND ARTIFICIAL INTELLIGENCE (BDAI 2022)

Year: 2022

Page: 111-116

Cited Count:

WoS CC Cited Count: 0

SCOPUS Cited Count:

ESI Highly Cited Papers on the List: 0 Unfold All

WanFang Cited Count:

Chinese Cited Count:

30 Days PV: 1

Online/Total:219/10043124
Address:FZU Library(No.2 Xuyuan Road, Fuzhou, Fujian, PRC Post Code:350116) Contact Us:0591-22865326
Copyright:FZU Library Technical Support:Beijing Aegean Software Co., Ltd. 闽ICP备05005463号-1