Target-Embedding Autoencoder With Knowledge Distillation for Multi-Label Classification

doi:10.1109/TETCI.2024.3372693

CORC > 自动化研究所 > 中国科学院自动化研究所 > 数字内容技术与服务研究中心 > 听觉模型与认知计算

	Target-Embedding Autoencoder With Knowledge Distillation for Multi-Label Classification
	Ma, Ying 1,2; Zou, Xiaoyan 3; Pan, Qizheng 3; Yan, Ming 4; Li, Guoqi 5
刊名	IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE
	2024-03-21
页码	12
关键词	Multi-label classification knowledge distillation autoencoder label embedding
ISSN号	2471-285X
DOI	10.1109/TETCI.2024.3372693
通讯作者	Zou, Xiaoyan(2122031196@s.xmut.edu.cn) ; Pan, Qizheng(panqizheng@s.xmut.edu.cn)
英文摘要	In the task of multi-label classification, it is a key challenge to determine the correlation between labels. One solution to this is the Target Embedding Autoencoder (TEA), but most TEA-based frameworks have numerous parameters, large models, and high complexity, which makes it difficult to deal with the problem of large-scale learning. To address this issue, we provide a Target Embedding Autoencoder framework based on Knowledge Distillation (KD-TEA) that compresses a Teacher model with large parameters into a small Student model through knowledge distillation. Specifically, KD-TEA transfers the dark knowledge learned from the Teacher model to the Student model. The dark knowledge can provide effective regularization to alleviate the over-fitting problem in the training process, thereby enhancing the generalization ability of the Student model, and better completing the multi-label task. In order to make the Student model learn the knowledge of the Teacher model directly, we improve the distillation loss: KD-TEA uses MSE loss instead of KL divergence loss to improve the performance of the model in multi-label tasks. Experiments on multiple datasets show that our KD-TEA framework is superior to the most advanced multi-label classification methods in both performance and efficiency.
资助项目	Fujian Provincial Science Foundation for Outstanding Youth
WOS关键词	TEXT
WOS研究方向	Computer Science
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:001189443800001
资助机构	Fujian Provincial Science Foundation for Outstanding Youth
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/58011]
专题	数字内容技术与服务研究中心_听觉模型与认知计算
通讯作者	Zou, Xiaoyan; Pan, Qizheng
作者单位	1.Harbin Inst Technol, Fac Comp, Harbin 150001, Peoples R China 2.Xiamen Univ Technol, Fac Comp & Informat Engn, Xiamen 361024, Peoples R China 3.Xiamen Univ Technol, Xiamen 361021, Peoples R China 4.Singapore Sci & Technol Bur, Inst High Performance Comp, Singapore, Singapore 5.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Ma, Ying,Zou, Xiaoyan,Pan, Qizheng,et al. Target-Embedding Autoencoder With Knowledge Distillation for Multi-Label Classification[J]. IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE,2024:12.
APA	Ma, Ying,Zou, Xiaoyan,Pan, Qizheng,Yan, Ming,&Li, Guoqi.(2024).Target-Embedding Autoencoder With Knowledge Distillation for Multi-Label Classification.IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE,12.
MLA	Ma, Ying,et al."Target-Embedding Autoencoder With Knowledge Distillation for Multi-Label Classification".IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE (2024):12.