Efficient convolutional networks learning through irregular convolutional kernels

doi:10.1016/j.neucom.2022.02.065

CORC > 自动化研究所 > 中国科学院自动化研究所 > 智能感知与计算研究中心

	Efficient convolutional networks learning through irregular convolutional kernels
	Guo, Weiyu 2; Ma, Jiabin 1; Ouyang, Yidong 2; Wang, Liang 1; Huang, Yongzhen 3,4
刊名	NEUROCOMPUTING
	2022-06-07
卷号	489 页码:167-178
关键词	Model compression Interpolation Irregular convolutional kernels
ISSN号	0925-2312
DOI	10.1016/j.neucom.2022.02.065
通讯作者	Huang, Yongzhen(hyz@watrix.ai)
英文摘要	As deep neural networks are increasingly used in applications suited for low-power devices, a fundamen-tal dilemma emerges: the trend is to develop models to use the increasing amount of data, resulting in memory-intensive models; however, low-power devices have very limited memory and cannot store large models. Parameters pruning is critical for deep model deployment on low-power devices. Existing efforts mainly focus on designing highly efficient structures or pruning redundant connections in networks. They are typically sensitive to the tasks or rely on dedicated and expensive hashing storage strategies. In this work, we introduce a novel approach to achieve a lightweight model from the perspec-tive of reconstructing the structure of convolution kernels for efficient storage. Our approach transforms a traditional square convolution kernel into line segments, and automatically learns a proper strategy for equipping these line segments to model diverse features. Experimental results show that our approach can significantly reduce the number of parameters (pruned 69% on DenseNet-40) and calculation costs (pruned 59% on DenseNet-40) while maintaining acceptable performance (only lose less than 2% accuracy). (c) 2022 Elsevier B.V. All rights reserved.
资助项目	National Natural Science Foundation of China[62106290] ; Program for Innovation Research in Central University of Finance and Economics
WOS研究方向	Computer Science
语种	英语
出版者	ELSEVIER
WOS记录号	WOS:000782510100014
资助机构	National Natural Science Foundation of China ; Program for Innovation Research in Central University of Finance and Economics
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/48332]
专题	自动化研究所_智能感知与计算研究中心
通讯作者	Huang, Yongzhen
作者单位	1.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing, Peoples R China 2.Cent Univ Finance & Econ, Informat Sch, Beijing, Peoples R China 3.Watrix Technol Co Ltd, Beijing, Peoples R China 4.Beijing Normal Univ, Sch Artificial Intelligence, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Guo, Weiyu,Ma, Jiabin,Ouyang, Yidong,et al. Efficient convolutional networks learning through irregular convolutional kernels[J]. NEUROCOMPUTING,2022,489:167-178.
APA	Guo, Weiyu,Ma, Jiabin,Ouyang, Yidong,Wang, Liang,&Huang, Yongzhen.(2022).Efficient convolutional networks learning through irregular convolutional kernels.NEUROCOMPUTING,489,167-178.
MLA	Guo, Weiyu,et al."Efficient convolutional networks learning through irregular convolutional kernels".NEUROCOMPUTING 489(2022):167-178.