Effective Model Compression via Stage-wise Pruning

doi:10.1007/s11633-022-1357-9

CORC > 自动化研究所 > 中国科学院自动化研究所 > 学术期刊 > International Journal of Automation and Computing

	Effective Model Compression via Stage-wise Pruning
	Ming-Yang Zhang, Xin-Yi Yu, Lin-Lin Ou
刊名	Machine Intelligence Research
	2023
卷号	20 期号:6 页码:937-951
关键词	Automated machine learning (AutoML), channel pruning, model compression, distillation, convolutional neural networks (CNN)
ISSN号	2731-538X
DOI	10.1007/s11633-022-1357-9
英文摘要	Automated machine learning (AutoML) pruning methods aim at searching for a pruning strategy automatically to reduce the computational complexity of deep convolutional neural networks (deep CNNs). However, some previous work found that the results of many Auto-ML pruning methods cannot even surpass the results of the uniformly pruning method. In this paper, the ineffectiveness of Auto-ML pruning, which is caused by unfull and unfair training of the supernet, is shown. A deep supernet suffers from unfull training because it contains too many candidates. To overcome the unfull training, a stage-wise pruning (SWP) method is proposed, which splits a deep supernet into several stage-wise supernets to reduce the candidate number and utilize inplace distillation to supervise the stage training. Besides, a wide supernet is hit by unfair training since the sampling probability of each channel is unequal. Therefore, the full net and the tinynet are sampled in each training iteration to ensure that each channel can be overtrained. Remarkably, the proxy performance of the subnets trained with SWP is closer to the actual performance than that of most of the previous AutoML pruning work. Furthermore, experiments show that SWP achieves the state-of-the-art in both CIFAR-10 and ImageNet under the mobile setting.
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/54177]
专题	自动化研究所_学术期刊_International Journal of Automation and Computing
作者单位	School of Information Engineering, Zhejiang University of Technology, Hangzhou 310000, China
推荐引用方式 GB/T 7714	Ming-Yang Zhang, Xin-Yi Yu, Lin-Lin Ou. Effective Model Compression via Stage-wise Pruning[J]. Machine Intelligence Research,2023,20(6):937-951.
APA	Ming-Yang Zhang, Xin-Yi Yu, Lin-Lin Ou.(2023).Effective Model Compression via Stage-wise Pruning.Machine Intelligence Research,20(6),937-951.
MLA	Ming-Yang Zhang, Xin-Yi Yu, Lin-Lin Ou."Effective Model Compression via Stage-wise Pruning".Machine Intelligence Research 20.6(2023):937-951.