Encoder-Decoder Full Residual Deep Networks for Robust Regression and Spatiotemporal Estimation

doi:10.1109/TNNLS.2020.3017200

	Encoder-Decoder Full Residual Deep Networks for Robust Regression and Spatiotemporal Estimation
	Li, Lianfa 1; Fang, Ying 1; Wu, Jun 2; Wang, Jinfeng 1; Ge, Yong 1
刊名	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
	2021-09-01
卷号	32 期号:9 页码:4217-4230
关键词	Bias deep learning encoder-decoder full residual deep network non-linear regression prediction of satellite aerosol optical depth (AOD) and PM2.5 spatiotemporal modeling
ISSN号	2162-237X
DOI	10.1109/TNNLS.2020.3017200
通讯作者	Li, Lianfa(lspatial@gmail.com)
英文摘要	Although increasing hidden layers can improve the ability of a neural network in modeling complex nonlinear relationships, deep layers may result in degradation of accuracy due to the problem of vanishing gradient. Accuracy degradation limits the applications of deep neural networks to predict continuous variables with a small sample size and/or weak or little invariance to translations. Inspired by residual convolutional neural network in computer vision, we developed an encoder-decoder full residual deep network to robustly regress and predict complex spatiotemporal variables. We embedded full shortcuts from each encoding layer to its corresponding decoding layer in a systematic encoder-decoder architecture for efficient residual mapping and error signal propagation. We demonstrated, theoretically and experimentally, that the proposed network structure with full residual connections can successfully boost the backpropagation of signals and improve learning outcomes. This novel method has been extensively evaluated and compared with four commonly used methods (i.e., plain neural network, cascaded residual autoencoder, generalized additive model, and XGBoost) across different testing cases for continuous variable predictions. For model evaluation, we focused on spatiotemporal imputation of satellite aerosol optical depth with massive nonrandomness missingness and spatiotemporal estimation of atmospheric fine particulate matter <= 2.5 mu m (PM2.5). Compared with the other approaches, our method achieved the state-of-the-art accuracy, had less bias in predicting extreme values, and generated more realistic spatial surfaces. This encoder-decoder full residual deep network can be an efficient and powerful tool in a variety of applications that involve complex nonlinear relationships of continuous variables, varying sample sizes, and spatiotemporal data with weak or little invariance to translation.
资助项目	Strategic Priority Research Program of the Chinese Academy of Sciences[XDA19040501] ; National Natural Science Foundation of China[41471376] ; National Institute of Environmental Health Sciences[ES030353]
WOS关键词	LEVEL PM2.5 CONCENTRATIONS ; AEROSOL OPTICAL DEPTH ; MODEL
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000692208800038
资助机构	Strategic Priority Research Program of the Chinese Academy of Sciences ; National Natural Science Foundation of China ; National Institute of Environmental Health Sciences
内容类型	期刊论文
源URL	[http://ir.igsnrr.ac.cn/handle/311030/165271]
专题	中国科学院地理科学与资源研究所
通讯作者	Li, Lianfa
作者单位	1.Chinese Acad Sci, Inst Geog Sci & Nat Resources Res, State Key Lab Resources & Environm Informat Syst, Beijing, Peoples R China 2.Univ Calif Irvine, Dept Environm & Occupat Hlth, Irvine, CA 92697 USA
推荐引用方式 GB/T 7714	Li, Lianfa,Fang, Ying,Wu, Jun,et al. Encoder-Decoder Full Residual Deep Networks for Robust Regression and Spatiotemporal Estimation[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2021,32(9):4217-4230.
APA	Li, Lianfa,Fang, Ying,Wu, Jun,Wang, Jinfeng,&Ge, Yong.(2021).Encoder-Decoder Full Residual Deep Networks for Robust Regression and Spatiotemporal Estimation.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,32(9),4217-4230.
MLA	Li, Lianfa,et al."Encoder-Decoder Full Residual Deep Networks for Robust Regression and Spatiotemporal Estimation".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 32.9(2021):4217-4230.