Fast Data-Obtaining Algorithm for Data Assimilation with Large Data Set

doi:10.1007/s10766-019-00653-y

CORC > 计算技术研究所 > 中国科学院计算技术研究所 > 中国科学院计算技术研究所期刊论文 > 英文

	Fast Data-Obtaining Algorithm for Data Assimilation with Large Data Set
	Xiao, Junmin 1,2; Zhang, Guizhao 3; Gao, Yanan 4; Ho, Xuehai 3; Tan, Guangming 1,2
刊名	INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING
	2019-12-06
页码	21
关键词	Data assimilation I O optimization Communication optimization Parallel implementation Domain localization
ISSN号	0885-7458
DOI	10.1007/s10766-019-00653-y
英文摘要	Data assimilation is an analysis technique which combines observations and the numerical results from theoretical models to deduce more realistic and accurate data. It is widely used in investigations of the atmosphere, ocean and land surface. Due to the complicated data structure of the inputs from dynamical models and the increase of the amount of model data, the parallelization of data assimilation suffers from high overhead on file reading and data communication. In this paper, we propose a flexible parallel data access approach for reading a large number of data from disks firstly. Using this approach, the data access conflict is avoided successfully, and the frequency of disk addressing operations is also decreased significantly. Next, we design a communication-avoiding strategy to reduce the communication volume at the cost of some additional computations. Furthermore, we present a "pipe-flow" scheme for data exchange to conduct conflict-free message passing. Consequently, a fast data-obtaining algorithm is developed for the data assimilation. Our experiments show that the fast data-obtaining algorithm gains a performance of 5x speedup compared with the baseline, which is excellent at data-obtaining for the parallel data assimilation. Due to the reduction of disk addressing operations, the new approach achieves 6x speedup on average for the file reading process. Since a large amount of data movement can be avoided, the new approach achieves 2.7x speedup on average for the communication between processors.
资助项目	National Key Research and Development Program of China[2016YFC1401706] ; National Natural Science Foundation of China[61802369]
WOS研究方向	Computer Science
语种	英语
出版者	SPRINGER/PLENUM PUBLISHERS
WOS记录号	WOS:000529493100001
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/15465]
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Xiao, Junmin
作者单位	1.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 4.Beijing Inst Control Engn, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Xiao, Junmin,Zhang, Guizhao,Gao, Yanan,et al. Fast Data-Obtaining Algorithm for Data Assimilation with Large Data Set[J]. INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING,2019:21.
APA	Xiao, Junmin,Zhang, Guizhao,Gao, Yanan,Ho, Xuehai,&Tan, Guangming.(2019).Fast Data-Obtaining Algorithm for Data Assimilation with Large Data Set.INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING,21.
MLA	Xiao, Junmin,et al."Fast Data-Obtaining Algorithm for Data Assimilation with Large Data Set".INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING (2019):21.