Data Management Challenges and Event Index Technologies in High Energy Physics

doi:10.7544/issn1000-1239.2017.20160939

	Data Management Challenges and Event Index Technologies in High Energy Physics
	Cheng YD(程耀东); Cheng, Yaodong; Zhang, Xiao; Wang, Peijian ; Zha, Li ; Hou, Di ; Qi, Yong ; Ma, Can
刊名	Computer Research and Development计算机研究与发展
	2017
卷号	54 期号:2 页码:258-266
ISSN号	1000-1239
DOI	10.7544/issn1000-1239.2017.20160939
其他题名	高能物理大数据挑战与海量事例特征索引技术研究
文献子类	Article
英文摘要	Nowadays, more and more scientific data has been produced by new generation high energy physics facilities. The scale of the data can be achieved to PB or EB level even by one experiment, which brings big challenges to data management technologies such as data acquisition, storage, transmission, sharing, analyzing and processing. Event is the basic data unit of high energy physics, and one large high energy physics experiment can produce trillions of events. The traditional high energy physical data processing technology adopts file as a basic data management unit, and each file contains thousands of events. The benefit of file-based method is to simplify the complexity of data management system. However, one physical analysis task is only interested in very few events, which leads to some problems including transferring too much redundant data, I/O bottleneck and low efficiency of data processing. To solve these problems, this paper proposes an event-oriented high energy physical data management method, which focuses on high efficiency indexing technology of massive events. In this method, event data is still stored in ROOT file while a large amount of events are indexed by some specified properties and stored in NoSQL database. Finally, experimental test results show the feasibility of the method, and optimized HBase system can meet the requirements of event index. © 2017, Science Press. All right reserved.
语种	中文
CSCD记录号	CSCD:5928217
内容类型	期刊论文
源URL	[http://ir.ihep.ac.cn/handle/311005/285450]
专题	中国科学院高能物理研究所
作者单位	1.Institute of Information Engineering, Chinese Academy of Sciences, Beijing; 100193, China 2.Institute of Computing Technology, Chinese Academy of Sciences, Beijing; 100190, China; 3.Department of Computer Science and Technology, Xi'an Jiaotong University, Xi'an; 710049, China; 4.Institute of High Energy Physics, Chinese Academy of Sciences, Beijing; 100049, China;
推荐引用方式 GB/T 7714	Cheng YD,Cheng, Yaodong,Zhang, Xiao,et al. Data Management Challenges and Event Index Technologies in High Energy Physics[J]. Computer Research and Development计算机研究与发展,2017,54(2):258-266.
APA	程耀东.,Cheng, Yaodong.,Zhang, Xiao.,Wang, Peijian.,Zha, Li.,...&Ma, Can.(2017).Data Management Challenges and Event Index Technologies in High Energy Physics.Computer Research and Development计算机研究与发展,54(2),258-266.
MLA	程耀东,et al."Data Management Challenges and Event Index Technologies in High Energy Physics".Computer Research and Development计算机研究与发展 54.2(2017):258-266.