Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems

doi:10.1109/TPDS.2021.3067731

	Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems
	Tan, Guangming; Shui, Chaoyang; Wang, Yinshan; Yu, Xianzhi; Yan, Yujin
刊名	IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS
	2021-09-01
卷号	32 期号:9 页码:2367-2380
关键词	Pipeline processing Graphics processing units Computer architecture Supercomputers Clustering algorithms Programming Optimization LINPACK algorithm software pipeline performance model heterogeneous computing cluster
ISSN号	1045-9219
DOI	10.1109/TPDS.2021.3067731
英文摘要	There is a widening gap between GPU and other components (CPU, PCIe bus and communication network) in heterogeneous parallel system. The gap forces us to orchestrate cooperative execution among these components much more carefully than ever before. By taking the LINPACK benchmark as a case study, this article proposes a fine-grained pipelining algorithm on large-scale CPU-GPU heterogeneous cluster systems. First, we build an algorithmic model that reveals a new approach to GPU-centric and fine-grained pipelining algorithm design. Then, we present four model-driven pipelining algorithms that incrementally squeeze bubbles in the pipeline so that it is occupied by more useful floating-point calculations. The algorithms are implemented on both the AMD and NVIDIA GPU platforms. The finally optimized LINPACK program achieves 107 PFlops on 25, 600 GPUs (70 percent floating-point efficiency). Several insights have been drawn to suggest tradeoff of algorithm design, programming support, and architecture design.
资助项目	National Key Research and Development Program of China[2018YFB0204400] ; National Key Research and Development Program of China[XDC01030000] ; National Key Research and Development Program of China[XDC05010100] ; National Natural Science Foundation of China[62032023] ; National Natural Science Foundation of China[61972377] ; National Natural Science Foundation of China[61702483] ; CAS[QYZDJ-SSW-JSC035]
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE COMPUTER SOC
WOS记录号	WOS:000638398900004
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/16647]
专题	中国科学院计算技术研究所
通讯作者	Shui, Chaoyang
作者单位	Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100864, Peoples R China
推荐引用方式 GB/T 7714	Tan, Guangming,Shui, Chaoyang,Wang, Yinshan,et al. Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems[J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,2021,32(9):2367-2380.
APA	Tan, Guangming,Shui, Chaoyang,Wang, Yinshan,Yu, Xianzhi,&Yan, Yujin.(2021).Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems.IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,32(9),2367-2380.
MLA	Tan, Guangming,et al."Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems".IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 32.9(2021):2367-2380.