A Multi-Agent Reinforcement Learning Method With Route Recorders for Vehicle Routing in Supply Chain Management

doi:10.1109/TITS.2022.3150151

CORC > 自动化研究所 > 中国科学院自动化研究所 > 复杂系统管理与控制国家重点实验室 > 先进控制与自动化团队

	A Multi-Agent Reinforcement Learning Method With Route Recorders for Vehicle Routing in Supply Chain Management
	Ren, Lei 1,5; Fan, Xiaoyang 1,5; Cui, Jin 6,7; Shen, Zhen 4; Lv, Yisheng 4; Xiong, Gang 2,3
刊名	IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
	2022-02-15
页码	11
关键词	Reinforcement learning Costs Task analysis Transportation Optimization Computational modeling Vehicle routing Vehicle routing supply chain management multi-agent reinforcement learning (MARL) route recorder
ISSN号	1524-9050
DOI	10.1109/TITS.2022.3150151
通讯作者	Cui, Jin(jincui@buaa.edu.cn)
英文摘要	In the modern supply chain system, large-scale transportation tasks require the collaborative work of multiple vehicles to be completed on time. Over the past few decades, multi-vehicle route planning was mainly implemented by heuristic algorithms. However, these algorithms face the dilemma of long computation time. In recent years, some machine learning-based methods are also proposed for vehicle route planning, but the existing algorithms can hardly solve multi-vehicle time-sensitive problems. To overcome this problem, we propose a novel multi-agent reinforcement learning model, which optimizes the route length and the vehicle's arrival time simultaneously. The model is based on the encoder-decoder framework. The encoder mines the relationship between the customer nodes in the problem, and the decoder generates the route of each vehicle iteratively. Specially, we design multiple route recorders to extract the route history information of vehicles and realize the communication between them. In the inferring phase, the model could immediately generate routes for all vehicles in a new instance. To further improve the performance of the model, we devise a multi-sampling strategy and obtain the balance boundary between computation time and performance improvement. In addition, we propose a simulation-based vehicle configuration method to select the optimal number of vehicles in real applications. For validation, we conduct a series of experiments on problems with different customer amounts and various vehicle numbers. The results show that the proposed model outperforms other typical algorithms in both performance and calculation time.
资助项目	National Key Research and Development Program of China[2019YFB1705502]
WOS关键词	OPTIMIZATION ; ALGORITHM ; NUMBER
WOS研究方向	Engineering ; Transportation
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000758740900001
资助机构	National Key Research and Development Program of China
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/47906]
专题	自动化研究所_复杂系统管理与控制国家重点实验室_先进控制与自动化团队
通讯作者	Cui, Jin
作者单位	1.Beihang Univ, Beijing Adv Innovat Ctr Big Data Based Precis Med, Beijing 100191, Peoples R China 2.Chinese Acad Sci, Inst Automat, Beijing Engn Res Ctr Intelligent Syst & Technol, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Cloud Comp Ctr, Dongguan 523808, Peoples R China 4.Chinese Acad Sci, State Key Lab Management & Control Complex Syst, Inst Automat, Beijing 100190, Peoples R China 5.Beihang Univ, Sch Automat Sci & Elect Engn, Beijing 100191, Peoples R China 6.Beihang Univ, Res Inst Frontier Sci, Beijing 100191, Peoples R China 7.Beihang Univ, Ningbo Inst Technol, Ningbo 315800, Peoples R China
推荐引用方式 GB/T 7714	Ren, Lei,Fan, Xiaoyang,Cui, Jin,et al. A Multi-Agent Reinforcement Learning Method With Route Recorders for Vehicle Routing in Supply Chain Management[J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,2022:11.
APA	Ren, Lei,Fan, Xiaoyang,Cui, Jin,Shen, Zhen,Lv, Yisheng,&Xiong, Gang.(2022).A Multi-Agent Reinforcement Learning Method With Route Recorders for Vehicle Routing in Supply Chain Management.IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,11.
MLA	Ren, Lei,et al."A Multi-Agent Reinforcement Learning Method With Route Recorders for Vehicle Routing in Supply Chain Management".IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS (2022):11.