Multi-Correlation Siamese Transformer Network With Dense Connection for 3D Single Object Tracking

doi:10.1109/LRA.2023.3325715

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 视频内容安全团队

	Multi-Correlation Siamese Transformer Network With Dense Connection for 3D Single Object Tracking
	Feng, Shihao 4; Liang, Pengpeng 4; Gao, Jin 2,3; Cheng, Erkang 1
刊名	IEEE ROBOTICS AND AUTOMATION LETTERS
	2023-12-01
卷号	8 期号:12 页码:8066-8073
关键词	3D object tracking Point cloud Transformer
ISSN号	2377-3766
DOI	10.1109/LRA.2023.3325715
通讯作者	Liang, Pengpeng(liangpcs@gmail.com)
英文摘要	Point cloud-based 3D object tracking is an important task in autonomous driving. Though great advances regarding Siamese-based 3D tracking have been made recently, it remains challenging to learn the correlation between the template and search branches effectively with the sparse LIDAR point cloud data. Instead of performing correlation of the two branches at just one point in the network, in this letter, we present a multi-correlation Siamese Transformer network that has multiple stages and carries out feature correlation at the end of each stage based on sparse pillars. More specifically, in each stage, self-attention is first applied to each branch separately to capture the non-local context information. Then, cross-attention is used to inject the template information into the search area. This strategy allows the feature learning of the search area to be aware of the template while keeping the individual characteristics of the template intact. To enable the network to easily preserve the information learned at different stages and ease the optimization, for the search area, we densely connect the initial input sparse pillars and the output of each stage to all subsequent stages and the target localization network, which converts pillars to bird's eye view (BEV) feature maps and predicts the state of the target with a small densely connected convolution network. Deep supervision is added to each stage to further boost the performance as well. The proposed algorithm is evaluated on the popular KITTI, nuScenes, and Waymo datasets, and the experimental results show that our method achieves promising performance compared with the state-of-the-art. Ablation study that shows the effectiveness of each component is provided as well.
资助项目	Natural Science Foundation of China[61806181] ; Natural Science Foundation of China[U22B2056] ; Natural Science Foundation of China[61972394] ; Natural Science Foundation of China[62102417] ; Beijing Natural Science Foundation[L223003] ; Beijing Natural Science Foundation[JQ22014] ; Youth Innovation Promotion Association, CAS
WOS研究方向	Robotics
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:001093405900002
资助机构	Natural Science Foundation of China ; Beijing Natural Science Foundation ; Youth Innovation Promotion Association, CAS
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/54409]
专题	自动化研究所_模式识别国家重点实验室_视频内容安全团队
通讯作者	Liang, Pengpeng
作者单位	1.Nullmax Inc, Shanghai 201210, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Automat, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China 4.Zhengzhou Univ, Sch Comp & Artificial Intelligence, Zhengzhou 450001, Peoples R China
推荐引用方式 GB/T 7714	Feng, Shihao,Liang, Pengpeng,Gao, Jin,et al. Multi-Correlation Siamese Transformer Network With Dense Connection for 3D Single Object Tracking[J]. IEEE ROBOTICS AND AUTOMATION LETTERS,2023,8(12):8066-8073.
APA	Feng, Shihao,Liang, Pengpeng,Gao, Jin,&Cheng, Erkang.(2023).Multi-Correlation Siamese Transformer Network With Dense Connection for 3D Single Object Tracking.IEEE ROBOTICS AND AUTOMATION LETTERS,8(12),8066-8073.
MLA	Feng, Shihao,et al."Multi-Correlation Siamese Transformer Network With Dense Connection for 3D Single Object Tracking".IEEE ROBOTICS AND AUTOMATION LETTERS 8.12(2023):8066-8073.