Skeleton-Based Action Recognition With Multi-Stream Adaptive Graph Convolutional Networks | |
Shi, Lei; Zhang, Yifan; Cheng, Jian; Lu, Hanqing | |
刊名 | IEEE TRANSACTIONS ON IMAGE PROCESSING |
2020-12-01 | |
期号 | 29页码:9532-9545 |
关键词 | Skeleton-based action recognition, graph convolutional network, adaptive graph, multi-stream network. |
英文摘要 | Graph convolutional networks (GCNs), which generalize CNNs to more generic non-Euclidean structures, have achieved remarkable performance for skeleton-based action recognition. However, there still exist several issues in the previous GCN-based models. First, the topology of the graph is set heuristically and fixed over all the model layers and input data. This may not be suitable for the hierarchy of the GCN model and the diversity of the data in action recognition tasks. Second, the second-order information of the skeleton data, i.e., the length and orientation of the bones, is rarely investigated, which is naturally more informative and discriminative for the human action recognition. In this work, we propose a novel multi-stream attention-enhanced adaptive graph convolutional neural network (MS-AAGCN) for skeleton-based action recognition. The graph topology in our model can be either uniformly or individually learned based on the input data in an end-to-end manner. This data-driven approach increases the flexibility of the model for graph construction and brings more generality to adapt to various data samples. Besides, the proposed adaptive graph convolutional layer is further enhanced by a spatial-temporal channel attention module, which helps the model pay more attention to important joints, frames and features. Moreover, the information of both the joints and bones, together with their motion information, are simultaneously modeled in a multi-stream framework, which shows notable improvement for the recognition accuracy. Extensive experiments on the two large-scale datasets, NTU-RGBD and Kinetics-Skeleton, demonstrate that the performance of our model exceeds the state-of-the-art with a significant margin. |
WOS记录号 | WOS:000583596000001 |
内容类型 | 期刊论文 |
源URL | [http://ir.ia.ac.cn/handle/173211/41469] |
专题 | 自动化研究所_模式识别国家重点实验室_图像与视频分析团队 |
通讯作者 | Zhang, Yifan |
推荐引用方式 GB/T 7714 | Shi, Lei,Zhang, Yifan,Cheng, Jian,et al. Skeleton-Based Action Recognition With Multi-Stream Adaptive Graph Convolutional Networks[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2020(29):9532-9545. |
APA | Shi, Lei,Zhang, Yifan,Cheng, Jian,&Lu, Hanqing.(2020).Skeleton-Based Action Recognition With Multi-Stream Adaptive Graph Convolutional Networks.IEEE TRANSACTIONS ON IMAGE PROCESSING(29),9532-9545. |
MLA | Shi, Lei,et al."Skeleton-Based Action Recognition With Multi-Stream Adaptive Graph Convolutional Networks".IEEE TRANSACTIONS ON IMAGE PROCESSING .29(2020):9532-9545. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论