Inverse Kinematics Embedded Network for Robust Patient Anatomy Avatar Reconstruction From Multimodal Data | |
Tongxi Zhou1,5; Mingcong Chen2,4; Guanglin Cao1,5; Jian, Hu1,2; Hongbin Liu1,2,3 | |
刊名 | IEEE Robotics and Automation Letters
![]() |
2024 | |
页码 | 3395-3402 |
英文摘要 | —Patient modelling has a wide range of applications in medicine and healthcare, such as clinical teaching, surgery navigation and automatic robotized scanning. While patients are typically covered or occluded in medical scenes, directly regressing human meshes from single RGB images is challenging. To this end, we design a deep learning-based patient anatomy recon struction network from RGB-D images with three key modules: 1) the attention-based multimodal fusion module, 2) the analytical inverse kinematics module and 3) the anatomical layer module. In our pipeline, the color and depth modality are fully fused by the multimodal attention module to obtain a cover-insensitive feature map. The estimated 3D keypoints, learned from the fused feature, are further converted to patient model parameters through the embedded analytical inverse kinematics module. To capture more detailed patient structures, we also present a parametric anatomy avatar by extending the Skinned Multi-Person Linear Model (SMPL) with internal bone and artery models. Final meshes are driven by the predicted parameters via the anatomical layer module, generating digital twins of patients. Experimental results on the Simultaneously-Collected Multimodal Lying Pose Dataset demonstrate that our approach surpasses state-of-the-art human mesh recovery methods and shows robustness to occlusions. |
语种 | 英语 |
内容类型 | 期刊论文 |
源URL | [http://ir.ia.ac.cn/handle/173211/57356] ![]() |
专题 | 智能微创医疗技术团队 |
通讯作者 | Hongbin Liu |
作者单位 | 1.State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China 2.Centre for Artificial In telligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences, Hong Kong 3.School of Biomedical Engineering and Imaging Sciences, King’s College London, SE1 7EU London, U.K. 4.Department of Biomedical Engineering, City University of Hong Kong, Hong Kong 5.School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China |
推荐引用方式 GB/T 7714 | Tongxi Zhou,Mingcong Chen,Guanglin Cao,et al. Inverse Kinematics Embedded Network for Robust Patient Anatomy Avatar Reconstruction From Multimodal Data[J]. IEEE Robotics and Automation Letters,2024:3395-3402. |
APA | Tongxi Zhou,Mingcong Chen,Guanglin Cao,Jian, Hu,&Hongbin Liu.(2024).Inverse Kinematics Embedded Network for Robust Patient Anatomy Avatar Reconstruction From Multimodal Data.IEEE Robotics and Automation Letters,3395-3402. |
MLA | Tongxi Zhou,et al."Inverse Kinematics Embedded Network for Robust Patient Anatomy Avatar Reconstruction From Multimodal Data".IEEE Robotics and Automation Letters (2024):3395-3402. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论