A Two-Stream CNN Framework for American Sign Language Recognition Based on Multimodal Data Fusion

CORC > 沈阳自动化研究所 > 中国科学院沈阳自动化研究所 > 空间自动化技术研究室

	A Two-Stream CNN Framework for American Sign Language Recognition Based on Multimodal Data Fusion
	Gao Q(高庆)1,2,5; Ogenyi, Uchenna Emeoha 4; Liu JG(刘金国)1,2; Liu, Honghai 4
	2019
会议日期	September 4, 2019 - September 6, 2019
会议地点	Portsmouth, United kingdom
关键词	Hand gesture recognition CNN Multimodal data fusion
页码	107-118
英文摘要	At present, vision-based hand gesture recognition is very important in human-robot interaction (HRI). This non-contact method enables natural and friendly interaction between people and robots. Aiming at this technology, a two-stream CNN framework (2S-CNN) is proposed to recognize the American sign language (ASL) hand gestures based on multimodal (RGB and depth) data fusion. Firstly, the hand gesture data is enhanced to remove the influence of background and noise. Secondly, hand gesture RGB and depth features are extracted for hand gesture recognition using CNNs on two streams, respectively. Finally, a fusion layer is designed for fusing the recognition results of the two streams. This method utilizes multimodal data to increase the recognition accuracy of the ASL hand gestures. The experiments prove that the recognition accuracy of 2S-CNN can reach 92.08 $$\%$$ on ASL fingerspelling database and is higher than that of baseline methods.
产权排序	1
会议录	Advances in Computational Intelligence Systems - Contributions Presented at the 19th UK Workshop on Computational Intelligence, 2019
会议录出版者	Springer Verlag
语种	英语
ISSN号	2194-5357
ISBN号	978-3-030-29932-3
WOS记录号	WOS:000618179900009
内容类型	会议论文
源URL	[http://ir.sia.cn/handle/173321/25741]
专题	沈阳自动化研究所_空间自动化技术研究室
通讯作者	Liu JG(刘金国)
作者单位	1.State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China 2.Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, China 3.PO1 3HE, United Kingdom 4.School of Computing, University of Portsmouth, Portsmouth 5.University of Chinese Academy of Sciences, Beijing 100049, China
推荐引用方式 GB/T 7714	Gao Q,Ogenyi, Uchenna Emeoha,Liu JG,et al. A Two-Stream CNN Framework for American Sign Language Recognition Based on Multimodal Data Fusion[C]. 见:. Portsmouth, United kingdom. September 4, 2019 - September 6, 2019.