Multi-Level Multi-Modal Cross-Attention Network for Fake News Detection

doi:10.1109/ACCESS.2021.3114093

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 多媒体计算与图形学团队

	Multi-Level Multi-Modal Cross-Attention Network for Fake News Detection
	Ying, Long 1; Yu, Hui 1; Wang, Jinguang 2; Ji, Yongze 3; Qian, Shengsheng 4
刊名	IEEE ACCESS
	2021
卷号	9 页码:132363-132373
关键词	Feature extraction Semantics Visualization Task analysis Bit error rate Convolutional neural networks Social networking (online) Multi-level neural networks fake news detection multi-modal fusion
ISSN号	2169-3536
DOI	10.1109/ACCESS.2021.3114093
通讯作者	Ying, Long(lorin_ying@hotmail.com) ; Yu, Hui(20191221027@nuist.edu.cn)
英文摘要	With the development of the Mobile Internet, more and more users publish multi-modal posts on social media platforms. Fake news detection has become an increasingly challenging task. Although there are many works using deep schemes to extract and combine textual and visual representation in the post, most existing methods do not sufficiently utilize the complementary multi-modal information containing semantic concepts and entities to complement and enhance each modality. Moreover, these methods do not model and incorporate the rich multi-level semantics of text information to improve fake news detection tasks. In this paper, we propose a novel end-to-end Multi-level Multi-modal Cross-attention Network (MMCN) which exploits the multi-level semantics of textual content and jointly integrates the relationships of duplicate and different modalities (textual and visual modality) of social multimedia posts in a unified framework. Pre-trained BERT and ResNet models are employed to generate high-quality representations for text words and image regions respectively. A multi-modal cross-attention network is then designed to fuse the feature embeddings of the text words and image regions by simultaneously considering data relationships in duplicate and different modalities. Specially, due to different layers of the transformer architecture have different feature representations, we employ a multi-level encoding network to capture the rich multi-level semantics to enhance the presentations of posts. Extensive experiments on the two public datasets (WEIBO and PHEME) demonstrate that compared with the state-of-the-art models, the proposed MMCN has an advantageous performance.
资助项目	National Natural Science Foundation of China[61902193] ; Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD)
WOS研究方向	Computer Science ; Engineering ; Telecommunications
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000702539000001
资助机构	National Natural Science Foundation of China ; Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD)
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/45759]
专题	自动化研究所_模式识别国家重点实验室_多媒体计算与图形学团队
通讯作者	Ying, Long; Yu, Hui
作者单位	1.Nanjing Univ Informat Sci & Technol, Sch Comp & Software, Nanjing 210044, Peoples R China 2.Hefei Univ Technol, Sch Comp Sci & Informat Engn, Hefei 230601, Peoples R China 3.China Univ Petr, Sch Informat Sci & Engn, Beijing 102249, Peoples R China 4.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Ying, Long,Yu, Hui,Wang, Jinguang,et al. Multi-Level Multi-Modal Cross-Attention Network for Fake News Detection[J]. IEEE ACCESS,2021,9:132363-132373.
APA	Ying, Long,Yu, Hui,Wang, Jinguang,Ji, Yongze,&Qian, Shengsheng.(2021).Multi-Level Multi-Modal Cross-Attention Network for Fake News Detection.IEEE ACCESS,9,132363-132373.
MLA	Ying, Long,et al."Multi-Level Multi-Modal Cross-Attention Network for Fake News Detection".IEEE ACCESS 9(2021):132363-132373.