Context-Driven Image Caption With Global Semantic Relations of the Named Entities

doi:10.1109/ACCESS.2020.3013321

CORC > 计算技术研究所 > 中国科学院计算技术研究所 > 中国科学院计算技术研究所期刊论文 > 英文

	Context-Driven Image Caption With Global Semantic Relations of the Named Entities
	Jing, Yun 1,2; Zhiwei Xu 3; Guanglai Gao 2
刊名	IEEE ACCESS
	2020
卷号	8 页码:143584-143594
关键词	Semantics Visualization Task analysis Computational modeling Knowledge based systems Correlation Organizations Image caption named entity semantic relation
ISSN号	2169-3536
DOI	10.1109/ACCESS.2020.3013321
英文摘要	Automatic image captioning has achieved a great progress. However, the existing captioning frameworks basically enumerate the objects in the image. The generated captions lack the real-world knowledge about named entities and their relations, such as the relations among famous persons, organizations and buildings. On the contrary, humans interpret images in a specific way by providing real-world knowledge with relations of the aforementioned named entities. To generate human-like captions, we focus on captioning the images of news, which could provide real-world knowledge of the whole story behind the images. Then we propose a novel model that makes captions closer to the human-like description of the image, by leveraging the semantic relevance of the named entities. The named entities are not only extracted from news under the guidance of the image content, but also extended with external knowledge based on the semantic relations. In detail, we propose a sentence correlation analysis algorithm to selectively draw the contextual information from news, and use entity-linking algorithm based on knowledge graph to discover the relations of entities with a global sight. The results of extensive experiments based on real-world dataset which is collected from the news show that our model generates image captions closer to the corresponding real-world captions.
资助项目	Natural Science Foundation of China[61650205] ; Natural Science Foundation of China[61540004] ; Natural Science Foundation of Inner Mongolia Autonomous Region[2018MS06003] ; Natural Science Foundation of Inner Mongolia Autonomous Region[2020MS06025] ; Natural Science Foundation of Education Department Inner Mongolian Autonomous Region[NJZY19083]
WOS研究方向	Computer Science ; Engineering ; Telecommunications
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000560332200001
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/15786]
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jing, Yun
作者单位	1.Inner Mongolia Univ Technol, Coll Data Sci & Applicat, Hohhot 010080, Peoples R China 2.Inner Mongolia Univ, Dept Comp Sci, Hohhot 010021, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing 100080, Peoples R China
推荐引用方式 GB/T 7714	Jing, Yun,Zhiwei Xu,Guanglai Gao. Context-Driven Image Caption With Global Semantic Relations of the Named Entities[J]. IEEE ACCESS,2020,8:143584-143594.
APA	Jing, Yun,Zhiwei Xu,&Guanglai Gao.(2020).Context-Driven Image Caption With Global Semantic Relations of the Named Entities.IEEE ACCESS,8,143584-143594.
MLA	Jing, Yun,et al."Context-Driven Image Caption With Global Semantic Relations of the Named Entities".IEEE ACCESS 8(2020):143584-143594.