Rethinking semantic-visual alignment in zero-shot object detection via a softplus margin focal loss

doi:10.1016/j.neucom.2021.03.073

CORC > 自动化研究所 > 中国科学院自动化研究所 > 复杂系统管理与控制国家重点实验室 > 先进机器人控制团队

	Rethinking semantic-visual alignment in zero-shot object detection via a softplus margin focal loss
	Li, Qianzhong 1,2; Zhang, Yujia 1; Sun, Shiying 1; Zhao, Xiaoguang 1; Li, Kang 3; Tan, Min 1
刊名	NEUROCOMPUTING
	2021-08-18
卷号	449 页码:117-135
关键词	Zero-shot object detection Softplus margin focal loss Semantic-visual alignment Auto-encoder architecture
ISSN号	0925-2312
DOI	10.1016/j.neucom.2021.03.073
通讯作者	Li, Qianzhong(liqianzhong2017@ia.ac.cn)
英文摘要	Zero-shot object detection (ZSD) aims to locate and recognize novel objects without additional training samples. Most existing methods usually map visual features to semantic space, resulting in a hubness problem, and learning an effective feature mapping between the two modalities remains a considerable challenge. In this work, we propose a novel end-to-end framework, Semantic-Visual Auto-Encoder (SVAE) network, to tackle the above issues. Distinct from previous works that utilize fully-connected layers to learn the feature mapping, we implement a 1-dimensional convolution with various shared filters to con-struct the auto-encoder, which maps semantic features to visual space to alleviate the hubness problem. Specifically, we design a novel loss function, Softplus Margin Focal Loss (SMFL), for object classification channel to align the projected semantic features in visual space and address the class imbalance problem. The SMFL improves the discrimination of projections on positive and negative categories and maintains the property of focal loss. Besides, to promote the localization performance for novel objects, we also pro -vide semantic information for object localization channel and utilize a trainable matrix to align the semantic-visual mapping, considering noises in semantic representations. We conduct extensive exper-iments on four challenging benchmarks. The experimental results show the competitive performances compared with state-of-the-art approaches. Especially, we achieve 8.39%/6.58% mean average precision (mAP) improvements for ZSD/general-ZSD on Microsoft COCO benchmark. (c) 2021 Elsevier B.V. All rights reserved.
资助项目	National Key Research and Development Project of China[2019YFB1310601] ; National Key R&D Program of China[2017YFC082020303] ; National Natural Science Foundation of China[61673378]
WOS关键词	ATTRIBUTES
WOS研究方向	Computer Science
语种	英语
出版者	ELSEVIER
WOS记录号	WOS:000652818400011
资助机构	National Key Research and Development Project of China ; National Key R&D Program of China ; National Natural Science Foundation of China
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/45217]
专题	自动化研究所_复杂系统管理与控制国家重点实验室_先进机器人控制团队
通讯作者	Li, Qianzhong
作者单位	1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligences, Beijing, Peoples R China 3.Information Sci Acad China Elect Technol Grp Corp, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Li, Qianzhong,Zhang, Yujia,Sun, Shiying,et al. Rethinking semantic-visual alignment in zero-shot object detection via a softplus margin focal loss[J]. NEUROCOMPUTING,2021,449:117-135.
APA	Li, Qianzhong,Zhang, Yujia,Sun, Shiying,Zhao, Xiaoguang,Li, Kang,&Tan, Min.(2021).Rethinking semantic-visual alignment in zero-shot object detection via a softplus margin focal loss.NEUROCOMPUTING,449,117-135.
MLA	Li, Qianzhong,et al."Rethinking semantic-visual alignment in zero-shot object detection via a softplus margin focal loss".NEUROCOMPUTING 449(2021):117-135.