Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition

doi:10.1109/TIP.2019.2929447

CORC > 计算技术研究所 > 中国科学院计算技术研究所 > 中国科学院计算技术研究所期刊论文 > 英文

	Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition
	Jiang, Shuqiang 1,2; Min, Weiqing 1,2; Liu, Linhu 1,2; Luo, Zhengdong 1,2
刊名	IEEE TRANSACTIONS ON IMAGE PROCESSING
	2020
卷号	29 页码:265-276
关键词	Food recognition ingredient knowledge feature aggregation convolutional neural networks
ISSN号	1057-7149
DOI	10.1109/TIP.2019.2929447
英文摘要	Recently, food recognition has received more and more attention in image processing and computer vision for its great potential applications in human health. Most of the existing methods directly extracted deep visual features via convolutional neural networks (CNNs) for food recognition. Such methods ignore the characteristics of food images and are, thus, hard to achieve optimal recognition performance. In contrast to general object recognition, food images typically do not exhibit distinctive spatial arrangement and common semantic patterns. In this paper, we propose a multi-scale multi-view feature aggregation (MSMVFA) scheme for food recognition. MSMVFA can aggregate high-level semantic features, mid-level attribute features, and deep visual features into a unified representation. These three types of features describe the food image from different granularity. Therefore, the aggregated features can capture the semantics of food images with the greatest probability. For that solution, we utilize additional ingredient knowledge to obtain mid-level attribute representation via ingredient-supervised CNNs. High-level semantic features and deep visual features are extracted from class-supervised CNNs. Considering food images do not exhibit distinctive spatial layout in many cases, MSMVFA fuses multi-scale CNN activations for each type of features to make aggregated features more discriminative and invariable to geometrical deformation. Finally, the aggregated features are more robust, comprehensive, and discriminative via two-level fusion, namely multi-scale fusion for each type of features and multi-view aggregation for different types of features. In addition, MSMVFA is general and different deep networks can be easily applied into this scheme. Extensive experiments and evaluations demonstrate that our method achieves state-of-the-art recognition performance on three popular large-scale food benchmark datasets in Top-1 recognition accuracy. Furthermore, we expect this paper will further the agenda of food recognition in the community of image processing and computer vision.
资助项目	National Natural Science Foundation of China[61532018] ; National Natural Science Foundation of China[61602437] ; Beijing Natural Science Foundation[L182054] ; National Program for Special Support of Eminent Professionals ; National Program for Support of Top-notch Young Professionals
WOS研究方向	Computer Science ; Engineering
语种	英语
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号	WOS:000487069300020
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/4655]
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jiang, Shuqiang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Jiang, Shuqiang,Min, Weiqing,Liu, Linhu,et al. Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2020,29:265-276.
APA	Jiang, Shuqiang,Min, Weiqing,Liu, Linhu,&Luo, Zhengdong.(2020).Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition.IEEE TRANSACTIONS ON IMAGE PROCESSING,29,265-276.
MLA	Jiang, Shuqiang,et al."Multi-Scale Multi-View Deep Feature Aggregation for Food Recognition".IEEE TRANSACTIONS ON IMAGE PROCESSING 29(2020):265-276.