Few-shot Food Recognition via Multi-view Representation Learning

doi:10.1145/3391624

CORC > 计算技术研究所 > 中国科学院计算技术研究所 > 中国科学院计算技术研究所期刊论文 > 英文

	Few-shot Food Recognition via Multi-view Representation Learning
	Jiang, Shuqiang 1; Min, Weiqing 1; Lyu, Yongqiang 2,3; Liu, Linhu 1
刊名	ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
	2020-09-01
卷号	16 期号:3 页码:20
关键词	Food recognition few-shot learning visual recognition deep learning
ISSN号	1551-6857
DOI	10.1145/3391624
英文摘要	This article considers the problem of few-shot learning for food recognition. Automatic food recognition can support various applications, e.g., dietary assessment and food journaling. Most existing works focus on food recognition with large numbers of labelled samples, and fail to recognize food categories with few samples. To address this problem, we propose a Multi-View Few-Shot Learning (MVFSL) framework to explore additional ingredient information for few-shot food recognition. Besides category-oriented deep visual features, we introduce ingredient-supervised deep network to extract ingredient-oriented features. As general and intermediate attributes of food, ingredient-oriented features are informative and complementary to category-oriented features, and thus they play an important role in improving food recognition. Particularly in few-shot food recognition, ingredient information can bridge the gap between disjoint training categories and test categories. To take advantage of ingredient information, we fuse these two kinds of features by first combining their feature maps from their respective deep networks and then convolving combined feature maps. Such convolution is further incorporated into a multi-view relation network, which is capable of comparing pairwise images to enable fine-grained feature learning. MVFSL is trained in an end-to-end fashion for joint optimization on two types of feature learning subnetworks and relation subnetworks. Extensive experiments on different food datasets have consistently demonstrated the advantage of MVFSL in multi-view feature fusion. Furthermore, we extend another two types of networks, namely, Siamese Network and Matching Network, by introducing ingredient information for few-shot food recognition. Experimental results have also shown that introducing ingredient information into these two networks can improve the performance of few-shot food recognition.
资助项目	National Natural Science Foundation of China[61532018] ; National Natural Science Foundation of China[61972378] ; National Natural Science Foundation of China[U19B2040] ; Beijing Natural Science Foundation[L182054] ; National Program for Special Support of Eminent Professionals ; National Program for Support of Top-notch Young Professionals
WOS研究方向	Computer Science
语种	英语
出版者	ASSOC COMPUTING MACHINERY
WOS记录号	WOS:000569375200013
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/15560]
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jiang, Shuqiang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, 6 Kexueyuan South Rd, Beijing 100190, Peoples R China 2.Qingdao KingAgroot Precis Agr Technol Co Ltd, Qingdao, Peoples R China 3.Shandong Reebow Automat Equipment Co LTD, Qingdao Branch, Room 1901,Bldg 5, Qingdao, Shandong, Peoples R China
推荐引用方式 GB/T 7714	Jiang, Shuqiang,Min, Weiqing,Lyu, Yongqiang,et al. Few-shot Food Recognition via Multi-view Representation Learning[J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,2020,16(3):20.
APA	Jiang, Shuqiang,Min, Weiqing,Lyu, Yongqiang,&Liu, Linhu.(2020).Few-shot Food Recognition via Multi-view Representation Learning.ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS,16(3),20.
MLA	Jiang, Shuqiang,et al."Few-shot Food Recognition via Multi-view Representation Learning".ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS 16.3(2020):20.