Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation

	Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation
	Hua, Yan; Wang, Shuhui; Liu, Siyuan; Cai, Anni; Huang, Qingming
刊名	IEEE TRANSACTIONS ON MULTIMEDIA
	2016
英文摘要	With the explosive growth of web data, effective and efficient technologies are in urgent need for retrieving semantically relevant contents of heterogeneous modalities. Previous studies devote efforts to modeling simple cross-modal statistical dependencies, and globally projecting the heterogeneous modalities into a measurable subspace. However, global projections cannot appropriately adapt to diverse contents, and the naturally existing multilevel semantic relation in web data is ignored. We study the problem of semanticcoherent retrieval, where documents from different modalities should be ranked by the semantic relevance to the query. Accordingly, we propose TINA, a correlation learning method by adaptive hierarchical semanticaggregation. First, by joint modeling of content and ontology similarities, we build a semantic hierarchy to measure multilevel semantic relevance. Second, with a set of local linear projections and probabilistic membership functions, we propose two paradigms for local expert aggregation, i.e., local projectionaggregation and local distance aggregation. To learn the cross-modal projections, we optimize the structure risk objective function that involves semantic coherence measurement, local projection consistency, and the complexity penalty of local projections. Compared to existing approaches, a better bias-variance tradeoff is achieved by TINA in real-world cross-modal correlation learning tasks. Extensive experiments on widely used NUS-WIDE and ICML-Challenge for image-text retrieval demonstrate that TINA better adapts to the multilevel semantic relation and content divergence, and, thus, outperforms state of the art with bettersemantic coherence.
收录类别	SCI
原文出处	http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7422147
语种	英语
内容类型	期刊论文
源URL	[http://ir.siat.ac.cn:8080/handle/172644/10244]
专题	深圳先进技术研究院_数字所
作者单位	IEEE TRANSACTIONS ON MULTIMEDIA
推荐引用方式 GB/T 7714	Hua, Yan,Wang, Shuhui,Liu, Siyuan,et al. Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2016.
APA	Hua, Yan,Wang, Shuhui,Liu, Siyuan,Cai, Anni,&Huang, Qingming.(2016).Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation.IEEE TRANSACTIONS ON MULTIMEDIA.
MLA	Hua, Yan,et al."Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation".IEEE TRANSACTIONS ON MULTIMEDIA (2016).