Spatiotemporal Grid Flow for Video Retargeting

CORC > 自动化研究所 > 中国科学院自动化研究所 > 模式识别国家重点实验室 > 图像与视频分析团队

	Spatiotemporal Grid Flow for Video Retargeting
	Li, Bing 1,2; Duan, Ling-Yu 2; Wang, Jinqiao3 ; Ji, Rongrong 2; Lin, Chia-Wen 4; Gao, Wen 2
刊名	IEEE TRANSACTIONS ON IMAGE PROCESSING
	2014-04-01
卷号	23 期号:4 页码:1615-1628
关键词	Video retargeting video warping dynamic programming quadratic programming
英文摘要	Video retargeting is a useful technique to adapt a video to a desired display resolution. It aims to preserve the information contained in the original video and the shapes of salient objects while maintaining the temporal coherence of contents in the video. Existing video retargeting schemes achieve temporal coherence via constraining each region/pixel to be deformed consistently with its corresponding region/pixel in neighboring frames. However, these methods often distort the shapes of salient objects, since they do not ensure the content consistency for regions/pixels constrained to be coherently deformed along time axis. In this paper, we propose a video retargeting scheme to simultaneously meet the two requirements. Our method first segments a video clip into spatiotemporal grids called grid flows, where the consistency of the content associated with a grid flow is maintained while retargeting the grid flow. After that, due to the coarse granularity of grid, there still may exist content inconsistency in some grid flows. We exploit the temporal redundancy in a grid flow to avoid that the grids with inconsistent content be incorrectly constrained to be coherently deformed. In particular, we use grid flows to select a set of key-frames which summarize a video clip, and resize subgrid-flows in these key-frames. We then resize the remaining nonkey-frames by simply interpolating their grid contents from the two nearest retargeted key-frames. With the key-frame-based scheme, we only need to solve a small-scale quadratic programming problem to resize subgrid-flows and perform grid interpolation, leading to low computation and memory costs. The experimental results demonstrate the superior performance of our scheme.
WOS标题词	Science & Technology ; Technology
类目[WOS]	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
研究领域[WOS]	Computer Science ; Engineering
收录类别	SCI
语种	英语
WOS记录号	WOS:000332123900005
内容类型	期刊论文
源URL	[http://ir.ia.ac.cn/handle/173211/3343]
专题	自动化研究所_模式识别国家重点实验室_图像与视频分析团队
作者单位	1.Univ Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Peking Univ, Sch Elect & Engn Comp Sci, Inst Digital Media, Beijing 100871, Peoples R China 3.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China 4.Natl Tsing Hua Univ, Hsinchu 30013, Taiwan
推荐引用方式 GB/T 7714	Li, Bing,Duan, Ling-Yu,Wang, Jinqiao,et al. Spatiotemporal Grid Flow for Video Retargeting[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2014,23(4):1615-1628.
APA	Li, Bing,Duan, Ling-Yu,Wang, Jinqiao,Ji, Rongrong,Lin, Chia-Wen,&Gao, Wen.(2014).Spatiotemporal Grid Flow for Video Retargeting.IEEE TRANSACTIONS ON IMAGE PROCESSING,23(4),1615-1628.
MLA	Li, Bing,et al."Spatiotemporal Grid Flow for Video Retargeting".IEEE TRANSACTIONS ON IMAGE PROCESSING 23.4(2014):1615-1628.