首页 > 论文 > 光学学报 > 39卷 > 1期(pp:110001--1)

基于超像素时空特征的视频显著性检测方法

Video Saliency Detection Method Based on Spatiotemporal Features of Superpixels

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

提出一种基于超像素时空特征的视频显著性检测方法。所提方法可对图像进行超像素分割, 提取颜色梯度和运动梯度特征, 以构建超像素级时空梯度图。用平均加权测地距离来衡量时空梯度图上每一个超像素相对于其邻域的时空显著程度, 形成时空显著图。根据时间域上目标运动的连续性, 并借助熵的概念来表征运动模式的一致程度, 构建运动一致性图。融合时空显著图和运动一致性图, 通过自适应阈值处理定位运动目标。实验从可视化分析和定量评估两个方面将所提方法与其他算法进行对比, 结果表明所提方法具有较强的抗环境干扰能力, 适用于背景纹理复杂或环境随机变化的视频中运动目标的检测,其检测准确率高达92%。

Abstract

A novel video saliency detection method is proposed based on the spatiotemporal features of superpixels, which is used to the superpixel segmentation of images and extract the features of color gradient and motion gradient for the construction of a spatial-temporal gradient map of superpixels. The average weighted geodesic distance is used to measure the spatiotemporal saliency degree of each superpixel relative to its neighbor on the spatiotemporal gradient map, and thus the spatiotemporal saliency map is formed. In order to obtain the motion coherency map, the motion entropy in the multiple continuous frames is computed to represent the motion coherence of motion object over time. The fusion of spatiotemporal saliency maps and motion coherency maps is applied to locate in the salient motion using adaptive segmentation. In addition, the performance of the proposed method is compared with those of the other algorithms in experiments from two perspectives of visual analysis and qualitative evaluation. The results show that the proposed method is robust and suitable for the detection of moving targets in videos within complex background texture and changeable environment. Moreover, the detection precision is up to 92%.

Newport宣传-MKS新实验室计划
补充资料

中图分类号:TP391

DOI:10.3788/aos201939.0110001

所属栏目:图像处理

基金项目:吉林省科技发展计划 (20160520018JH)

收稿日期:2018-07-10

修改稿日期:2018-07-22

网络出版日期:2018-08-13

作者单位    点击查看

李艳荻:长春理工大学光电工程学院, 吉林 长春 130022
徐熙平:长春理工大学光电工程学院, 吉林 长春 130022

联系人作者:徐熙平(xxp@cust.edu.cn)

【1】Borji A, Itti L. State-of-the-art in visual attention modeling[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 185-207.

【2】Lee W F, Huang T H, Yeh S L, et al. Learning-based prediction of visual attention for video signals[J]. IEEE Transactions on Image Processing, 2011, 20(11): 3028-3038.

【3】Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(11): 1254-1259.

【4】Zhang Q R, Xiao H M. Biologically motivated salient regions detection approach[C]. Second International Symposium on Intelligent Information Technology Application, 2008: 1100-1104.

【5】Lin W S, Huang Y W. Intention-oriented computational visual attention model for learning and seeking image content[C]. IEEE Conference on Industrial Electronics and Applications, 2009: 1250-1255.

【6】Itti L, Baldi P. A principled approach to detecting surprising events in video[C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR′05), 2005: 631-637.

【7】Guo C L, Zhang L M. A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression[J]. IEEE Transactions on Image Processing, 2010, 19(1): 185-198.

【8】Li X H, Lu H C, Zhang L H, et al. Saliency detection via dense and sparse reconstruction[C]. IEEE International Conference on Computer Vision, 2013: 2976-2983.

【9】Liu Z, Zhang X, Luo S H, et al. Superpixel-based spatiotemporal saliency detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(9): 1522-1540.

【10】Wang W G, Shen J B, Shao L. Consistent video saliency using local gradient flow optimization and global refinement[J]. IEEE Transactions on Image Processing, 2015, 24(11): 4185-4196.

【11】Singh A, Henry C H, Pratt M A. Learning to predict video saliency using temporal superpixels[C]. International Conference on Pattern Recognition Applications and Methods, 2015: 201-209.

【12】Rahtu E, Kannala J, Salo M, et al. Segmenting Salient Objects from Images and Videos[M]//Rahtu E, Kannala J, Salo M, et al. eds. Heidelberg: Springer Berlin Heidelberg, 2010: 366-379.

【13】Fang Y M, Wang Z, Lin W S. Video saliency incorporating spatiotemporal cues and uncertainty weighting[C]. IEEE International Conference on Multimedia and Expo (ICME), 2013: 3910-3921.

【14】Li Q W, Zhou Y Q, Ma Y P, et al. Salient object detection method based on binocular vision[J]. Acta Optica Sinica, 2018, 38(3): 331-343.
李庆武, 周亚琴, 马云鹏, 等. 基于双目视觉的显著性目标检测方法[J]. 光学学报, 2018, 38(3): 331-343.

【15】Brox T, Malik J. Large displacement optical flow: descriptor matching in variational motion estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(3): 500-513.

【16】Wei Y C, Wen F, Zhu W J, et al. Geodesic saliency using background priors[M]//Wei Y C, Wen F, Zhu W J, et al. eds. Heidelberg: Springer Berlin Heidelberg, 2012: 29-42.

【17】Yang X, Zhang L L, Liang Y M. Salient region detection based on super-pixels and shortest geodesic distance after L0 smoothing[J]. Journal of Optoelectronics·Laser, 2017, 28(6): 657-662.
杨鑫, 张雷雷, 梁艳梅. 基于L0平滑的超像素块最短测地距离的显著区域提取方法[J]. 光电子·激光, 2017, 28(6): 657-662.

【18】Tsai D, Flagg M, Nakazawa A, et al. Motion coherent tracking using multi-label MRF optimization[J]. International Journal of Computer Vision, 2012, 100(2): 190-202.

【19】Hou X D, Harel J, Koch C. Image signature: highlighting sparse salient regions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(1): 194-201.

【20】Li Q L, He J F. Vehicles detection based on three-frame-difference method and cross-entropy threshold method[J]. Computer Engineering, 2011, 37(4): 172-174.
李秋林, 何家峰. 基于三帧差法和交叉熵阈值法的车辆检测[J]. 计算机工程, 2011, 37(4): 172-174.

【21】Barnich O, van Droogenbroeck M. ViBe: A universal background subtraction algorithm for video sequences[J]. IEEE Transactions on Image Processing, 2011, 20(6): 1709-1724.

【22】Wang X B, Liu C P, Fei L Y, et al. Foreground object detection method using kernel density estimation of a local spatio-temporal model[J]. Journal of Image and Graphics, 2012, 17(7): 813-820.
王兴宝, 刘纯平, 费兰英, 等. 局部时空域模型的核密度估计目标检测方法[J]. 中国图象图形学报, 2012, 17(7): 813-820.

【23】Suo P, Wang Y J. An improved adaptive background modeling algorithm based on Gaussian Mixture Model[C]. 9th International Conference on Signal Processing, 2008: 1436-1439.

【24】Zhang Z H, Xu Y, Su Q, et al. Moving object detection based on background subtraction and hybrid frame differencing[J]. Radio Engineering, 2012, 42(8): 14-17.
张正华, 许晔, 苏权, 等. 基于背景差分和混合帧差的运动目标检测[J]. 无线电工程, 2012,42(8): 14-17.

引用该论文

Li Yandi,Xu Xiping. Video Saliency Detection Method Based on Spatiotemporal Features of Superpixels[J]. Acta Optica Sinica, 2019, 39(1): 0110001

李艳荻,徐熙平. 基于超像素时空特征的视频显著性检测方法[J]. 光学学报, 2019, 39(1): 0110001

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF