首页 > 论文 > 激光与光电子学进展 > 56卷 > 13期(pp:131007--1)

基于多尺度特征提取和全连接条件随机场的图像语义分割方法

Image Semantic Segmentation Based on Multi-Scale Feature Extraction and Fully Connected Conditional Random Fields

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

针对图像语义分割中图像的上下文信息利用不充分、边缘分割不清等问题,提出一种基于多尺度特征提取与全连接条件随机场的网络模型。分别以多尺度形式将RGB图像和深度图像输入网络,利用卷积神经网络提取图像特征;将深度信息作为补充信息添加到RGB特征图,得到语义粗分割结果;采用全连接条件随机场优化语义粗分割结果,最终得到语义精细分割结果。实验结果表明,所提方法提高了图像语义分割的精度,优化了图像语义分割的边缘,具有实际应用价值。

Abstract

Aim

ing at the problems of insufficient usage of context information and unclear image edge segmentation in image semantic segmentation, a network model based on multi-scale feature extraction and fully connected conditional random fields is proposed. RGB and depth images are input into the network in a multi-scale form, and their features are extracted by a Convolutional neural network. Depth information is added to supplement the RGB feature map and obtain a rough semantic segmentation, which is optimized by the fully connected conditional random fields. Finally, fine semantic segmentation results are obtained. This proposed method improves the precision of semantic segmentation and optimizes the image edge segmentation, which has a practical application.

Newport宣传-MKS新实验室计划
补充资料

DOI:10.3788/LOP56.131007

所属栏目:图像处理

基金项目:天津市基础研究计划、天津市基础研究计划 、河北省自然科学基金;

收稿日期:2018-11-13

修改稿日期:2019-01-30

网络出版日期:2019-07-01

作者单位    点击查看

董永峰:河北工业大学人工智能与数据科学学院, 天津 300401河北省大数据计算重点实验室, 天津 300401
杨雨訢:河北工业大学人工智能与数据科学学院, 天津 300401
王利琴:河北工业大学人工智能与数据科学学院, 天津 300401河北省大数据计算重点实验室, 天津 300401

联系人作者:王利琴(wangliqin@scse.hebut.edu.cn)

备注:天津市基础研究计划、天津市基础研究计划 、河北省自然科学基金;

【1】Guo Y M, Liu Y, Georgiou T et al. A review of semantic segmentation using deep neural networks. International Journal of Multimedia Information Retrieval. 7(2), 87-93(2018).

【2】Guo C C, Yu F Q and Chen Y. Image semantic segmentation based on Convolutional neural network feature and improved superpixel matching. Laser & Optoelectronics Progress. 55(8), (2018).
郭呈呈, 于凤芹, 陈莹. 基于卷积神经网络特征和改进超像素匹配的图像语义分割. 激光与光电子学进展. 55(8), (2018).

【3】Wei Y C and Zhao Y. A review on image semantic segmentation based on DCNN. Journal of Beijing Jiaotong University. 40(4), 82-91(2016).
魏云超, 赵耀. 基于DCNN的图像语义分割综述. 北京交通大学学报. 40(4), 82-91(2016).

【4】Zhang X M, Zhu X B, Cai Q et al. Survey of the deep learning models for image semantic segmentation. Chinese High Technology Letters. 27(9), 808-815(2017).
张新明, 祝晓斌, 蔡强 等. 图像语义分割深度学习模型综述. 高技术通讯. 27(9), 808-815(2017).

【5】Rother C, Kolmogorov V and Blake A. GrabCut: interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics. 23(3), 309-314(2004).

【6】Long J, Shelhamer E and Darrell T. Fully Convolutional networks for semantic segmentation. [C]∥2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA. New York: IEEE. 3431-3440(2015).

【7】Chen L C, Papandreou G, Kokkinos I, fully connected CRFs[EB/OL] et al. -06-07)[2018-10-25]. https:∥arxiv. org/abs/1412, (2016).

【8】Badrinarayanan V, Kendall A and Cipolla R. SegNet: a deep Convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 39(12), 2481-2495(2017).

【9】Noh H, Hong S and Han B. Learning deConvolution network for semantic segmentation. [C]∥2015 IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile. New York: IEEE. 1520-1528(2015).

【10】Yu F. -04-30)[2018-10-25]. https:∥arxiv. org/abs/1511, (2016).

【11】Chen L C, Papandreou G, Kokkinos I et al. DeepLab: semantic image segmentation with deep Convolutional nets, atrous Convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence. 40(4), 834-848(2018).

【12】Zhao H S, Shi J P, Qi X J et al. Pyramid scene parsing network. [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE. 6230-6239(2017).

【13】Lin G S. Shen C H, van den Hengel A, et al. Efficient piecewise training of deep structured models for semantic segmentation. [C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE. 3194-3203(2016).

【14】Xiong Z Y, Zhang G F and Wang J Q. Multi-scale feature extract for image sematic segmentation. Journal of South-Central University for Nationalities(Natural Science Edition). 36(3), 118-124(2017).
熊志勇, 张国丰, 王江晴. 基于多尺度特征提取的图像语义分割. 中南民族大学学报(自然科学版). 36(3), 118-124(2017).

【15】Jiang Y F, Zhang H, Xue Y B et al. A new multi-scale image semantic understanding method based on deep learning. Journal of Optoelectronics·Laser. 27(2), 224-230(2016).
蒋应锋, 张桦, 薛彦兵 等. 一种新的多尺度深度学习图像语义理解方法研究. 光电子·激光. 27(2), 224-230(2016).

【16】Liu D, Liu X J and Wang M Z. Semantic segmentation with multi-scale Convolutional neural network. Remote Sensing Information. 32(1), 57-64(2017).
刘丹, 刘学军, 王美珍. 一种多尺度CNN的图像语义分割算法. 遥感信息. 32(1), 57-64(2017).

【17】Zheng S, Jayasumana S, Romera-Paredes B et al. Conditional random fields as recurrent neural networks. [C]∥2015 IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile. New York: IEEE. 1529-1537(2015).

【18】Lin G S. Shen C H, van den Hengel A, et al. Exploring context with deep structured models for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 40(6), 1352-1366(2018).

【19】Lafferty J D. McCallum A, Pereira F C N. Conditional random fields: probabilistic models for segmenting and labeling sequence data. [C]∥Proceedings of the Eighteenth International Conference on Machine Learning, June 28-July 01, 2001. San Francisco: Morgan Kaufmann Publishers Inc. 282-289(2001).

【20】Kr?henbühl P and Koltun V. Efficient inference in fully connected CRFs with Gaussian edge potentials. [C]∥Proceedings of the 24th International Conference on Neural Information Processing Systems, December 12-15, 2011, Granada, Spain. USA: Curran Associates Inc. 109-117(2011).

【21】Vemulapalli R, Tuzel O, Liu M Y et al. Gaussian conditional random field network for semantic segmentation. [C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. 3224-3233(2016).

【22】Hazirbas C, Ma L N, Domokos C et al. FuseNet: incorporating depth into semantic segmentation via fusion-based CNN architecture. ∥Lai S H, Lepetit V, Nishino K, et al.Computer vision-ACCV 2016. Lecture notes in computer science. Cham: Springer. 10111, 213-228(2017).

【23】Silberman N, Hoiem D, Kohli P et al. Indoor segmentation and support inference from RGBD images. ∥Fitzgibbon A, Lazebnik S, Perona P, et al. Computer vision-ECCV 2012. Lecture notes in computer science. Berlin, Heidelberg: Springer. 7576, 746-760(2012).

【24】Gupta S, Arbeláez P and Malik J. Perceptual organization and recognition of indoor scenes from RGB-D images. [C]∥2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 23-28, 2013, Portland, OR, USA. New York: IEEE. 564-571(2013).

【25】Gupta S, Girshick R, Arbeláez P et al. Learning rich features from RGB-D images for object detection and segmentation. ∥ Fleet D, Pajdla T, Schiele B, et al. Computer vision-ECCV 2014. Lecture notes in computer science. Cham: Springer. 8695, 345-360(2014).

【26】Deng Z, Todorovic S and Latecki L J. Semantic segmentation of RGBD images with mutex constraints. [C]∥2015 IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile. New York: IEEE. 1733-1741(2015).

引用该论文

Yongfeng Dong, Yuxin Yang, Liqin Wang. Image Semantic Segmentation Based on Multi-Scale Feature Extraction and Fully Connected Conditional Random Fields[J]. Laser & Optoelectronics Progress, 2019, 56(13): 131007

董永峰, 杨雨訢, 王利琴. 基于多尺度特征提取和全连接条件随机场的图像语义分割方法[J]. 激光与光电子学进展, 2019, 56(13): 131007

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF