Semantic segmentation of urban street scene images based on improved U-Net network

ZHU Fuzhen; CUI Jingyi; ZHU Bing; LI Huiling; and LIU Yan

doi:doi:10.1007/s11801-023-2128-8

光电子快报（英文版）, 2023, 19 (3): 179, Published Online: Mar. 18, 2023

Semantic segmentation of urban street scene images based on improved U-Net network

论文大纲

ZHU Fuzhen ^1,*CUI Jingyi ¹ZHU Bing ²LI Huiling ¹and LIU Yan ¹

Author Affiliations

¹ College of Electronic Engineering, Heilongjiang University, Harbin 150080, China

² Institute of Image Information Technology and Engineering, Harbin Institute of Technology, Harbin 150001, China

Abstract

To balance the speed and accuracy in semantic segmentation of the urban street images for autonomous driving, we proposed an improved U-Net network. Firstly, to improve the model representation capability, our improved U-Net network structure was designed as three parts, shallow layer, intermediate layer and deep layer. Different attention mechanisms were used according to their feature extraction characteristics. Specifically, a spatial attention module was used in the shallow network, a dual attention module was used in the intermediate layer network and a channel attention module was used in the deep network. At the same time, the traditional convolution was replaced by depthwise separable convolution in above three parts, which can largely reduce the number of network parameters, and improve the network operation speed greatly. The experimental results on three datasets show that our improved U-Net semantic segmentation model for street images can get better results in both segmentation accuracy and speed. The average mean intersection over union (MIoU) is 68.8%, which is increased by 9.2% and the computation speed is about 38 ms/frame. We can process 27 frames images for segmentation per second, which meets the real-time process and accuracy requirements for semantic segmentation of urban street images.

References

[1] ZHOU J M, LI B J, CHEN S Z. A real-time segmentation method of road scene based on multi-layer feature fusion[J]. Surveying and mapping bulletin, 2020, (1): 10-15.

[2] MO Y, WU Y, YANG X, et al. Review the state-of-the-art technologies of semantic segmentation based on deep learning[J]. Neurocomputing, 2022, 493: 626-646.

[3] BAI J, HAO P H, CHEN S H. Traffic scene understanding using lightweight convolutional neural network image semantic segmentation[J]. Journal of automotive safety and energy, 2018, 9(04): 433-440.

[4] SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(4): 640-651.

[5] LIU W M, XIN Y L, JIANG X Y. Semantic segmentation of residual network image combined with jump connection[J]. Information technology, 2020, 44(06): 5-9.

[6] YANG C J. Image semantic segmentation based on convolutional neural network[D]. Lanzhou: Northwest Normal University, 2020: 25.

[7] BADRINARAYANAN V, KENDALL A, CIPOLLA R, et al. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(12): 2481-2495.

[8] YU F, KOLTUN V, FUNKHOUSER T. Dilated residual network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, July 21-26, 2017, Hawaii, USA. New York: IEEE, 2017: 472-480.

[9] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL]. (2014-12-22)[2022-06-20]. https: //arxiv.org/abs/1412.7062.

[10] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 40(4): 834-848.

[11] ZHANG Y H, LIU H, TIAN W, et al. A method of rain cloud cluster segmentation in Tibet based on DeepLabV3[J]. Journal of computer applications, 2020, 40(09): 2781-2788.

[12] KUMAR P, SHANKAR H A. Convolutional neural network with batch normalisation for fault detection in squirrel cage induction motor[J]. IET electric power applications, 2021, 15(1): 39-50.

[13] FU J, LIU J, TIAN H, et al. Dual attention network for scene segmentation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 16-20, 2019, Long Beach, CA, USA. New York: IEEE, 2019: 3146-3154.

[14] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional network for biomedical image segmention[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention, October 5-9, 2015, Munich, Germany.: Berlin, Heidelberg: Springer-Verlag, 2015: 234-241.

[15] CHEN Z, LI D, FAN W, et al. Self-attention in reconstruction bias U-Net for semantic segmentation of building rooftops in optical remote sensing images[J]. Remote sensing, 2021, 13(13): 2524.

[16] XIAO J Q. Semantic segmentation of road scene based on deep learning[D]. Changchun: Jilin University, 2019: 23-27.

[17] WU T. Research on road scene semantic segmentation algorithm based on fully convolutional neural network[D]. Chongqing: Southwest University, 2020: 14-16.

[18] YU F. Research and implementation of multi-scene image semantic segmentation based on fully convolutional neural network[C]//3rd International Conference on Mechatronics Engineering and Information Technology (ICMEIT 2019), March 29-30, 2019, Dalian, China. Paris: Atlantis Press, 2019: 156-161.

[19] CHEN Z, LI D, FAN W, et al. Self-attention in reconstruction bias U-Net for semantic segmentation of building rooftops in optical remote sensing images[J]. Remote sensing, 2021, 13(13): 2524.

[20] LUO P F. Research on semantic segmentation of autonomous driving city scene[D]. Wuhan: Wuhan University, 2019: 16-22.

[21] YUAN X, SHI J, GU L. A review of deep learning methods for semantic segmentation of remote sensing imagery[J]. Expert systems with applications, 2021, 169: 114417.

[22] ZHANG L, HU X, ZHOU Y, et al. Memristive DeepLab: a hardware friendly deep CNN for semantic segmentation[J]. Neurocomputing, 2021, 451: 181-191.

ZHU Fuzhen, CUI Jingyi, ZHU Bing, LI Huiling, and LIU Yan. Semantic segmentation of urban street scene images based on improved U-Net network[J]. 光电子快报（英文版）, 2023, 19(3): 179.

Semantic segmentation of urban street scene images based on improved U-Net network

关于本站 Cookie 的使用提示

全站搜索

Semantic segmentation of urban street scene images based on improved U-Net network

相关论文

相关资讯

关于本站 Cookie 的使用提示

全站搜索