为满足自动驾驶、人机交互等任务对语义分割算法准确度和实时性的要求,提出一种基于特征融合技术的实时语义分割算法。首先,利用卷积神经网络自动学习图像深层特征的功能,设计一个浅而宽的空间信息网络输出低级别的空间信息,以保持原始空间信息完整性,从而生成高分辨率特征;接着,设计一个语境信息网络来输出深层次、高级别的语境信息,并引入注意力优化机制来代替上采样,优化网络的输出;最后,将两路输出特征图进行多尺度融合,再上采样得到与原始输入尺寸相等的分割图像。两路网络并行计算,提高了算法的实时性。在Cityscapes、CamVid数据集上对该网络框架进行一系列实验。其中,在Cityscapes数据集上取得了68.43%的均交并比(MIOU)。对于640×480的图像输入,在一块NVIDIA 1050T显卡上的速度为14.14 frame/s。本文算法在准确度上大幅超越现有实时分割算法,基本满足人机交互类任务对实时性的要求。

In this study, we propose a real-time semantic segmentation algorithm based on the feature fusion technology to satisfy the requirements of autopilot, human-computer interaction, and other tasks with respect to accuracy and real-time capability . Here, we use a convolutional neural network to automatically learn deep features of the image. We design a shallow and wide spatial information network to output low-level spatial information for ensuring the integrity of the original spatial information and generating high-resolution features. Furthermore, we design a context information network to output deep-level high-level context information. Then, we introduce an attention optimization mechanism to replace upsampling for optimizing the network output. Finally, we fuse the two output feature maps on multiple scales and perform upsampling to obtain a segmented image with a size equal to the original input size. Subsequently, we perform a simulation using two-way network parallel computing to improve the real-time performance of the proposed algorithm. The network framework achieves 68.43% mean intersection over union (MIOU) on the Cityscapes dataset. In case of an image input of 640 × 480, the speed obtained using an NVIDIA 1050T graphics card is 14.14 frame/s. Furthermore, the accuracy considerably exceeds that of the existing real-time segmentation algorithm, satisfying the real-time requirements of the human-computer interaction tasks.