从不完整的视觉信息中推断出物体的三维几何形状是机器视觉系统应当具备的重要能力, 而识别出场景中物体的语义是机器视觉系统的核心。传统方法通常将二者分离实现, 本文将场景复原与目标语义紧密结合, 提出了一种三维语义场景复原网络模型, 仅以单一深度图作为输入, 实现对三维场景的语义分类和场景复原。首先, 建立一种端到端的三维卷积神经网络, 网络的输入是深度图, 使用三维上下文模块来对相机视锥体内的区域进行学习, 进而输出带有语义标签的三维体素; 其次, 建立了带有密集体积标签的合成三维场景数据集, 用于训练本文的深度学习网络模型; 最后通过实验表明, 与现有的语义分类和场景复原方法相比, 语义场景的复原接收区域增加了2.0%。结果表明: 三维学习网络的复原性能良好, 语义标注的准确率较高。

Reconstruction of 3D object is an important part in machine vision system, and the semantic understanding of 3D object is a core function for the machine vision system. In this paper, 3D restoration was combined with the semantic understanding of 3D object, a 3D semantic scene recovery network was proposed. The semantic classification and scene restoration of 3D scene were achieved only by using a single RGB-D map as input. Firstly, an end-to-end 3D convolution neural network was established. The input of the network was a depth map. The 3D context module was used for learning the region within the camera view, then the 3D voxels with semantic labels were generated. Secondly, a synthetic data set with dense volume labels was established to train the depth learning network. Finally, the experimental results showed that the recovery performance w improved by 2.0% compared with the state-of-art. It can be seen that the 3D learning network plays well in 3D scene restoration, it owns high accuracy in semantic annotation of object in the scene.