针对室内场景下光照变化、物体相互遮挡以及类别复杂等问题,提出了一种基于双流加权Gabor卷积网络融合的彩色-深度(RGB-D)图像语义分割方法。为了获得方向和尺度不变性特征,设计了一种加权Gabor方向滤波器用于构建深度卷积网络(DCN),提取对方向和尺度变化具有适应性的特征信息。为了构建轻量级特征提取网络,采用宽残差-加权Gabor卷积网络分别提取彩色和深度双流图像特征,并利用金字塔池化模块对提取的深度特征进行多尺度融合以丰富图像上下文信息。对所提语义分割方法在NYUDv2数据集上进行实验,分别设置不同的对比方法。结果表明所提方法具有合理性和有效性,并在分割效果上具有一定的竞争性。

To handle the problems of illumination change, mutual occlusion of objects, and complicated semantic categories in indoor scenes, a color-depth (RGB-D) image semantic segmentation method based on the dual-stream weighted Gabor convolutional network fusion is proposed in this work. In order to obtain direction and scale invariant features, a weighted Gabor direction filter is designed to construct a deep convolution network (DCN) to extract feature information that is adaptive to direction and scale changes. In order to build a lightweight feature extraction network, a wide residual weighted Gabor convolutional network module is used to extract color and depth dual-stream image features, and a pyramid pooling module is used to fuse the extracted depth features to enrich the image context information. The proposed semantic segmentation method is tested on NYUDv2 dataset, and different comparison methods are set up. The results show that the proposed method is reasonable and effective, and the segmentation effect is competitive.