基于多尺度特征提取和全连接条件随机场的图像语义分割方法 
下载： 1298次
Image Semantic Segmentation Based on Multi-Scale Feature Extraction and Fully Connected Conditional Random Fields
1 河北工业大学人工智能与数据科学学院, 天津 300401
2 河北省大数据计算重点实验室, 天津 300401
图 & 表
图 1. 特征融合前的单分支网络结构
Fig. 1. Single-branch network structure before feature fusion
下载图片 查看原文
图 2. 多尺度特征融合
Fig. 2. Multi-scale feature fusion
下载图片 查看原文
图 3. FullCRF优化语义粗分割结果
Fig. 3. FullCRF optimization semantic rough segmentation result
下载图片 查看原文
图 4. 分割结果对比图
Fig. 4. Comparison of segmentation results
下载图片 查看原文
表 1特征融合前单个分支编码器部分的参数设置表
Table1. Parameter setting table of single branch encoder before feature fusion
| RGB encoder | Depth encoder |
|---|
| Conv block1:3×3 Conv 643×3 Conv 642×2 maxpooling | Conv block2:3×3 Conv 1283×3 Conv 1282×2 maxpooling | Conv block3:3×3 Conv 2563×3 Conv 2562×2 maxpooling | Conv block1:3×3 Conv 643×3 Conv 642×2 maxpooling | Conv block2:3×3 Conv 1283×3 Conv 1282×2 maxpooling | Conv block3:3×3 Conv 2563×3 Conv 2562×2 maxpooling | | Conv block4:3×3 Conv 5123×3 Conv 5123×3 Conv 5122×2 maxpooling | Conv block5:3×3 Conv 5123×3 Conv 5123×3 Conv 5122×2 maxpooling | | Conv block4:3×3 Conv 5123×3 Conv 5123×3 Conv 5122×2 maxpooling | Conv block5:3×3 Conv 5123×3 Conv 5123×3 Conv 512 | |
|
查看原文
表 2不同网络在NYUv2数据集上的结果
Table2. Results of different networks on NYUv2 dataset
| Method | Inputdata type | PA /% | MA /% | MIoU /% |
|---|
| Method in Ref. [6] | RGB | 60.0 | 42.2 | 29.2 | | Method in Ref. [6] | Depth | 57.1 | 35.2 | 24.2 | | Method in Ref. [25] | RGB-depth | 60.3 | - | 28.6 | | Method in Ref. [26] | RGB-depth | 63.8 | 31.5 | - | | Method in Ref. [6] | RGB-depth | 61.5 | 42.4 | 30.5 | | Method in Ref. [22] | RGB-depth | 65.6 | 42.2 | 27.8 | | MSF-CRF | RGB-depth | 66.9 | 44.2 | 30.2 |
|
查看原文
表 340个类别的类别精度对比表
Table3. Comparison of classification accuracy of 40 categories
| Dataset | Wall | Floor | Cabinet | Bed | Chair | Sofa | Table | Door |
|---|
| FuseNet | 89.2 | 95.7 | 67.9 | 75.7 | 74.6 | 71.0 | 49.3 | 34.8 | | MSF-CRF | 91.8 | 96.5 | 71.0 | 73.7 | 73.5 | 83.1 | 49.5 | 27.1 | | Dataset | Window | Bookshelf | Picture | Counter | Blinds | Desk | Shelf | Curtain | | FuseNet | 52.9 | 48.0 | 68.1 | 56.4 | 67.2 | 15.1 | 12.6 | 56.5 | | MSF-CRF | 53.8 | 60.7 | 66.6 | 63.5 | 45.6 | 26.0 | 17.3 | 58.5 | | Dataset | Dresser | Pillow | Mirror | Floormat | Clothes | Ceiling | Books | Fridge | | FuseNet | 28.4 | 44.3 | 30.7 | 38.8 | 22.9 | 75.5 | 21.2 | 11.9 | | MSF-CRF | 45.3 | 49.3 | 54.9 | 19.0 | 15.9 | 69.2 | 10.7 | 21.0 | | Dataset | TV | Paper | Towel | Shower | Box | White board | Person | Nightstand | | FuseNet | 39.1 | 5.7 | 23.0 | 34.9 | 7 | 32.5 | 23.2 | 35.1 | | MSF-CRF | 50.8 | 4.3 | 29.6 | 30.6 | 3.3 | 24.3 | 49.4 | 54.0 | | Dataset | Toilet | Sink | Lamp | Bathtub | Bag | Other struct | Other furniture | Other prop | | FuseNet | 75.0 | 32.4 | 40.1 | 51.9 | 1.6 | 19.8 | 10.8 | 45.7 | | MSF-CRF | 78.7 | 32.9 | 40.2 | 50.1 | 1.0 | 9.3 | 18.7 | 46.8 |
|
查看原文
表 440个类别的IoU对比表
Table4. Comparison of IoU of 40 categories
| Dataset | Wall | Floor | Cabinet | Bed | Chair | Sofa | Table | Door |
|---|
| FuseNet | 59.5 | 70.8 | 44.7 | 59.3 | 41.2 | 47.5 | 31.8 | 19.6 | | MSF-CRF | 57.2 | 70.4 | 45.0 | 63.7 | 43.8 | 50.2 | 35.4 | 15.4 | | Dataset | Window | Bookshelf | Picture | Counter | Blinds | Desk | Shelf | Curtain | | FuseNet | 27.5 | 30.0 | 44.1 | 34.4 | 42.5 | 11.3 | 5.8 | 34.8 | | MSF-CRF | 32.7 | 30.8 | 48.0 | 38.5 | 36.3 | 17.0 | 6.1 | 43.1 | | Dataset | Dresser | Pillow | Mirror | Floormat | Clothes | Ceiling | Books | Fridge | | FuseNet | 23.7 | 29.6 | 24.3 | 29.5 | 8.5 | 42.3 | 14.8 | 8.9 | | MSF-CRF | 32.1 | 34.3 | 42.5 | 17.0 | 9.4 | 39.8 | 9.5 | 14.0 | | Dataset | TV | Paper | Towel | Shower | Box | White board | Person | Nightstand | | FuseNet | 31.5 | 3.8 | 18.5 | 20.3 | 4 | 22.4 | 14.8 | 26.6 | | MSF-CRF | 39.1 | 3.7 | 21.8 | 26.1 | 2.4 | 20.7 | 32.9 | 40.1 | | Dataset | Toilet | Sink | Lamp | Bathtub | Bag | Other struct | Other furniture | Other prop | | FuseNet | 49.1 | 24.3 | 28.8 | 41.1 | 1.1 | 11.1 | 7.9 | 21.9 | | MSF-CRF | 50.1 | 21.2 | 31.2 | 39.8 | 0.9 | 7.3 | 13.4 | 25.0 |
|
查看原文
董永峰, 杨雨訢, 王利琴. 基于多尺度特征提取和全连接条件随机场的图像语义分割方法[J]. 激光与光电子学进展, 2019, 56(13): 131007. Yongfeng Dong, Yuxin Yang, Liqin Wang. Image Semantic Segmentation Based on Multi-Scale Feature Extraction and Fully Connected Conditional Random Fields[J]. Laser & Optoelectronics Progress, 2019, 56(13): 131007.
PDF全文
