首页 > 论文 > 激光与光电子学进展 > 57卷 > 20期(pp:201012--1)

基于YOLOv3和视觉SLAM的语义地图构建

Semantic Mapping Based on YOLOv3 and Visual SLAM

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

以相机作为输入的视觉同时定位与建图(SLAM)系统在地图构建过程中虽然可以保留点云的空间几何信息,但是并没有完全利用环境中物体的语义信息。针对这个问题,对当前主流视觉SLAM系统和基于Faster R-CNN、YOLO等神经网络结构的目标检测算法进行研究。并提出一种有效的点云分割方法,该方法引入支撑平面以提升分割结果的鲁棒性。最后在ORB-SLAM系统的基础上,结合YOLOv3算法进行环境场景的物体检测并保证构建的点云地图具有语义信息。实验结果表明,所提方法可以构建几何信息复杂的语义地图,从而可应用于无人车或机器人的导航工作中。

Abstract

Visual simultaneous localization and mapping (SLAM) systems that use cameras as input can retain the spatial geometry information of a point cloud in the map construction process. However, such systems do not fully utilize the semantic information of objects in the environment. To address this problem, the mainstream visual SLAM system and object detection algorithms based on neural network structures, such as Faster R-CNN and YOLO, are investigated. Moreover, an effective point cloud segmentation method that adds supporting planes to improve the robustness of the segmentation results is considered. Finally, the YOLOv3 algorithm is combined with ORB-SLAM system to detect objects in the environment scene and ensures that the constructed point cloud map has semantic information. The experimental results demonstrate that the proposed method constructs a semantic map with complex geometric information that can be applied to the navigation of unmanned vehicles or mobile robots.

广告组1 - 空间光调制器+DMD
补充资料

中图分类号:TP391

DOI:10.3788/LOP57.201012

所属栏目:图像处理

基金项目:国家重点研发计划、新能源汽车科学与关键技术学科创新引智基地基金;

收稿日期:2019-12-24

修改稿日期:2020-02-25

网络出版日期:2020-10-01

作者单位    点击查看

邹斌:武汉理工大学现代汽车零部件技术湖北省重点实验室, 湖北 武汉 430070汽车零部件技术湖北省协同创新中心, 湖北 武汉 430070
林思阳:武汉理工大学现代汽车零部件技术湖北省重点实验室, 湖北 武汉 430070
尹智帅:武汉理工大学现代汽车零部件技术湖北省重点实验室, 湖北 武汉 430070汽车零部件技术湖北省协同创新中心, 湖北 武汉 430070

联系人作者:林思阳(xyz5016@whut.edu.cn)

备注:国家重点研发计划、新能源汽车科学与关键技术学科创新引智基地基金;

【1】Whelan T. Salas-Moreno R F, Glocker B, et al. ElasticFusion: real-time dense SLAM and light source estimation [J]. The International Journal of Robotics Research. 2016, 35(14): 1697-1716.

【2】McCormac J, Handa A, Davison A, et al. -09-28)[2019-11-10] . https:∥arxiv. 2016, org/abs/1609: 05130.

【3】Girshick R, Donahue J, Darrell T, et al. Region-based convolutional networks for accurate object detection and segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2016, 38(1): 142-158.Girshick R, Donahue J, Darrell T, et al. Region-based convolutional networks for accurate object detection and segmentation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2016, 38(1): 142-158.

【4】Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017, 39(6): 1137-1149.

【5】He K M, Gkioxari G, Dollar P, et al. Mask R-CNN [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2020, 42(2): 386-397.

【6】Stein S C, Schoeler M, Papon J, et al. Object partitioning using local convexity . [C]∥2014 IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2014, Columbus, OH, USA. New York: IEEE. 2014, 304-311.

【7】Charles R Q, Su H, Mo K C, et al. PointNet: deep learning on point sets for 3D classification and segmentation . [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE. 2017, 77-85.

【8】Mur-Artal R, Tardós J D. ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras [J]. IEEE Transactions on Robotics. 2017, 33(5): 1255-1262.

【9】Oron S, Bar-Hille A, Avidan S. Extended Lucas-Kanade tracking [M]. ∥Fleet D, Pajdla T, Schiele B, et al. Computer vision - ECCV 2014. Lecture notes in computer science. 2014, 8693: 142-156.

【10】Feature extraction and image processing[M]. Yang G B: Li S Y, Transl. 3rd ed. Beijing: Electronic Industry Press, 2010, 127-133.
Nixon M S, Aguado A S, Nixon M S, Aguado A S. 特征提取与图像处理[M]. 杨高波: 李实英, 译. 3版. 北京: 电子工业出版社, 2010, 127-133.

【11】Liu W, Anguelov D, Erhan D, et al. SSD: single shot MultiBox detector [M]. ∥Leibe B, Matas J, Sebe N, et al. Computer vision - ECCV 2016. Lecture notes in computer science. 2016, 9905: 21-37.

【12】Guo J X, Liu L B, Xu F, et al. Airport scene aircraft detection method based on YOLOv3 [J]. Laser & Optoelectronics Progress. 2019, 56(19): 191003.
郭进祥, 刘立波, 徐峰, 等. 基于YOLOv3的机场场面飞机检测方法 [J]. 激光与光电子学进展. 2019, 56(19): 191003.

【13】Redmon J. -04-08)[2019-11-10] . https:∥arxiv. 2018, org/abs/1804: 02767.

【14】Li Y, Tong G F, Yang J C, et al. 3D point cloud scene data acquisition and its key technologies for scene understanding [J]. Laser & Optoelectronics Progress. 2019, 56(4): 040002.
李勇, 佟国峰, 杨景超, 等. 三维点云场景数据获取及其场景理解关键技术综述 [J]. 激光与光电子学进展. 2019, 56(4): 040002.

【15】Papon J, Abramov A, Schoeler M, et al. Voxel cloud connectivity segmentation - supervoxels for point clouds . [C]∥2013 IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2013, Portland, OR, USA. New York: IEEE. 2013, 2027-2034.

【16】-05-26)[2019-11-10] . http:∥citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1. 2005, 217: 3313.

【17】Boykov Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2001, 23(11): 1222-1239.

【18】Sturm J, Engelhard N, Endres F, et al. A benchmark for the evaluation of RGB-D SLAM systems . [C]∥2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 7-12, 2012, Vilamoura, Portugal. New York: IEEE. 2012, 573-580.

【19】Wu H, Chi J X, Tian G H. Instance recognition and semantic mapping based on visual SLAM [J]. Journal of Huazhong University of Science and Technology (Natural Science Edition). 2019, 47(9): 48-54.
吴皓, 迟金鑫, 田国会. 基于视觉SLAM的物体实例识别与语义地图构建 [J]. 华中科技大学学报(自然科学版). 2019, 47(9): 48-54.

引用该论文

Zou Bin,Lin Siyang,Yin Zhishuai. Semantic Mapping Based on YOLOv3 and Visual SLAM[J]. Laser & Optoelectronics Progress, 2020, 57(20): 201012

邹斌,林思阳,尹智帅. 基于YOLOv3和视觉SLAM的语义地图构建[J]. 激光与光电子学进展, 2020, 57(20): 201012

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF