首页 > 论文 > 光学学报 > 38卷 > 6期(pp:620002--1)

一种自学习不确定度的神经网络架构

An Neural Network Framework of Self-Learning Uncertainty

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

在自动驾驶这类多传感器融合任务中,每个子任务的不确定度对数据融合的策略和结果都有着至关重要的影响,为使整体系统能够在多工况下稳定运行,必须要求计算模型以较低的不确定度运行。现有方法仅能在神经网络预测过程中求得不确定度,很少有方法能够通过自学习的方式降低模型的不确定度。为解决上述问题,提出了不确定度学习层和不确定度损失项等概念,设计了一种能够通过自学习的方式降低不确定度的神经网络架构(ULNN),从而增强神经网络模型预测的稳健性。在CIFAR-10和CIFAR-100数据集上的实验表明,ULNN能够有效地降低模型不确定度,在两个数据集上分别降低了26倍和12倍的不确定度。进一步在CamVid数据集上的语义分割实验中证明了ULNN的通用性。

Abstract

In the multi-sensor fusion tasks of automatic drive, the strategy and the results of the data fusion are greatly influenced by the uncertainty of each subtask. To keep the whole system run steadily in multiple circumstances, the calculation model must operate with low uncertainty. The existing methods can only obtain uncertainty in the neural network prediction process, and few methods can reduce the uncertainty of the model in a self-learning method. To address the above problems, the concepts of uncertainty learning layer and uncertainty loss term are proposed, and a neural network architecture (ULNN) which can reduce uncertainty by self-learning method is designed to enhance the robustness of neural network model prediction. Experiments on CIFAR-10 and CIFAR-100 datasets show that ULNN can effectively reduce the model uncertainty and obtain 26 and 12 times lower uncertainty on the two data sets respectively. The universality of ULNN is proved by the experimental results of semantic segmentation on CamVid dataset.

Newport宣传-MKS新实验室计划
补充资料

中图分类号:TP183

DOI:10.3788/aos201838.0620002

所属栏目:光计算

基金项目:国家自然科学基金重点项目(61632081)

收稿日期:2017-12-29

修改稿日期:2018-02-07

网络出版日期:--

作者单位    点击查看

孙汉卿:天津大学电气自动化与信息工程学院, 天津 300072
庞彦伟:天津大学电气自动化与信息工程学院, 天津 300072

联系人作者:联系作者

备注:孙汉卿(1993-),男,硕士研究生,主要从事模式识别和自动驾驶方面的研究。E-mail: HQSun@tju.edu.cn

【1】Matti D, Ekenel H K, Thiran J P. Combining LiDAR space clustering and convolutional neural networks for pedestrian detection[C]. Lecce: IEEE International Conference on Advanced Video and Signal Based Surveillance, 2017: 1-6.

【2】Liu C H, Li Z, Xu C, et al. BRDF model for commonly used materials of space targets based on deep neural network[J]. Acta Optica Sinica, 2017, 37(11): 1129001.
刘程浩, 李智, 徐灿, 等. 基于深度神经网络的空间目标常用材质BRDF模型[J]. 光学学报, 2017, 37(11): 1129001.

【3】Lu Y S, Li Y X, Liu B, et al. Hyperspectral data haze monitoring based on deep residual network[J]. Acta Optica Sinica, 2017, 37(11): 1128001.
陆永帅, 李元祥, 刘波, 等. 基于深度残差网络的高光谱遥感数据霾监测[J]. 光学学报, 2017, 37(11): 1128001.

【4】Li S M, Lei G Q, Fan R. Depth map super-resolution reconstruction based on deep convolutional neural networks[J]. Acta Optica Sinica, 2017, 37(12): 1210002.
李素梅, 雷国庆, 范如. 基于卷积神经网络的深度图超分辨率重建[J]. 光学学报, 2017, 37(12): 1210002.

【5】Xiao J S, Liu E Y, Zhu L, et al. Improved image super-resolution algorithm based on convolutional neural network[J]. Acta Optica Sinica, 2017, 37(03): 0318011.
肖进胜, 刘恩雨, 朱力, 等. 改进的基于卷积神经网络的图像超分辨率算法[J]. 光学学报, 2017, 37(03): 0318011.

【6】Pang Y, Sun M, Jiang X, et al. Convolution in convolution for network in network[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 47(1): 1-11.

【7】Cao J, Pang Y, Li X, et al. Randomly translational activation inspired by the input distributions of ReLU[J]. Neurocomputing, 2017.

【8】Hou C C, He Y Q, Jiang X H, et al. A deep convolutional neural network based on two-stream convolutional unit[J]. Laser & Optoelectronics Progress, 2018, 55(02): 021005.
侯聪聪, 何宇清, 姜晓恒, 等. 基于二分支卷积单元的深度卷积神经网络[J]. 激光与光电子学进展, 2018, 55(02): 021005.

【9】Cao J, Pang Y, Li X. Learning multilayer channel features for pedestrian detection[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3210-3220.

【10】Morchhale S, Pauca V P, Plemmons R J, et al. Classification of pixel-level fused hyperspectral and lidar data using deep convolutional neural networks[C]. Los Angeles: Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing, 2016: 1-5.

【11】Zhao G, Xiao X, Yuan J, et al. Fusion of 3D-LIDAR and camera data for scene parsing[J]. Journal of Visual Communication and Image Representation, 2014, 25(1): 165-183.

【12】Du X, Ang M H, Rus D. Car detection for autonomous vehicle: LIDAR and vision fusion approach through deep learning framework[C]. Vancouver: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017: 749-754.

【13】Dieterle T, Particke F, Patino-Studencki L, et al. Sensor data fusion of LIDAR with stereo RGB-D camera for object tracking[A]. Glasgow: 2017 IEEE Sensors, 2017: 1-3.

【14】Gal Y, Ghahramani Z. Dropout as a Bayesian approximation: representing model uncertainty in deep learning[C]. New York: Proceedings of the 33rd International Conference on Machine Learning, 2016, 48: 1050-1059.

【15】Krzywinski M, Altman N. Points of significance: importance of being uncertain[J]. Nature Methods, 2013, 10(10): 809-810.

【16】Ghahramani Z. Probabilistic machine learning and artificial intelligence[J]. Nature, 2015, 521(7553): 452.

【17】Teerapittayanon S, McDanel B, Kung H T. BranchyNet: fast inference via early exiting from deep neural networks[C]. Cancun: 23rd International Conference on Pattern Recognition, 2016: 2464-2469.

【18】Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014, 15: 1929-1958.

【19】Huang G, Liu Z, van der Maaten L, et al. Densely connected convolutional networks[C]. Honolulu: IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2261-2269.

【20】Kendall A, Badrinarayanan V, Cipolla R. Bayesian SegNet: model uncertainty in deep convolutional encoder-decoder architectures for scene understanding[J]. Computer Science, 2015: 1511.

【21】He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition[C]. Las Vegas: 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.

【22】Wan L, Zeiler M, Zhang S, et al. Regularization of neural networks using dropconnect[C]. Atlanta: Proceedings of the 30th International Conference on Machine Learning, 2013, 28: 1058-1066.

【23】Barrow E, Eastwood M, Jayne C. Selective dropout for deep neural networks[C]. Kyoto: Proceedings of the 23rd International Conference on Neural Information Processing, 2016: 519-528.

【24】Isobe S, Arai S. Deep convolutional encoder-decoder network with model uncertainty for semantic segmentation[C]. IEEE International Conference on Innovations in Intelligent Systems and Applications, 2017: 365-370.

【25】Goodfellow I, Warde-Farley D, Mirza M, et al. Maxout networks[C]. Atlanta: 2013 International Conference on Machine Learning, 2013, 28: 1319-1327.

【26】Chang J-R, Chen Y-S. Batch-normalized maxout network in network[EB/OL]. [2018-4-24]. https://arxiv.org/abs/1511.02583.

【27】Krizhevsky A, Hinton G E. Learning multiple layers of features from tiny images[R]. CiteSeerX, 2009.

【28】Nesterov Y E. A method for solving the convex programming problem with convergence rate O(1/k2)[J]. Soviet Math Dokl, 1983, 269(3): 543-547.

【29】Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for scene segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.

引用该论文

Sun Hanqing,Pang Yanwei. An Neural Network Framework of Self-Learning Uncertainty[J]. Acta Optica Sinica, 2018, 38(6): 0620002

孙汉卿,庞彦伟. 一种自学习不确定度的神经网络架构[J]. 光学学报, 2018, 38(6): 0620002

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF