Infrared-PV: 面向监控应用的红外目标检测数据集

陈旭; 吴蔚; 彭冬亮; 谷雨

红外技术, 2023, 45 (12): 1304, 网络出版: 2024-01-17

Infrared-PV: 面向监控应用的红外目标检测数据集

Infrared-PV: an Infrared Target Detection Dataset for Surveillance Application

论文大纲

陈旭 ¹吴蔚 ²彭冬亮 ¹谷雨 ¹

作者单位

¹ 杭州电子科技大学自动化学院, 浙江杭州 310018

² 中国电子科技集团第 28研究所, 江苏南京 210007

红外图像数据集监控应用深度学习基准测试 infrared image, dataset, surveillance application,

摘要

红外摄像机虽然能够全天候 24 h工作, 但是相比于可见光摄像机, 其获得的红外图像分辨率和信杂比低, 目标纹理信息缺乏, 因此足够的标记图像和进行模型优化设计对于提高基于深度学习的红外目标检测性能具有重要意义。为解决面向监控应用场景的红外目标检测数据集缺乏的问题, 首先采用红外摄像机采集了不同极性的红外图像, 基于自研图像标注软件实现了 VOC格式的图像标注任务, 构建了一个包含行人和车辆两类目标的红外图像数据集( Infrared-PV), 并对数据集中的目标特性进行了统计分析。然后采用主流的基于深度学习的目标检测模型进行了模型训练与测试, 定性和定量分析了 YOLO系列和 Faster R-CNN系列等模型对于该数据集的目标检测性能。构建的红外目标数据集共包含图像 2138张, 场景中目标包含白热、黑热和热力图 3种模式。当采用各模型进行目标检测性能测试时, Cascade R-CNN模型性能最优, mAP0.5值达到了 82.3%, YOLO v5系列模型能够兼顾实时性和检测精度的平衡, 推理速度达到 175.4帧/s的同时 mAP0.5值仅降低 2.7%。构建的红外目标检测数据集能够为基于深度学习的红外图像目标检测模型优化研究提供一定的数据支撑, 同时也可以用于目标的红外特性分析。

Abstract

Although infrared cameras can operate day and night under all-weather conditions compared with visible cameras, the infrared images obtained by them have low resolution and signal-to-clutter ratio, lack of texture information, so enough labeled images and optimization model design have great influence on improving infrared target detection performance based on deep learning. First, to solve the lack of an infrared target detection dataset used for surveillance applications, an infrared camera was used to capture images with multiple polarities, and an image annotation task that outputted the VOC format was performed using our developed annotation software. An infrared image dataset containing two types of targets, person and vehicle, was constructed and named infrared-PV. The characteristics of the targets in this dataset were statistically analyzed. Second, state-of-the-art target detection models based on deep learning were adopted to perform model training and testing. Target detection performances for this dataset were qualitatively and quantitatively analyzed for the YOLO and Faster R-CNN series detection models. The constructed infrared dataset contained 2138 images, and the targets in this dataset included three types of modes: white hot, black hot, and heat map. In the benchmark test using several models, Cascade R-CNN achieves the best performance, where mean average precision when intersection over union exceeding 0.5 1304 (mAP0.5) reaches 82.3%, and YOLOv5 model can achieve the tradeoff between real-time performance and detection performance, where inference time achieves 175.4 frames per second and mAP0.5 drops only 2.7%. The constructed infrared target detection dataset can provide data support for research on infrared image target detection model optimization and can also be used to analyze infrared target characteristics.

参考文献

[1] 陈钱, 隋修宝 . 红外图像处理理论与技术 [M].北京: 电子工业出版社 , 2018. CHEN Qian, SUI Xiubao. Infrared Image Processing Theory and Technology[M]. Beijing: Electronic Industry Press, 2018.

[2] 刘让, 王德江, 贾平, 等. 红外图像弱小目标探测技术综述 [J].激光与光电子学进展, 2016, 53(5): 050004. LIU Rang, WANG Dejiang, JIA Ping, et al. Overview of detection technology for small and dim targets in infrared images[J]. Progress in Laser and Optoelectronics, 2016, 53(5): 050004.

[3] 武斌. 红外弱小目标检测技术研究 [D].西安: 西安电子科技大学 . 2009. WU Bing. Research on Infrared Dim Target Detection Technology[D]. Xi'an: Xidian University, 2009.

[4] Rawat S S, Verma S K, Kumar Y. Review on recent development in infrared small target detection algorithms[J]. Procedia Computer Science, 2020, 167: 2496-2505.

[5] 李俊宏, 张萍, 王晓玮 , 等. 红外弱小目标检测算法综述 [J].中国图象图形学报, 2020, 25(9): 1739-1753. LI Junhong, ZHANG Ping, WANG Xiaowei, et al. Infrared small-target detection algorithms: a survey[J]. Journal of Image and Graphics, 2020, 25(9): 1739-1753.

[6] 谷雨, 刘俊, 沈宏海, 等. 基于改进多尺度分形特征的红外图像弱小目标检测[J].光学精密工程 , 2020, 28(6): 1375-1386. GU Yu, LIU Jun, SHEN Honghai, et al. Infrared image dim target detection based on improved multi-scale fractal features[J]. Optics and Precision Engineering, 2020, 28(6): 1375-1386.

[7] LIU L, OUYANG W, WANG X G, et al. Deep learning for generic object detection: a survey[J]. International Journal of Computer Vision, 2020, 128(2): 261-318.

[8] REN S Q, HE K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.

[9] LIU W, Anguelov D, Erhan D, et al. Ssd: single shot multibox detector[C]//European Conference on Computer Vision, 2016: 21-37.

[10] Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 779-788.

[11] 王文秀, 傅雨田 , 董峰, 等. 基于深度卷积神经网络的红外船只目标检测方法[J].光学学报 , 2018, 38(7): 0712006. WANG W X, FU Y T, DONG F, et al. Infrared ship target detection method based on deep convolutional neural network[J]. Acta Optics, 2018, 38(7): 0712006.

[12] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.

[13] 蒋志新. 基于深度学习的海上红外小目标检测方法研究 [D].大连: 大连海事大学, 2019. JIANG Z X. Research on the Detection Method of Marine Infrared Small Target Based on Deep Learning[D]. Dalian: Dalian Maritime University, 2019.

[14] 陈铁明, 付光远, 李诗怡, 等. 基于 YOLO v3的红外末制导典型目标检测[J].激光与光电子学进展 , 2019, 56(16): 155-162. CHEN T M, FU G Y, LI S Y, et al. Infrared terminal guidance typical target detection based on YOLOv3[J]. Progress in Laser and Optoelectronics, 2019, 56(16): 155-162.

[15] 赵琰, 刘荻, 赵凌君 . 基于 Yolo v3的复杂环境红外弱小目标检测[J].航空兵器, 2020, 26(6): 29-34. ZHAO Y, LIU D, ZHAO L J. Infrared small target detection in complex environment based on Yolo v3[J]. Aviation Weaponry, 2020, 26(6): 29-34.

[16] 吴双忱, 左峥嵘. 基于深度卷积神经网络的红外小目标检测 [J]. 红外与毫米波学报, 2019, 38(3): 371-380. WU S C, ZUO Z G. Infrared small target detection based on deep convolutional neural network[J]. Journal of Infrared and Millimeter Waves, 2019, 38(3): 371-380.

[17] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.

[18] 李慕锴, 张涛, 崔文楠 . 基于 Yolo v3的红外行人小目标检测技术研究[J].红外技术 , 2020, 42(2): 176-181. LI M K, ZHANG T, CUI W N. Research on infrared pedestrian small target detection technology based on Yolo v3[J]. Infrared Technology, 2020, 42(2): 176-181.

[19] Everingham M, Eslami S A, Van Gool L, et al. The pascal visual object classes challenge: a retrospective[J]. International Journal of Computer Vision, 2015, 111(1): 98-136.

[20] LIN T Y, Maire M, Belongie S, et al. Microsoft coco: common objects in context[C]//European Conference on Computer Vision, 2014: 1312 740-755.

[21] XIA G S, BAI X, DING J, et al. DOTA: a large-scale dataset for object detection in aerial images[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 3974-3983.

[22] LI K, WAN G, CHENG G, et al. Object detection in optical remote sensing images: a survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159: 296-307.

[23] ZHU H, CHEN X, DAI W, et al. Orientation robust object detection in aerial images using deep convolutional neural network[C]//2015 IEEE International Conference on Image Processing (ICIP), 2015: 3735-3739.

[24] TAN M, PANG R, LE Q V. Efficientdet: scalable and efficient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 10781-10790.

[25] Hwang S, Park J, Kim N, et al. Multispectral pedestrian detection: Benchmark dataset and baseline[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 1037-1045.

[26] Teledyne FLIR Systems. FLIR ADAS Dataset[DB/OL] [2023-11-27]. https://www.flir.com/oem/adas/adas-dataset-form/.

[27] Davis J W, Keck M A. A two-stage template approach to person detection in thermal imagery[C]//2005 Seventh IEEE Workshops on Applications of Computer Vision, 2005, 1: 364-369.

[28] CAI Z, Vasconcelos N. Cascade r-cnn: delving into high quality object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 6154-6162.

[29] PANG J, CHEN K, SHI J, et al. Libra r-cnn: Towards balanced learning for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 821-830.

[30] WU Y, CHEN Y, YUAN L, et al. Rethinking classification and localization for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 10186-10195.

[31] Redmon J, Farhadi A. Yolov3: an incremental improvement [EB/OL] [2018-04-08]. https://arxiv.org/pdf/1804.02767.pdf.

[32] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.

[33] LINT Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.

[34] Bochkovskiy A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of object detection[EB/OL] [2020-04-22]. https://arxiv.org/ pdf/2004.10934.pdf.

[35] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.

[36] WANG K, LIEW J H, ZOU Y, et al. PaNet: Few-shot image semantic segmentation with prototype alignment[C]//Proceedings of the IEEE International Conference on Computer Vision, 2019: 9197-9206.

[37] ZHENG Z, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]//Proceedings of 2020 Association for the Advancement of Artificial Intelligence, 2020: 12993-13000.

[38] TIAN Z, SHEN C, CHEN H, et al. FCOS: Fully convolutional one-stage object detection[C]//Proceedings of the 2019 IEEE International Conference on Computer Vision, 2019: 9627-9636.

[39] CHEN K, WANG J Q, PANG J M, et al. Mmdetection: open mmlab detection toolbox and benchmark[EB/OL][2019-06-17]. https:// arxiv.org /pdf/ 1906. 07155. pdf.

[40] ZHANG H, WU C R, ZHANG Z Y, et al. Resnest: Split-attention networks[EB/OL] [2020-04-19]. https://arxiv.org/pdf/2004.08955.pdf.

陈旭, 吴蔚, 彭冬亮, 谷雨. Infrared-PV: 面向监控应用的红外目标检测数据集[J]. 红外技术, 2023, 45(12): 1304. CHEN Xu, WU Wei, PENG Dongliang, GU Yu. Infrared-PV: an Infrared Target Detection Dataset for Surveillance Application[J]. Infrared Technology, 2023, 45(12): 1304.

Infrared-PV: 面向监控应用的红外目标检测数据集

关于本站 Cookie 的使用提示

全站搜索

Infrared-PV: 面向监控应用的红外目标检测数据集

相关论文

相关资讯

关于本站 Cookie 的使用提示

全站搜索