提出了一种基于深度学习的红外与可见光决策级融合检测方法。首先, 提出了一种介于深度学习模型之间的参数传递模型, 进而从基于深度学习的可见光物体检测模型上抽取了用于红外物体检测的预训练模型, 并在课题组实地采集的红外数据集上进行fine-tuning, 从而得到基于深度学习的红外物体检测模型。在此基础上, 提出了一种基于深度学习的红外与可见光决策级融合检测模型, 并对模型设计、图像配准、决策级融合过程进行了详细地阐述。最后, 进行了白天和傍晚条件下基于深度学习的单波段检测实验和双波段融合检测实验。定性分析上, 由于波段之间的信息互补性, 相比于单波段物体检测, 双波段融合物体检测在检测结果上具有更高的置信度和更精确的物体框; 定量分析上, 白天时, 双波段融合检测的mAP为86.0%, 相比于红外检测和可见光检测分别提高了9.9%和5.3%; 傍晚时, 双波段融合检测的mAP为89.4%, 相比于红外检测和可见光检测分别提高了3.1%和14.4%。实验结果表明: 基于深度学习的双波段融合检测方法相比于单波段检测方法具有更好的检测性能和更强的鲁棒性, 同时也验证了所提出方法的有效性。

A fusion detection methodology for infrared and visible spectra was presented based on deep learning. First, a parameter transfer model for deep learning models was proposed. Then a pretraining model for infrared object detection was extracted from a visible object detection model based on deep learning and was fine-tuned on a collected infrared image dataset to obtain an infrared object detection model based on deep learning. On this basis, a decision-level fusion model for infrared and visible detection based on deep learning was established, and the model design, image registration and decision-level fusion processes were discussed in detail. Finally, an experiment comparing single-band detection and dual-band fusion detection during the daytime and nighttime was presented. Qualitatively, compared with the results of single-band detection, the confidences and bounding boxes achieved through dual-band fusion detection are superior, owing to the utility of their complementary information. Quantitatively, in the daytime, the mAP of dual-band fusion detection is 86.0% and is higher than those of infrared detection and visible detection by 9.9% and 5.3%, respectively; at nighttime, the mAP of dual-band fusion detection is 89.4% and is higher by 3.1% and 14.4%, respectively. The experimental results show that the dual-band fusion detection method proposed in this paper shows better performance and stronger robustness than the single-band object detection methods do, thus verifying the effectiveness of the proposed method.