基于ResNet和RF-Net的遥感影像匹配

针对Receptive Fields Network(RF-Net)中存在网络较浅、缺乏深层语义信息的问题, 提出了一种基于Residual Network(ResNet)和RF-Net的改进网络用于遥感影像匹配。首先, 通过对真实遥感影像进行裁剪、光照变换和仿射变换处理, 得到图像对并计算同一序列中不同图像间的单应性矩阵, 构建了一个遥感影像数据集。然后, 提出了一种双通道的网络结构用于关键点检测, 该双通道网络由Receptive Fields Detection(RF-Det)和ResNet构成, 前者提取含有细节信息的浅层特征图, 后者提取含有语义信息的深层特征图。此外, 采用特征描述子提取网络L2-Net, 得到128维特征向量用以描述关键点。最后, 分别采用最近邻、带阈值的最近邻和最近邻距离比的策略对特征描述子进行匹配。实验结果表明, 该网络在仅含光照变换、仅含仿射变换和同时包含这两种变换的遥感影像数据集上的匹配得分, 比RF-Net分别提高了0.002, 0.117, 0104, 在关键点检测和匹配精度方面具有更优的性能。

Abstract

Aiming at the problems of shallow network and lack of high-level semantic information in Receptive Fields Network(RF-Net), a new network based on Residual Network(ResNet) and RF-Net is proposed for the matching of remote sensing images. Firstly, the new remote sensing image datasets are setup, which consist of images and homography matrices. The images are obtained by cropping, illumination changing and affine transforming of the original remote sensing images. The matrices are obtained by calculating the homography between different images of one sequence. Secondly, a dual-channel network structure is proposed for the keypoints detection. Specifically, the dual-channel network composed by Receptive Fields Detection(RF-Det) and ResNet, respectively extracts receptive feature maps with detail information, and the deep layer feature maps with semantic information. The L2-Net is introduced at descriptor extraction module to obtain the 128-dimension feature vectors as description of keypoints. Lastly, the strategies of the nearest neighbor, nearest neighbor with a threshold and nearest neighbor distance ration are used to match the descriptors. Experiments on remote sensing images show that the matching scores of the proposed network are 0.002, 0.117, 0104, higher than RF-Net, when testing on different datasets. It outperforms on keypoints detection and matching accuracy.

参考文献

[1] ZITOV B, FLUSSER J. Image registration methods: a survey[J]. Image and Vision Computing, 2003, 21(11): 977-1000.

[2] 余先川, 吕中华, 胡丹．遥感图像配准技术综述[J]．光学精密工程, 2013, 21(11): 2960-2972．

YU X C, LV Z H, HU D. Review of remote sensing image registration techniques[J]. Optics and Precision Engineering, 2013, 21(11): 2960-2972. (in Chinese)

[3] 贾迪, 朱宁丹, 杨宁华, 等．图像匹配方法研究综述[J]．中国图象图形学报, 2019, 24(5): 677-699．

JIA D, ZHU N D, YANG N H, et al. Image matching methods[J]. Journal of Image and Graphics, 2019, 24(5): 677-699. (in Chinese)

[4] ONO Y, TRULLS E, FUA P, et al. LF-Net: learning local features from images[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montreal, Canada: ACM, 2018: 6237-6247.

[5] YI K M, TRULLS E, LEPETIT V, et al. LIFT: Learned invariant feature transform[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, the Netherlands: Springer, 2016: 467-483.

[6] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.

[7] DETONE D, MALISIEWICZ T, RABINOVICH A. SuperPoint: self-supervised interest point detection and description[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City, UT, USA: IEEE, 2018.

[8] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of 3rd International Conference on Learning Representations.San Diego, CA, USA: ICLR, 2015: 1-13.

[9] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA, USA: IEEE, 2005: 539-546.

[10] SHEN X L, WANG C, LI X, et al. RF-Net: an end-to-end image matching network based on receptive field[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019: 8124-8132.

[11] MISHCHUK A, MISHKIN D, RADENOVIC F, et al. Working hard to know your neighbor's margins: Local descriptor learning loss[C]//Proceedings of the 31st Conference on Neural Information Processing Systems. Long Beach, CA, USA: NeurIPS, 2017: 4826-4837.

[12] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, Nevada, USA: IEEE, 2016: 770-778.

[13] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning. Lille, France: ACM, 2015: 448-456.

[14] NAIR V, HINTON G E. Rectified linear units improve restricted boltzmann machines[C]//Proceedings of the 27th International Conference on Machine Learning. Haifa, Israel: ACM, 2010: 807-814.

[15] TIAN Y R, FAN B, WU F C. L2-Net: Deep learning of discriminative patch descriptor in euclidean space[C]//Proceedings of the 30nd IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, Hawaii, USA: IEEE, 2017: 6128-6136.

[16] LIN T Y, DOLLR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of the 30nd IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, Hawaii, USA: IEEE, 2017: 936-944.

[17] MIKOLAJCZYK K, SCHMID C. A performance evaluation of local descriptors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(10): 1615-1630.

[18] KINGMA D, BA J. Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations.San Diego, CA, USA: ICLR, 2015: 1-15.

廖明哲, 吴谨, 朱磊. 基于ResNet和RF-Net的遥感影像匹配[J]. 液晶与显示, 2020, 35(9): 972. LIAO Ming-zhe, WU Jin, ZHU Lei. Remote sensing images matching based on ResNet and RF-Net[J]. Chinese Journal of Liquid Crystals and Displays, 2020, 35(9): 972.

基于ResNet和RF-Net的遥感影像匹配

关于本站 Cookie 的使用提示

全站搜索

基于ResNet和RF-Net的遥感影像匹配

相关论文

相关资讯

关于本站 Cookie 的使用提示

全站搜索