首页 > 论文 > 激光与光电子学进展 > 56卷 > 11期(pp:111001--1)


Multimodal Remote Sensing Image Classification with Small Sample Size Based on High-Level Feature Fusion

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文




The training sample size for some objects on the ground is quite small when applying a deep learning model to study the classification of remote sensing images. Meanwhile, diversified remote sensing image acquisition methods generate numerous multimodal remote sensing images with different spatial resolutions. Fusing these multi-modal remote sensing images to remedy the small sample size defect and achieve a highly precise classification of remote sensing images is an urgent problem to be solved. To this end, the present study proposes a fusion method for image classification based on the correlation of two spatial resolutions. A deep learning network is utilized to extract the high-level features of the remote sensing images in two spatial resolutions. Two types of high-level features are integrated via the proposed fusion strategy and further used as the input to train the whole network model. The experimental results demonstrate that the proposed fusion algorithm can achieve high classification accuracy. Further, because different fusion rules have different classification accuracies, a suitable selection can improve the classification accuracy.

广告组1 - 空间光调制器+DMD




基金项目:国家自然科学基金、海洋大数据分析预报技术研发预报技术研发基金 、上海市科委部分地方院校能力建设项目;




作者单位    点击查看

贺琪:上海海洋大学信息学院, 上海 201306
李瑶:上海海洋大学信息学院, 上海 201306
宋巍:上海海洋大学信息学院, 上海 201306
黄冬梅:上海海洋大学信息学院, 上海 201306上海电力大学, 上海 200090
何盛琪:上海海洋大学信息学院, 上海 201306
杜艳玲:上海海洋大学信息学院, 上海 201306


备注:国家自然科学基金、海洋大数据分析预报技术研发预报技术研发基金 、上海市科委部分地方院校能力建设项目;

【1】Tanha J, van Someren M and Afsarmanesh H. Semi-supervised self-training for decision tree classifiers. International Journal of Machine Learning and Cybernetics. 8(1), 355-370(2017).

【2】Guo Y, Jia X, Remote Sensing and Spatial Information Sciences. IV-. 4/W2, 161-165(2017).

【3】Xia M, Cao G, Wang G Y et al. Remote sensing image classification based on deep learning and conditional random fields. Journal of Image and Graphics. 22(9), 1289-1301(2017).
夏梦, 曹国, 汪光亚 等. 结合深度学习与条件随机场的遥感图像分类. 中国图象图形学报. 22(9), 1289-1301(2017).

【4】Cao X Y, Xu L, Meng D Y et al. Integration of 3-dimensional discrete wavelet transform and Markov random field for hyperspectral image classification. Neurocomputing. 226, 90-100(2017).

【5】Krizhevsky A, Sutskever I and Hinton G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM. 60(6), 84-90(2017).

【6】Liu Y T, Li Z Q and Yang X L. Application of improved convolution neural network in remote sensing image classification. Journal of Computer Applications. 38(4), 949-954(2018).
刘雨桐, 李志清, 杨晓玲. 改进卷积神经网络在遥感图像分类中的应用. 计算机应用. 38(4), 949-954(2018).

【7】Yang H, Li L Q, Yang H H et al. Method of urban management cases' image classification based on convolutional neural network. Computer Engineering and Applications. 54(10), 242-248, 266(2018).
杨浩, 李灵巧, 杨辉华 等. 基于卷积神经网络的城管案件图像分类方法. 计算机工程与应用. 54(10), 242-248, 266(2018).

【8】Fu G Y, Gu H Y and Wang H Q. Spectral and spatial classification of hyperspectral images based on convolutional neural networks. Science Technology and Engineering. 17(21), 268-274(2017).
付光远, 辜弘炀, 汪洪桥. 基于卷积神经网络的高光谱图像谱-空联合分类. 科学技术与工程. 17(21), 268-274(2017).

【9】LeCun Y, Bottou L, Bengio Y et al. . Gradient-based learning applied to document recognition. Proceedings of the IEEE. 86(11), 2278-2324(1998).

【10】Simonyan K. -04-10)[2018-10-20]. https:∥arxiv. org/abs/1409, (2015).

【11】Eitel A, Springenberg J T, Spinello L et al. Multimodal deep learning for robust RGB-D object recognition. [C]∥2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), September 28-October 2, 2015, Hamburg, Germany. New York: IEEE. 15666832, (2015).

【12】Xu Y, Du J, Dai L R et al. Cross-language transfer learning for deep neural network based speech enhancement. [C]∥The 9th International Symposium on Chinese Spoken Language Processing, September 12-14, 2014, Singapore, Singapore. New York: IEEE. 14700579, (2014).

【13】Han D M, Liu Q G and Fan W G. A new image classification method using CNN transfer learning and web data augmentation. Expert Systems With Applications. 95, 43-56(2018).

【14】Ahmed A, Yu K, Xu W et al. Training hierarchical feed-forward visual recognition models using transfer learning from pseudo-tasks. ∥Forsyth D, Torr P, Zisserman A. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer. 69-82(2008).

【15】Camps-Valls G, Gomez-Chova L, Munoz-Mari J et al. Kernel-based framework for multitemporal and multisource remote sensing data classification and change detection. IEEE Transactions on Geoscience and Remote Sensing、. 46(6), 1822-1835(2008).

【16】Ding X, Jiang Y, Huang Y et al. Pan-sharpening with a Bayesian nonparametric dictionary learning model. [C]∥Artificial Intelligence and Statistics. [S.l.]: [s.n.]. 176-184(2014).

【17】Hane C, Zach C, Cohen A et al. Joint 3D scene reconstruction and class segmentation. [C]∥2013 IEEE Conference on Computer Vision and Pattern Recognition, June 23-28, 2013, Portland, OR, USA. New York: IEEE. 13824292, (2013).

【18】Fang X, Wang G H, Yang H C et al. High resolution remote sensing image classification combining with mean-shift segmentation and fully convolution neural network. Laser & Optoelectronics Progress. 55(2), (2018).
方旭, 王光辉, 杨化超 等. 结合均值漂移分割与全卷积神经网络的高分辨遥感影像分类. 激光与光电子学进展. 55(2), (2018).

【19】Tu S Q, Xue Y J, Liang Y et al. Review on RGB-D image classification. Laser & Optoelectronics Progress. 53(6), (2016).
涂淑琴, 薛月菊, 梁云 等. RGB-D图像分类方法研究综述. 激光与光电子学进展. 53(6), (2016).

【20】He M Y, Cheng Y L, Liao X J et al. Building extraction algorithm by fusing spectral and geometrical features. Laser & Optoelectronics Progress. 55(4), (2018).
何曼芸, 程英蕾, 廖湘江 等. 融合光谱特征和几何特征的建筑物提取算法. 激光与光电子学进展. 55(4), (2018).

【21】Couprie C, Farabet C, Najman L et al. -03-14)[2018-10-20]. https:∥arxiv. org/abs/1301, (2013).

【22】Wang W, Yang X Y, Ooi B C et al. Effective deep learning-based multi-modal retrieval. The VLDB Journal. 25(1), 79-101(2016).

【23】Cheng G and Han J W. A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing. 117, 11-28(2016).

【24】Cheng G, Zhou P C and Han J W. Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing. 54(12), 7405-7415(2016).

【25】Zhou R T, Li Z P, Wu C et al. Buddy routing: a routing paradigm for NanoNets based on physical layer network coding. [C]∥2012 21st International Conference on Computer Communications and Networks (ICCCN), July 30-August 2, 2012, Munich, Germany. New York: IEEE. 12965366, (2012).


Qi He,Yao Li,Wei Song,Dongmei Huang,Shengqi He,Yanling Du. Multimodal Remote Sensing Image Classification with Small Sample Size Based on High-Level Feature Fusion[J]. Laser & Optoelectronics Progress, 2019, 56(11): 111001

贺琪,李瑶,宋巍,黄冬梅,何盛琪,杜艳玲. 小样本的多模态遥感影像高层特征融合分类[J]. 激光与光电子学进展, 2019, 56(11): 111001


【1】余帅,汪西莉. 基于多级通道注意力的遥感图像分割方法. 激光与光电子学进展, 2020, 57(4): 41012--1

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF