首页 > 论文 > 激光与光电子学进展 > 55卷 > 11期(pp:111010--1)

基于优化全卷积神经网络的手语语义识别

Sign Language Semantic Recognition Based on Optimized Fully Convolutional Neural Network

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

手语特征提取的传统算法仅仅依靠底层特征完成识别,难以获得高层语义特征,进而对手语理解产生分歧。针对这一问题,将图像语义分析的思维引入手语识别研究中,提出一种优化全卷积神经网络算法。采用全卷积神经网络提取手语图像的语义特征,并通过判别随机场进行语义标注做后期平滑处理,恢复像素间的细节信息,从而完成手语识别。实验结果表明,所提出的算法具有较强的稳健性,能有效学习到语义特征。与传统算法对比分析表明,此方法能精准的识别到手语,其平均识别率达97.41%。

Abstract

The traditional algorithms for the extraction of sign language features only rely on the low-level features to realize recognition, which makes it difficult to obtain the high-level semantic features and the misunderstanding of sign language is further induced. To solve this problem, the idea of image semantic analysis is introduced into the study of sign language recognition and then an optimized fully convolutional neural network algorithm is proposed. The fully convolutional neural network is used to extract the semantic features of sign language images and the discriminative random fields for semantic annotation is used for the post-smoothing to recover the detailed information among pixels and thus the sign language recognition is completed. The experimental results show that the proposed algorithm has strong robustness and can be used to obtain the semantic features effectively. Compared with the traditional algorithms, this method can be used to identify sign language accurately with an average recognition rate of 97.41%.

Newport宣传-MKS新实验室计划
补充资料

中图分类号:TP391.41

DOI:10.3788/lop55.111010

所属栏目:图像处理

基金项目:国家自然科学基金(61373112)、住房和城乡建设部科学技术项目计划(2016-R2-045)、陕西省自然科学基础研究资金(2014JM8348)

收稿日期:2018-05-03

修改稿日期:2018-06-01

网络出版日期:2018-06-08

作者单位    点击查看

王民:西安建筑科技大学信息与控制工程学院, 陕西 西安 710055
郝静:西安建筑科技大学信息与控制工程学院, 陕西 西安 710055
要趁红:西安建筑科技大学信息与控制工程学院, 陕西 西安 710055
史其琦:西安建筑科技大学信息与控制工程学院, 陕西 西安 710055

联系人作者:郝静(haoanjing12@163.com)

【1】Jiang Y J. Research on isolated sign language recognition system based on Kinect[D]. Hefei: University of Science and Technology of China, 2015: 1-8.
江勇军. 基于Kinect的孤立词手语识别系统研究[D]. 合肥: 中国科学技术大学, 2015: 1-8.

【2】Zhang S J. Unmarked gesture recognition based on vision[M]. Jilin: Jilin University Press, 2016: 1-25.
张生军. 基于视觉的无标记手势识别[M]. 吉林: 吉林大学出版社, 2016: 1-25.

【3】Sun C, Zhang T, Bao B K, et al. Latent support vector machine for sign language recognition with Kinect[C]∥IEEE International Conference on Image Processing, 2014: 4190-4194.

【4】Li M. Algorithm of gesture recognition based on image analysis[D]. Beijing: North China University of Technology, 2015: 15-28.
李萌. 基于图像分析的手语识别技术研究[D]. 北京: 北方工业大学, 2015: 15-28.

【5】Huong T N T, Huu T V, Xuan T L, et al. Static hand gesture recognition for vietnamese sign language (VSL) using principle components analysis[C]∥International Conference on Communications, Management and Telecommunications, 2016: 138-141.

【6】Liu X J, Zhang Y. Gesture recognition based on multi-feature and SVM classification[J]. Computer Engineering and Design, 2017, 38(4): 953-958.
刘小建, 张元. 基于多特征提取和SVM分类的手势识别[J]. 计算机工程与设计, 2017, 38(4): 953-958.

【7】Bao Z Q, Lü C G. Real-time gesture recognition based on Kinect[J]. Laser & Optoelectronics Progress, 2018, 55(3): 031008.
鲍志强, 吕辰刚. 基于Kinect的实时手势识别[J]. 激光与光电子学进展, 2018, 55(3): 031008.

【8】Wang L, Liu H, Wang B, et al. Gesture recognition method combining skin color models and convolution neural network[J]. Computer Engineering and Applications, 2017, 53(6): 209-214.
王龙, 刘辉, 王彬, 等. 结合肤色模型和卷积神经网络的手势识别方法[J]. 计算机工程与应用, 2017, 53(6): 209-214.

【9】Sang N, Ni Z H. Gesture recognition based on R-FCN in complex scenes[J].Journal of Huazhong University of Science and Technology(Nature Science Edition), 2017, 45(10): 54-58.
桑农, 倪子涵. 复杂场景下基于R-FCN的手势识别[J]. 华中科技大学学报(自然科学版), 2017, 45(10): 54-58.

【10】Xiong Z Y, Zhang G F, Wang J Q. Multi-scale feature extract for image sematic segmentation[J]. Journal of South-Central University for Nationalities(Natural Science Edition), 2017, 36(3): 118-124.
熊志勇, 张国丰, 王江晴. 基于多尺度特征提取的图像语义分割[J]. 中南民族大学学报(自然科学版), 2017, 36(3): 118-124.

【11】Jiao L C, Zhao J, Yang S Y, et al. Deep learning, optimization and identification[M].Beijing: Tsinghua University Press, 2017: 77-80.
焦李成, 赵进, 杨淑媛, 等. 深度学习, 优化与识别[M]. 北京: 清华大学出版社, 2017: 77-80.

【12】Maggiori E, Tarabalka Y, Charpiat G, et al. Fully convolutional neural networks for remote sensing image classification[C]∥Geoscience and Remote Sensing Symposium, 2016: 5071-5074.

【13】Fang X, Wang G H, Yang H C, et al. High resolution remote sensing image classification combining with mean-shift segmentation and fully convolution neural network[J]. Laser & Optoelectronics Progress, 2018, 55(2): 022802.
方旭, 王光辉, 杨化超,等. 结合均值漂移分割与全卷积神经网络的高分辨遥感影像分类[J]. 激光与光电子学进展, 2018, 55(2): 022802.

【14】Milletari F, Navab N, Ahmadi S A. V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]∥International Conference on 3D Vision, 2016: 565-571.

【15】Cao X W, Bo H. Study on gesture recognition based on CNN[J]. Microcomputer & its Applications, 2016, 35(9): 55-57.
操小文, 薄华. 基于卷积神经网络的手势识别研究[J]. 微型机与应用, 2016, 35(9): 55-57.

【16】Kang K, Wang X. Fully Convolutional Neural Networks for Crowd Segmentation[J]. Computer Science, 2014, 49(1): 25-30.

【17】Li S M, Lei G Q, Fan R. Depth map super-resolution reconstruction based on convolutional neural networks[J]. Acta Optica Sinica, 2017, 37(12): 1210002.
李素梅, 雷国庆, 范如. 基于卷积神经网络的深度图超分辨率重建[J]. 光学学报, 2017, 37(12): 1210002.

【18】Yi M, Sui L C. Aerial image semantic classification method based on improved full convolution neural network[J]. Computer Engineering, 2017, 43(10): 216-221.
易盟, 隋立春. 基于改进全卷积神经网络的航拍图像语义分类方法[J]. 计算机工程, 2017, 43(10): 216-221.

【19】Luo B, Gao W, Tang J, et al. Learning corner regression-based fully convolutional neural network for license plate localization in complex scene[J]. Journal of Data Acquisition & Processing, 2016, 31(1): 65-72.
罗斌, 郜伟, 汤进, 等. 复杂环境下基于角点回归的全卷积神经网络的车牌定位[J]. 数据采集与处理, 2016, 31(1): 65-72.

【20】Pan C, Wang X J. Summary of algorithms for detecting man-made information in natural background[J]. Optical Technique, 2008, 34(6): 881-885.
潘聪, 王向军. 自然背景中人造信息的检测算法综述[J]. 光学技术, 2008, 34(6): 881-885.

【21】Ding H, Zhang X F. Connected handwritten and printed text discrimination in uneven lighted images[J]. Computer Engineering and Design, 2012, 33(12): 4634-4638.
丁红, 张晓峰. 非均匀光照图像中粘连手写体和印刷体的辨别[J]. 计算机工程与设计, 2012, 33(12): 4634-4638.

【22】Zeng W J, Qin F, Si Y. Sign language recognition system based on artificial neural network[J]. China Computer & Communication, 2017(11): 143-144.
曾维佳, 秦放, 思雨. 基于人工神经网络的手语识别系统[J]. 信息与电脑, 2017(11): 143-144.

【23】Xin P, Xu Y L, Tang H, et al. Fast airplane detection based on multi-layer feature fusion of fully convolutional networks[J]. Acta Optica Sinica, 2018, 38(3): 031500.
辛鹏, 许悦雷, 唐红, 等. 全卷积网络多层特征融合的飞机快速检测[J]. 光学学报, 2018, 38(3): 031500.

引用该论文

Wang Min,Hao Jing,Yao Chenhong,Shi Qiqi. Sign Language Semantic Recognition Based on Optimized Fully Convolutional Neural Network[J]. Laser & Optoelectronics Progress, 2018, 55(11): 111010

王民,郝静,要趁红,史其琦. 基于优化全卷积神经网络的手语语义识别[J]. 激光与光电子学进展, 2018, 55(11): 111010

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF