激光与光电子学进展, 2018, 55 (7): 071004, 网络出版: 2018-07-20   

视频序列中表情和姿态的双模态情感识别 下载: 1019次

Dual-Modal Emotion Recognition Based on Facial Expression and Body Posture in Video Sequences
作者单位
1 合肥工业大学计算机与信息学院情感计算与先进智能机器安徽省重点实验室, 安徽 合肥 230009
2 安徽国际商务职业学院信息服务系, 安徽 合肥 231131
3 德岛大学先端技术科学教育部, 日本 德岛 7708502
摘要
针对时空局部方向角模式应用到视频情感识别时,出现的特征稀疏、噪声敏感等问题,提出了一种新的特征提取算法——时空局部三值方向角模式(SLTOP)。考虑到表情和姿态特征的互补性,提出云加权决策融合的分类方法。对视频图像进行预处理,得到表情和姿态两种模态的序列;分别提取表情序列和姿态序列的SLTOP特征,并借鉴灰度矩阵思想解决特征直方图过于稀疏的问题;在决策分类阶段,引入云模型对表情和姿态两种模态进行云加权决策融合,实现双模态情感的最终识别。在FABO数据库中,表情和姿态单模态分别取得了92.21%和96.76%的平均识别率;与体积局部二值模式、三正交平面局部二值模式(LBP-TOP)、时空局部三值模式矩(TSLTPM)比较时,在表情模态上分别高约18.42%、22.01%、9.15%,而在姿态模态上分别高约26.59%、29.53%、1.98%。通过云加权融合得到平均识别率为97.54%,均高于其他实验得到的数据。所提出的SLTOP,对噪声和光照具有很好的稳健性。利用云模型的加权决策融合方法可以较好地发挥表情和姿态分类器的性能,得到较好的识别结果,与其他分类识别方法进行对比实验,结果同样表现出优越性。
Abstract
Aiming at the problems of feature sparseness and noise sensitivity when the temporal-spatial local direction angle mode is applied to the video emotion recognition, we propose a new feature extraction algorithm, the spatiotemporal local ternary orientation pattern (SLTOP). Considering the complementarity of facial expression and posture characteristics in recognition, a classification method based on the cloud weighted decision fusion is proposed. The video image is preprocessed to obtain the sequence of the two modes of facial expression and gesture. For reducing the sparseness of the feature histogram, we extract the SLTOP feature of the sequences of expression and posture, learning from the idea of gray level co-occurrence matrix. In the stage of decision fusion, the cloud model is introduced to implement the cloud weighted decision fusion for the two modes of expression and posture making to realize the final recognition of dual-modal emotion. The average recognition rate of the single modal of facial expression and body posture in the FABO database is 92.21% and 96.76%, respectively. And they are approximately 18.42%, 22.01% and 9.15% higher in expression, respectively, when compared with the volume local binary mode, local binary mode three orthogonal planes (LBP-TOP) and temporal-spatial local ternary pattern moment (TSLTPM). In the single-posture modal, they are 26.59%, 29.53%, 1.98% higher, respectively. The average recognition rate obtained by cloud-weighted fusion is 97.54%, which is higher than that of other experiments. The proposed SLTOP has good robustness to the noise and illumination. The weighted decision fusion method of cloud model is used to greatly express the performance of two classifiers with expression and posture. The superiority of the recognition results in this paper is shown comparing with other classification methods.

姜明星, 胡敏, 王晓华, 任福继, 王浩文. 视频序列中表情和姿态的双模态情感识别[J]. 激光与光电子学进展, 2018, 55(7): 071004. Jiang Mingxing, Hu Min, Wang Xiaohua, Ren Fuji, Wang Haowen. Dual-Modal Emotion Recognition Based on Facial Expression and Body Posture in Video Sequences[J]. Laser & Optoelectronics Progress, 2018, 55(7): 071004.

本文已被 3 篇论文引用
被引统计数据来源于中国光学期刊网
引用该论文: TXT   |   EndNote

相关论文

加载中...

关于本站 Cookie 的使用提示

中国光学期刊网使用基于 cookie 的技术来更好地为您提供各项服务,点击此处了解我们的隐私策略。 如您需继续使用本网站,请您授权我们使用本地 cookie 来保存部分信息。
全站搜索
您最值得信赖的光电行业旗舰网络服务平台!