首页 > 论文 > 光学学报 > 36卷 > 5期(pp:517001--1)

利用fNIRS研究情绪状态下的脑力负荷评估

Assessment of Mental Workload Influenced by Different Emotional State Using fNIRS

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

为研究采用功能性近红外光谱(fNIRS)技术评估不同情绪状态下操作者脑力负荷(MWL)的可行性,为开展以MWL评估为基础的操作者功能状态(OFS)评估提供技术支持,进行了多种情绪刺激下的图片n-back任务实验,包括负性、积极和中性三组情绪。采用任务绩效、主观量表和fNIRS生理测量等方法采集16名参试者的实验数据。任务绩效和主观状态评分均表明参试者MWL在外部因素影响下发生变化。从fNIRS信息中提取了时域、频域和非线性域共380个生理特征作为OFS评估模型输入,采用支持向量机作为分类器,建立了MWL评估模型。评估模型采用中性情绪刺激下的任务数据作为训练数据,积极情绪和负性情绪刺激下的实验数据作为测试数据,分别取得了92.49%、75.90%和79.99%的平均分类正确率。通过实验数据分析,验证了任务负荷和情绪刺激能够有效影响操作者MWL的实验假设,证明了采用fNIRS技术建立多种情绪状态下MWL评估模型的可行性,为开展复杂任务情况下以MWL为基础的OFS评估提供依据。

Abstract

In order to investigate the feasibility of mental workload (MWL) assessment influenced by different emotional state by using the functional near-infrared spectroscopy (fNIRS) and to support the operator functional state (OFS) assessment based on the MWL assessment, the picture n-back experiments stimulated by multiple emotions, including negative, neutral and positive emotions, are conducted. The experiment data of 16 participants are collected by the means of task performance, subjective rating and fNIRS physiological measurement. The results show that the MWL of participants is affected by external factors in terms of task performance and subjective rating. An MWL assessment model is established, with a total of 380 physiological features of time domain, frequency domain and nonlinearity domain extracted from the fNIRS information as input and the support vector machine as classifier. Average accuracy of classification is 92.49% for training data which is based on task data stimulated by neutral emotion, and 75.9% and 79.99% for test data which is based on experiment data stimulated by positive and negative emotions. By analyzing the data, the experiments verify the hypothesis that the operator′s MWL is effectively affected by task load and emotional stimulus, demonstrate the feasibility of MWL assessment in different emotional state based on fNIRS, and lay a foundation for the OFS assessment on the strength of the MWL evaluation during complex tasks.

Newport宣传-MKS新实验室计划
补充资料

中图分类号:O657.33;TP274.5

DOI:10.3788/aos201636.0517001

所属栏目:医用光学与生物光学

基金项目:国家自然科学基金青年基金(71201148)、飞天基金(FTK201509)、中国航天员科研训练中心国防科技重点实验室实验技术课题(9140C770208150C77320,2012SY54B1701)

收稿日期:2015-11-06

修改稿日期:2015-12-16

网络出版日期:--

作者单位    点击查看

姜劲:中国航天员科研训练中心人因工程国防科技重点实验室, 北京 100094
焦学军:中国航天员科研训练中心人因工程国防科技重点实验室, 北京 100094
潘津津:中国航天员科研训练中心人因工程国防科技重点实验室, 北京 100094
王春慧:中国航天员科研训练中心人因工程国防科技重点实验室, 北京 100094
张朕:中国航天员科研训练中心人因工程国防科技重点实验室, 北京 100094
曹勇:中国航天员科研训练中心人因工程国防科技重点实验室, 北京 100094
杨涵钧:中国航天员科研训练中心人因工程国防科技重点实验室, 北京 100094
徐凤刚:中国航天员科研训练中心人因工程国防科技重点实验室, 北京 100094

联系人作者:姜劲(jiangjin02180018@qq.com)

备注:姜劲(1991-),男,硕士研究生,主要从事功能性近红外光谱、脑电、心电等多生理参数表征脑力负荷、操作者功能状态等方面的研究。

【1】Hoc J M. From human-machine interaction to human-machine cooperation[J]. Ergonomics, 2000, 43(7): 833-843.

【2】Zhang J H, Peng X D, Hua L, et al.. Classifying human operator functional state based on electrophysiological and performance measures and fuzzy clustering method[J]. Cognitive Neurodynamics, 2013, 7(6): 477-494.

【3】Parasuraman R, Cosenzo K A, Visser E D. Adaptive automation for human supervision of multiple uninhabited vehicles: Effects on change detection, situation awareness and mental workload[J]. Journal of Environmental Science & Health Part A, 2010, 38(2): 289-299.

【4】Wilson G F, Russell C A, Davis I. The importance of determining individual operator capabilities when applying adaptive aiding[C]. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2006: 141-145.

【5】Borghini G, Astolfi L, Vecchiato G, et al.. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness [J]. Neuroscience & Biobehavioral Reviews, 2014, 44: 58-75.

【6】Qin G, Yang W, Fei S, et al.. Mental workload measurement for emergency operating procedures in digital nuclear power plants [J]. Ergonomics, 2013, 56(7): 1070-1085.

【7】Wang Raofen. Fuzzy modeling method′s research of operator functional state[D]. Shanghai: East China University of Science and Technology,2012: 3-5.
王娆芬. 过程控制操作员生理信号分析及功能状态建模[D]. 上海: 华东理工大学, 2012: 3-5.

【8】Yang S, Zhang J. An adaptive human-machine control system based on multiple fuzzy predictive models of operator functional state[J]. Biomedical Signal Processing & Control, 2013, 8(3): 302-310.

【9】Fallahi M, Motamedzade M, Heidarimoghadam R, et al.. Effects of mental workload on physiological and subjective responses during traffic density monitoring: A field study[J]. Applied Ergonomics, 2016, 52: 95-103.

【10】Kopf J, Dresler T, Reicherts P, et al.. Effect of emotional content on brain activation and the late positive potential in a word n-back task[J]. PLoS ONE, 2013, 8(9): e75598.

【11】Townsend J D, Sugar C A, Walshaw P D, et al.. Front striatal neuroimaging findings differ in patients with bipolar disorder who have or do not have ADHD comorbidity[J]. Journal of Affective Disorders, 2013, 147(1-3): 389-396.

【12】Yang Shaozeng. Prediction and regulation of operator functional state based on multiple physiological data and fuzzy modeling methods[D]. Shanghai: East China University of Science and Technology, 2014: 7-16.
杨少增. 基于多源生理数据与模糊建模方法的操作员功能状态预测与调节[D]. 上海: 华东理工大学, 2014: 7-16.

【13】Ayaz H, Shewokis P A, Bunce S, et al.. Optical brain monitoring for operator training and mental workload assessment [J]. NeuroImage, 2012, 59(1): 36-47.

【14】Pan Jinjin, Jiao Xuejun, Jiang Jin, et al.. Mental workload assessment based on functional near-infrared spectroscopy[J]. Acta Optica Sinica, 2014, 34(11): 1130002.
潘津津, 焦学军, 姜劲, 等. 利用功能性近红外光谱成像方法评估脑力负荷[J]. 光学学报, 2014, 34(11): 1130002.

【15】Pan Jinjin, Jiao Xuejun, Jiao Dian, et al.. Study on variation in cortex oxygen with task features using functional near-infrared spectroscopy[J]. Acta Optica Sinica, 2015, 35(8): 0817001.
潘津津, 焦学军, 焦典, 等. 利用功能性近红外光谱法研究大脑皮层血氧情况随任务特征变化规律[J]. 光学学报, 2015, 35(8): 0817001.

【16】Izzetoglu K, Ayaz H, Hing J T, et al.. UAV operators workload assessment by optical brain imaging technology (fNIR)[M]. //Handbook of unmanned aerial vehicles. New York: Springer, 2015: 2475-2500

【17】Zhang Yong, Chen Bin, Li Dong. A three-dimensional geometric Monte Carlo method for simulation of light propagation in biological tissues[J]. Chinese J Lasers, 2015, 42(1): 0104003.
张永, 陈斌, 李东. 一种模拟生物组织内光传播的三维几何蒙特卡洛方法[J]. 中国激光, 2015, 42(1): 0104003.

【18】Xiong Yang, Si Minzhen, Gao Fei, et al.. Study on cervical cancer oxyhemoglobin using near-infrared surface-enhanced Raman spectroscopy[J]. Chinese J Lasers, 2015, 42(1): 0115001.
熊洋, 司民真, 高飞, 等. 基于NIR-SERS光谱技术分析宫颈癌氧合血红蛋白[J]. 中国激光, 2015, 42(1): 0115001.

【19】Chiarelli A M, Maclin E L, Fabiani M, et al.. A kurtosis-based wavelet algorithm for motion artifact correction of fNIRS data[J]. NeuroImage, 2015, 112: 128-137.

【20】Wu Chunyang, Lu Qipeng, Ding Haiquan, et al.. Near-infrared non-invasive blood glucose measurement using human tissue fluid[J]. Acta Optica Sinica, 2013, 33(11): 1117001.
吴春阳, 卢启鹏, 丁海泉, 等. 利用人体组织液进行近红外无创血糖测量[J]. 光学学报, 2013, 33(11): 1117001.

【21】Zhou Zhenyu, Yang Hongyu, Gong Hui, et al.. Brain signal analysis of functional near-infrared imaging based on Hilbert-Huang transform[J]. Acta Optica Sinica, 2007, 27(2): 307-312.
周振宇, 杨宏宇, 龚辉, 等. 基于希尔伯特-黄变换的近红外脑功能成像信号分析[J]. 光学学报, 2007, 27(2): 307-312.

【22】Piper S K, Krueger A, Koch S P, et al.. A wearable multi-channel fNIRS system for brain imaging in freely moving subjects [J]. NeuroImage, 2014, 85(2): 64-67.

【23】Hall M, Chaudhary U, Rey G, et al.. Fronto-temporal mapping and connectivity using NIRS for language-related paradigms[J]. Journal of Neurolinguistics, 2013, 26(1): 178-194.

【24】Doi H, Nishitani S, Shinohara K. NIRS as a tool for assaying emotional function in the prefrontal cortex[J]. Frontiers in Human Neuroscience, 2013, 7(1): 56-67.

【25】Ozawa S, Matsuda G, Hiraki K. Negative emotion modulates prefrontal cortex activity during a working memory task: A NIRS study[J]. Frontiers in Human Neuroscience, 2014, 8(2): 46.

【26】Fishburn F A, Norr M E, Medvedev A V, et al.. Sensitivity of fNIRS to cognitive state and load[J]. Frontiers in Human Neuroscience, 2014, 8(2): 76.

【27】Lobacheva E M, Galatenko Y N, Gabidullina R F, et al.. Automated real-time classification of functional states based on physiological parameters[J]. Procedia-Social and Behavioral Sciences, 2013, 86: 373-378.

【28】Wilson G F, Russell C A. Real-time assessment of mental workload using psychophysiological measures and artificial neural networks[J]. Journal of the Human Factors & Ergonomics Society, 2003, 45(4): 635-643.

【29】Zhong Y, Zhang J. Operator functional state classification using least-square support vector machine based on recursive feature elimination technique[J]. Computer Methods & Programs in Biomedicine, 2014, 113(1): 101-115.

【30】Ting C H, Mahfouf M, Nassef A, et al.. Real-time adaptive automation system based on identification of operator functional state in simulated process control operations[J]. IEEE Transactions on Systems Man & Cybernetics Part A, 2010, 40(2): 251-262.

【31】Kirilina E, Jelzow A, Heine A, et al.. The physiological origin of task-evoked systemic artefacts in functional near infrared spectroscopy[J]. NeuroImage, 2012, 61(1): 70-81.

【32】Xu C, Bray S, Reiss A L. Functional near infrared spectroscopy (NIRS) signal improvement based on negative correlation between oxygenated and deoxygenated hemoglobin dynamics[J]. NeuroImage, 2010, 49(4): 3039-3046.

【33】Haeussinger F B, Sler T, Heinzel S, et al.. Reconstructing functional near-infrared spectroscopy (fNIRS) signals impaired by extra-cranial confounds: An easy-to-use filter method[J]. NeuroImage, 2014, 95(8): 69-79.

【34】Peng Mingjin, Li Zhi. Analysis and feature extraction of laser micro-Doppler signatures based on Hilbert-Huang transforms [J]. Chinese J Lasers, 2013, 40(8): 0809004.
彭明金, 李智. 基于希尔伯特-黄变换的激光微多普勒信号分析与特征提取[J]. 中国激光, 2013, 40(8): 0809004.

【35】Xu Lu. A study on feature selection algorithm based on SVM-RFE and particle swarm optimization[D]. Changsha: Hunan Normal University, 2014: 63-75.
许露. 基于SVM-RFE和粒子群算法的特征选择算法研究[D]. 长沙: 湖南师范大学, 2014: 63-75.

【36】Durantin G, Gagnon J F, Tremblay S, et al.. Using near infrared spectroscopy and heart rate variability to detect mental overload[J]. Behavioural Brain Research, 2014, 259(2): 16-23.

【37】Hu T Y, Xie X, Li J. Negative or positive? The effect of emotion and mood on risky driving[J]. Transportation Research Part F, 2013, 16: 29-40.

引用该论文

Jiang Jin,Jiao Xuejun,Pan Jinjin,Wang Chunhui,Zhang Zhen,Cao Yong,Yang Hanjun,Xu Fenggang. Assessment of Mental Workload Influenced by Different Emotional State Using fNIRS[J]. Acta Optica Sinica, 2016, 36(5): 0517001

姜劲,焦学军,潘津津,王春慧,张朕,曹勇,杨涵钧,徐凤刚. 利用fNIRS研究情绪状态下的脑力负荷评估[J]. 光学学报, 2016, 36(5): 0517001

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF