人眼自动对焦及瞳孔中心自动对准系统 
下载： 661次
The measurement of ocular optical biological parameters is a key part of the ophthalmic examination, and it is very important for the diagnosis and treatment of ophthalmic diseases such as myopia and cataract. For the existing ophthalmic measuring instruments, operators mainly use the joystick to manually control the movement of the instruments, so as to realize focus and alignment of the subject's eyes. The evaluation criteria of this process are subjective, and the process is slow and involves complicated operations. In recent years, many scholars have carried out research on human eye autofocus and pupil center positioning methods and have proposed some solutions. However, there are still problems such as poor real-time performance of the algorithm, relatively insufficient positioning accuracy, and poor generality. Therefore, the application of autofocus and auto-alignment technology reflects a development direction and practical demand of automatic ophthalmic measuring instruments.
A human eye autofocus and pupil center auto-alignment system for ophthalmic measuring instruments is built. The system is mainly composed of a fixed vision module, an LED lamp bead plate, an imaging module, a three-axis electric displacement stage, and a head holder. By analyzing the characteristics of human eye images, an autofocus algorithm based on a four-neighborhood-multi-directional and two-level gradient function (FMTG algorithm) was proposed. The algorithm replaces the center pixel with the mean value of the center pixel and the four-neighborhood pixel for calculation, calculates the two-level gradient in the horizontal direction, vertical direction, and direction with an oblique of 45° at the same time, and takes the product of the two-level gradient operators as the sharpness evaluation value. A pupil center positioning algorithm (LPC algorithm) based on the intelligent region of interest (ROI) window of a convolutional neural network was proposed. The algorithm extracts the image coordinates of the pupil center and optical axis for alignment through steps including the extraction of the pupil region of interest, histogram-based binarization segmentation, and high-precision ellipse recognition and fitting. The performance of the proposed FMTG algorithm was verified by the system built in this paper and by actually shooting four groups of human eye-defocused image sequences. Six commonly used evaluation functions were selected for comparison, and their normalized sharpness evaluation curves were drawn. Three quantitative indicators, namely, single-frame processing time, sharpness ratio, and sensitivity, as well as two important properties such as unimodality and accuracy, were selected to comprehensively evaluate the performance of the functions. The performance of the proposed LPC algorithm and other algorithms for comparison was evaluated in terms of the average computation time, accuracy, and average error through the test set of real-shot human eye images. In addition, the proposed LPC algorithm was tested under complex conditions with different illumination and partial pupil occlusion to evaluate the universality of the algorithm.
The FMTG algorithm proposed in this paper has a sharper evaluation curve shape and a stronger ability to distinguish on-focus images from out-of-focus images (Fig. 10). Its average calculation time is about 13 ms, with a sharpness ratio of 93.531, and the sensitivity is 0.329, which are all better than that of other six traditional evaluation algorithms and reflect good unimodality and accuracy (Table 1). The error curve of the LPC algorithm proposed in this paper has small fluctuation and good stability, and the algorithm can adapt to certain environmental changes (Fig. 12). The average calculation time of the LPC algorithm is 10.2 ms, with an accuracy of 97.14%, and the average error is 1.4 pixel, which indicate that the LPC algorithm has been greatly improved compared with the Hough method and the improved Hough method (Table 2). In addition, the proposed LPC algorithm can adapt to lighting environments with different brightness and darkness and situations where the pupil is partially occluded, and thus it shows good universality (Fig. 13).
This paper proposed a human eye autofocus and pupil center auto-alignment system applied to ophthalmic measuring instruments. The system realized the autofocus of the human eye region through an autofocus algorithm based on the FMTG algorithm and then delimited the ROI window through a convolutional neural network. In addition, it adopted an ellipse recognition and fitting algorithm to accurately locate the pupil center and optical axis and finally used the mechanical movement of the three-axis high-precision displacement stage to achieve the alignment of the human eye pupil center and optical axis. The experimental results show that the average calculation time of the FMTG algorithm proposed in this paper is about 13 ms, and the evaluation indicators such as sharpness ratio and sensitivity are higher than those of the traditional algorithms, and the FMTG algorithm has good unimodality and accuracy. The proposed LPC algorithm has an average calculation time of 10.2 ms and an accuracy of 97.14%. It can adapt to complex environments with different lighting and partially occluded pupils and has great accuracy and universality. The experimental results show that the system built in this paper can meet the requirements of accuracy, timeliness, and robustness of ocular biological parameter measurement, improve the automation level of the instruments, and contribute to the intelligent development of ophthalmic measurement instruments.
1 引言
眼光学生物参数的测定是眼科检查时的关键环节，对于近视、白内障、青光眼等眼科疾病的预防和诊疗具有重要意义［1］。常用的非接触眼科测量设备有眼底相机、眼生物参数测量仪、角膜地形图仪等，用于获取眼轴长度、角膜曲率、前房深度和晶状体厚度等参数［2-3］。人眼区域清晰成像，同时使测量光穿过瞳孔中心是保证眼生物参数准确性的前提。现今市场上主流的眼科测量仪器是操作者通过摇杆来手动控制仪器的移动，从而对受试者的眼睛进行对焦和对准，对焦清晰度以及对准精确度由操作者来主观判断；同时受试者难以长时间保持紧盯固视灯的状态，眨眼、眼球不自觉运动均会使测量结果出现偏差，固视效果无法得到保障。因此，现今的手动操作模式对于操作者提出了较高的要求。
为提高眼科测量仪器的测量精度和自动化水平，近年来有许多学者分别就人眼自动对焦和瞳孔中心定位方法展开研究。方丽萍等［4］提出了一种四方向Sobel算子对数灰度级像素统计算法，用于角膜曲率测量中的人眼对焦评价；王萌等［5］通过同轴红外光源在人眼处反射的光斑来判断清晰程度，并对爬山搜索策略进行了改进；苑玮琦等［6］评估了多种评价函数对于虹膜图像的性能，同时建立了清晰程度与距离的关系曲线；宋保全等［7］提出了一种用于非接触眼压计的基于亚像素轮廓的多模板匹配算法，该算法具有较好的鲁棒性和泛化性；林舒欣等［8］提出基于空洞全卷积网络的非理想状态的瞳孔定位算法，有效提高了非理想状态的瞳孔定位准确度；蔡怀宇等［9］根据红外眼科疾病检测的要求开发出了瞳孔定位算法，该算法具有定位误差小、速度快和抗干扰能力强的特点；王晶等［10］研发了一款瞳孔中心点自动定位与对准装置，该装置可作为无人值守式眼底筛查设备的辅助设备，有效提高了眼底筛查的效率和准确率。然而，上述方法依然存在着算法实时性较差、定位精度相对不足和通用性不佳等问题，自动对焦和自动对准技术的应用是自动化眼科测量仪器的发展方向和现实需求。
基于以上实际要求，本文提出了基于四邻域-多方向两级梯度函数的自动对焦评价算法以及融合卷积神经网络智能感兴趣区域（ROI）窗口的瞳孔中心定位算法，并将该算法应用于自主研制的人眼自动对焦及瞳孔中心自动对准系统。该装置可在眼生物参数测量前快速自动寻找并移动到人眼成像最清晰的位置，同时实现瞳孔中心与测量光轴的精确对准。本文所提出的应用于眼科测量仪器的人眼自动对焦和瞳孔中心自动对准装置，对于提升仪器的测量精确度、实时性、可重复性以及对受试者的友好度都具有重要价值。
2 系统组成
应用于眼科测量仪器的人眼自动对焦及瞳孔中心自动对准系统如
图 1. 人眼自动对焦及瞳孔中心自动对准装置组成结构
Fig. 1. Structure of human eye auto-focus and pupil center auto-alignment device
3 人眼自动对焦
3.1 传统对焦评价函数
对焦评价函数又称为清晰度评价函数，是自动对焦过程中用来判断某个位置图像清晰程度的依据。评价函数一般分为梯度类、频域类和统计学类，其中梯度类和统计学类函数通常具有计算量小、鲁棒性好等优点，因此得到广泛应用。经典的评价函数有Brenner、Tenengrad、灰度差分（SMD）函数、Laplace、自相关（Vollaths）函数和方差（Variance）函数等［11-16］。
3.2 四邻域-多方向两级梯度函数
人眼自动对焦时，由于成像系统的景深限制，图像中眼球区域与干扰区域（如睫毛、眼睑等）的正焦位置不同，因此对焦评价函数的曲线需能准确反映眼球区域的清晰度变化；同时受试者很难长时间保持眼球的稳定，而眨眼、眼球移动会对自动对焦产生严重影响，所以设计的对焦评价函数需具备较高的实时性。
根据人眼图像的特点，本文提出的四邻域-多方向两级梯度函数（后文图表中均表述为FMTG函数），将中心像素与四邻域像素的均值替代中心像素参与计算，同时计算水平、垂直和斜45°方向两级梯度，用两级梯度算子的乘积作为清晰度评价值。该算法能避免局部锐度的影响，准确评价整体图像的清晰程度，具有更好的稳定性。该算法的公式如下：
式中：
4 瞳孔中心定位与对准
通过分析在正常光照下拍摄的人眼图像发现，睫毛和眼睑的遮挡、虹膜的泛灰色区域以及眼睑的大片明亮区域均可能对瞳孔中心的定位造成干扰，因此去除这些干扰项的影响是实现瞳孔中心定位的前提。本文提出一种融合卷积神经网络智能ROI窗口的瞳孔中心定位算法，该算法能有效提取瞳孔区域并进行中心定位，流程如
图 3. 瞳孔中心定位与对准算法的流程图
Fig. 3. Flowchart of pupil center positioning and alignment algorithm
4.1 感兴趣区域提取与二值化分割
人眼图像的灰度主要分为三部分：瞳孔及睫毛的低灰度部分、虹膜的中灰度部分，以及反射亮点和明亮区域的高灰度部分。但是各灰度分布之间并非以单独块状形式存在，这将对二值化分割以及后续的瞳孔椭圆识别造成干扰，影响算法的普适性。因此，本文采用YOLOv4-tiny算法［17］实现瞳孔区域的提取，该算法是YOLOv4目标检测算法的轻量化版本，骨干网络为CSPDarknet53-tiny，具有极高的检测速度，能够满足本系统的实时性要求。
图 4. 采集的人眼图像与ROI图像。（a）人眼原图像；（b）瞳孔ROI图像
Fig. 4. Collected image of human eye and ROI image. (a) Original image of human eye; (b) ROI image of pupil
感兴趣区域提取前后的灰度分布如
图 5. ROI提取前后的灰度直方图分布。（a）原图像的灰度直方图分布；（b）ROI的灰度直方图分布
Fig. 5. Grayscale histogram distributions before and after extraction of ROI. (a) Grayscale histogram distribution of original image; (b) grayscale histogram distribution of ROI
图 6. 阈值分割后的ROI图像。（a）T1阈值分割后的图像；（b）T2阈值分割后的图像
Fig. 6. ROI image after threshold segmentation. (a) Image after T1 threshold segmentation; (b) image after T2 threshold segmentation
4.2 椭圆识别和拟合
本文采取了一种高精度椭圆识别的方法［18］，对T1阈值分割后的图像进行瞳孔边缘检测、椭圆拟合及中心计算。该方法处理的结果如
图 7. 瞳孔中心与光轴中心示意图。（a）瞳孔识别及中心点；（b）光轴椭圆拟合及中心点
Fig. 7. Schematic diagrams of pupil center and optical axis center. (a) Pupil identification and center point; (b) optical axis ellipse fitting and center point
经阈值T2分割后的图像中的8个白色小区域代表8个LED灯珠在角膜上反射的光点，这8个光点的中心即为测量光轴的中心。首先通过连通域标记的方法［19］，将这8个小区域进行标记，然后利用质心法计算其质心坐标，最后对8个质心坐标进行最小二次椭圆拟合，并求取椭圆中心，该中心点即为测量光轴在图像上的坐标。本步骤处理后的结果如
在得到瞳孔中心和光轴的图像坐标后，计算两者之间的相互关系，再通过位移台X和Y方向的移动，使得瞳孔中心与光轴重合，实现瞳孔对准。瞳孔中心与光轴相对位置的示意图如
图 8. 瞳孔中心与光轴相对位置示意图
Fig. 8. Schematic diagram of relative position of pupil center and optical axis
5 实验结果与分析
本文设计了两组实验分别用于验证人眼自动对焦算法和瞳孔中心定位算法的有效性及准确性，实验图像均使用本文搭建的系统进行采集。
5.1 人眼自动对焦实验
在验证人眼自动对焦算法方面，将受试者头部固定在额托支架上，调整瞳孔至观测视场中心位置，驱动电动位移台以1 mm为步长沿Z方向全行程（20 mm）采集人眼离焦图像序列，共采集4名受试者的离焦图像序列，分别标记为序列1~4，每个序列中包含21张图像。将本文提出的四邻域-多方向两级梯度函数与6种传统评价函数进行对比。
图 9. 人眼离焦序列1示意图。（a）序列第5张图像；（b）序列第9张图像；（c）序列第13张图像；（d）序列第17张图像；（e）序列第21张图像
Fig. 9. Schematic diagrams of defocus sequence-1 of human eyes. (a) 5th image of sequence; (b) 9th image of sequence; (c) 13th image of sequence; (d) 17th image of sequence; (e) 21st image of sequence
图 10. 人眼离焦图像序列的归一化清晰度评价曲线。（a）序列1；（b）序列2；（c）序列3；（d）序列4
Fig. 10. Normalized sharpness evaluation curves of human eye defocused image sequences. (a) Sequence 1; (b) sequence 2;(c) sequence 3; (d) sequence 4
图 11. 人眼离焦图像序列4中部分图像。（a）序列第8幅图像；（b）序列第9幅图像
Fig. 11. Partial images of sequence 4 in human eye defocused image. (a) 8th image of sequence; (b) 9th image of sequence
本文选取了单帧处理时间TPF、清晰度比率R、灵敏度Fsen、单峰性、准确性来综合评价函数的性能［20］。单帧处理时间是评判函数实时性的最重要依据。清晰度比率用以表征函数区分准焦-离焦图像的能力，公式如下：
式中：
灵敏度表征的是函数在准焦位置附近变化的剧烈程度，灵敏度越高则说明函数能胜任更精细的对焦，其公式如下：
式中：F为序列某位置的清晰度评价值；
单峰性指图像序列仅有单个峰值点，准确性指函数平均值最大的图像与实际序列中的准焦图像相对应，这两种指标在后文表中以“√”表示性能优秀，“—”表示性能良好，“×”表示性能一般。
分析
表 1. 人眼图像离焦序列1~4评价曲线的平均性能指标
Table 1. Average performance indexes of evaluation curves of out-of-focus sequences 1-4 of human eye images
|
5.2 瞳孔中心定位实验
本文在MatlabR2022a环境下对瞳孔中心定位与对准算法进行实验。首先，使用Matlab内置的yolov4ObjectDetector及相关实例文件来训练本系统采集的人眼图像集；通过系统采集的105张人眼图像测试本文算法，并与常用的Hough变换法［21］、改进的Hough法［22］进行对比分析。本文的瞳孔自动定位算法在下文用LPC表示。
图 12. 3种算法得到的瞳孔中心坐标与手动标定的瞳孔中心坐标的误差
Fig. 12. Error between pupil center coordinates obtained by three algorithms and manually calibrated pupil center coordinates
表 2. 3种算法在实验图像下的平均计算时间、准确度和平均误差
Table 2. Average computation time, accuracy, and average error of three algorithms under experimental images
|
本文算法还在不同光线和瞳孔部分遮挡等复杂条件下进行了测试。
图 13. 复杂条件下本文算法的处理结果。（a）暗光条件；（b）亮光条件；（c）（d）瞳孔部分遮挡
Fig. 13. Processing results of algorithm in this paper under complex conditions. (a) Dark light condition; (b) bright light condition; (c)(d) partial occlusion of pupil
6 结论
提出了一种应用于眼科测量仪器的人眼自动对焦及瞳孔中心自动对准系统，该系统通过基于四邻域-多方向两级梯度函数的自动对焦算法实现人眼区域自动对焦，由卷积神经网络划定ROI窗口，结合椭圆识别与拟合算法对瞳孔中心和光轴进行准确定位，最后利用三维高精度位移台的机械移动实现人眼瞳孔中心与测量光轴的对准。
实验结果证明，所提出的自动对焦算法的平均计算时间约为13 ms，瞳孔中心定位对准算法的平均计算时间为10.2 ms，准确度为97.14%。相较于传统方法，这两种算法均具有更高的准确性以及较好的实时性和鲁棒性，能够在瞳孔部分遮挡等复杂环境中使用，满足了眼科测量仪器的自动化要求，有助于眼生物参数测量仪等眼科检查设备的智能化发展。
[1] O′Donnell C, Hartwig A, Radhakrishnan H. Correlations between refractive error and biometric parameters in human eyes using the LenStar 900[J]. Contact Lens and Anterior Eye, 2011, 34(1): 26-31.
[2] 胡丹, 赵刚平, 余建洪, 等. Lenstar 900与A型超声和角膜曲率计在白内障患者眼生物学测量的比较[J]. 国际眼科杂志, 2014, 14(8): 1440-1443.
Hu D, Zhao G P, Yu J H, et al. Comparison on eye biometry of Lenstar 900, A-scan ultrasound and keratometer in patients with cataract[J]. International Eye Science, 2014, 14(8): 1440-1443.
[3] Kumar K K, Prakash A A, Neeraja T G, et al. To compare central corneal thickness measurements obtained by Pentacam with those obtained by IOLMaster 700, Cirrus anterior segment optical coherence tomography and Tomey specular microscopy in normal healthy eyes[J]. Indian Journal of Ophthalmology, 2021, 69(7): 1713-1717.
[4] 方丽萍, 陈兆学, 陈鹏, 等. 角膜曲率测量中自动识别的对焦评价[J]. 光学技术, 2019, 45(2): 218-223.
[5] 王萌, 陈鸿. 基于图像处理的眼球自动对焦系统研究[J]. 中国科技信息, 2013(7): 46-47.
Wang M, Chen H. Research on autofocus of the eyeball system based on image processing[J]. China Science and Technology Information, 2013(7): 46-47.
[6] 苑玮琦, 杨冉冉, 张开营. 虹膜清晰程度与人眼拍摄距离对应关系研究[J]. 计算机应用研究, 2011, 28(8): 3162-3165.
Yuan W Q, Yang R R, Zhang K Y. Research of relationship between iris clarity and human eye shooting distance[J]. Application Research of Computers, 2011, 28(8): 3162-3165.
[7] 宋保全, 杜文华, 段能全, 等. 非接触眼压计的人眼瞳孔定位算法研究[J]. 电子测量技术, 2022, 45(3): 112-117.
Song B Q, Du W H, Duan N Q, et al. Research on pupil location algorithm of non-contact tonometer[J]. Electronic Measurement Technology, 2022, 45(3): 112-117.
[8] 林舒欣, 陆启桐, 王尚媛, 等. 基于空洞全卷积网络的非理想状态的瞳孔定位[J]. 激光杂志, 2021, 42(8): 33-38.
Lin S X, Lu Q T, Wang S Y, et al. Non-ideal pupil positioning based on hole fully convolutional network[J]. Laser Journal, 2021, 42(8): 33-38.
[9] 蔡怀宇, 史玉, 娄世良, 等. 应用于红外眼科疾病检测的瞳孔定位算法[J]. 中国光学, 2021, 14(3): 605-614.
[10] 王晶, 高峰, 李婉越, 等. 瞳孔中心点自动定位与对准装置[J]. 光学 精密工程, 2019, 27(6): 1370-1377.
[11] Brenner J F, Dew B S, Horton J B, et al. An automated microscope for cytologic research a preliminary evaluation[J]. Journal of Histochemistry & Cytochemistry, 1976, 24(1): 100-111.
[12] TenenbaumJ M. Accommodation in computer vision[D]. Stanford: Stanford University, 1970.
[13] Vollath D. Automatic focusing by correlative methods[J]. Journal of Microscopy, 1987, 147(3): 279-288.
[14] 夏浩盛, 余飞鸿. 数码显微镜自动对焦算法[J]. 激光与光电子学进展, 2021, 58(4): 0400002.
[15] 熊锐, 顾乃庭, 徐洪艳. 一种适应多方向灰度梯度变化的自动对焦评价函数[J]. 激光与光电子学进展, 2022, 59(4): 0418001.
[16] 曾海飞, 韩昌佩, 李凯, 等. 改进的梯度阈值图像清晰度评价算法[J]. 激光与光电子学进展, 2021, 58(22): 2211001.
[17] Zhu R S, Zhao L J. Research and application of face mask detection algorithm based on YOLOV4-tiny[J]. Academic Journal of Computing & Information Science, 2022, 5(1): 64-69.
[18] Lu C, Xia S, Shao M, et al. Arc-support line segments revisited: an efficient high-quality ellipse detection[J]. IEEE Transactions on Image Processing, 2019, 29: 768-781.
[19] He L F, Ren X W, Gao Q H, et al. The connected-component labeling problem: a review of state-of-the-art algorithms[J]. Pattern Recognition, 2017, 70: 25-43.
[20] 翟永平, 周东翔, 刘云辉, 等. 聚焦函数性能评价指标设计及最优函数选取[J]. 光学学报, 2011, 31(4): 0418002.
[21] Wildes R P. Iris recognition: an emerging biometric technology[J]. Proceedings of the IEEE, 1997, 85(9): 1348-1363.
Article Outline
汪路涵, 巩岩, 张艳微, 郎松, 郑汉青. 人眼自动对焦及瞳孔中心自动对准系统[J]. 光学学报, 2023, 43(5): 0511002. Luhan Wang, Yan Gong, Yanwei Zhang, Song Lang, Hanqing Zheng. Human Eye Autofocus and Pupil Center Auto-Alignment System[J]. Acta Optica Sinica, 2023, 43(5): 0511002.
PDF全文
