首页 > 论文 > 光学 精密工程 > 26卷 > 1期(pp:184-192)


Face reconstruction fused with generic morphable model

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文


基于图像对的立体重建是用于获取人脸三维信息的通用方法, 但根据图像数据和重建算法所得到的三维重建结果存在各种误差, 本文对通用形变模型进行改进并与三维立体重建融合以得到更精确的重建结果。首先使用Max-Margin对象检测算法来获取面部边界框, 其中回归树集合法能直接从像素强度的稀疏子集识别面部特征点。然后通过PCA颜色模型生成形状和颜色的三维面部统计模型, 利用ISOMAP算法将三维网格转换为二维表面并提取纹理信息, 得到面部模型。最后在源网格上进行两步非刚性表面配准的变形过程: 先通过对源网格进行二次采样来选择少量网格点来表示源的全局变化, 并选取径向基函数(RBF)进行非刚性全局变形; 再对源顶点进行Procrustes分析获得非刚性变换, 再通过加权方案来进行k-近邻变换, 得到平滑的局部变形。将单图像重建的面部模型, 立体重建的面部模型和本文的面部变形模型与高质量扫描云图进行对齐比较, 得到面部变形模型的3个RMS值分别为2.795 2, 2.102 8和2.153 4, 相比于其他模型, 面部变形模型更接近高质量扫描云图, 即与原图像一致性更高, 误差更小。面部变形模型的定性和定量分析表明, 立体重建与人脸一般形状信息的组合在几何信息的表达上优于基于通用模型的单个图像重建以及未考虑通用模型的立体重建。


Stereo reconstruction from image pairs is a genetic method for 3D acquisition of human faces. Depending on available imagery and reconstruction algorithm, the resulting 3D reconstructions may have deficits. We improve generic morphable model and fused with stereo reconstruction to remedy such deficits. Firstly, we obtain the face bounding box by Max-Margin Object Detection, and recognition the face feature points directly from a sparse subset of pixel intensities by the Ensemble of Regression Treesmethod. Secondly, through the color PCA model to generate the shape and color of the 3D facial statistical model, the ISOMAP algorithm is used to convert the 3D mesh to 2D surface and extract the texture information to get the facial model.Finally, the surface registration with two step non-rigid change on source mesh: we mean the global deformation in the source by using few anchor points on the source mesh, and using the Radial Basis Function (RBF) to express non-rigid global deformation; and then Procrustes analysis is applied for non-rigid transformation of the source vertex, the k-nearest neighbor transformation through weighted scheme tosmooth the local deformation. The experiment result shows the high-quality scan which is used for cloud comparison with the face model, stereo reconstruction and our deformed face model, after aligning all models with the high quality scan, three RMS facial deformation model values are 2.795 2, 2.102 8, 2.153 4.Comparing to other models, deformed face model has smaller discrepancies dominate more strongly, and the deformed face model is visually more shape consistent to the image. Qualitative and quantitative analysis of the deformed face model shows that the combination of stereo reconstruction with general shape information about human faces is geometrically superior to both the reconstruction based on single image from generic model as well as the stereo reconstruction without consideration of the generic model.









作者单位    点击查看

高瞻宇:中国科学院 长春光学精密机械与物理研究所 空间机器人中心创新研究室, 吉林 长春 130033中国科学院大学, 北京 100049
顾营迎:中国科学院 长春光学精密机械与物理研究所 空间机器人中心创新研究室, 吉林 长春 130033
吕耀宇:中国科学院 长春光学精密机械与物理研究所 空间机器人中心创新研究室, 吉林 长春 130033中国科学院大学, 北京 100049
徐振邦:中国科学院 长春光学精密机械与物理研究所 空间机器人中心创新研究室, 吉林 长春 130033
吴清文:中国科学院 长春光学精密机械与物理研究所 空间机器人中心创新研究室, 吉林 长春 130033


备注:高瞻宇(1991-), 男, 辽宁沈阳人, 博士研究生, 2013年于东北大学获得学士学位, 主要从事计算机视觉与三维重建方面的研究。

【1】BLANZ V, VETTER T. A morphable model for the synthesis of 3D faces[C]. ACM Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 1999: 187-194.

【2】PAYSAN P, KNOTHE R, AMBERG B, et al.. A 3D face model for pose and illumination invariant face recognition[C]. IEEE Proceedings of the 6th IEEE International Conference on Advanced Video and Signal Based Surveillance, 2009: 296-301.

【3】ROMDHANI S, BLANZ V, VETTER T. Face identification by fitting a 3D morphable model using linear shape and texture error functions [M]// HEYDEN A, SPARR G, NIELSEN M, et al.. Computer Vision-ECCV 2002. Berlin, Heidelberg: Springer, 2002: 3-19.

【4】VAN ROOTSELER R T A, SPREEUWERS L J, VELDHUIS R N J. Using 3D morphable models for face recognition in video[C]. Proceedings of the 33rd WIC Symposium on Information Theory in the Benelux, Werkgemeenschap voor Informatie-en Co mmunicatietheorie(WIC), 2012: 235-242.

【5】HUBER P, FENG ZH H, CHRISTMAS W, et al.. Fitting 3D morphable face models using local features[C]. IEEE Proceedings of 2015 IEEE International Conference on Image Processing (ICIP), 2015.

【6】高瞻宇, 顾营迎, 刘宇航, 等. 采用简化Brown模型及改进BFGS法的相机自标定 [J]. 光学 精密工程, 2017, 25(9): 2532-2540.
GAO ZH Y, GU Y Y, LIU Y H, et al.. Self-calibration based on simplified brown non-linear camera model and modified BFGS algorithm [J]. Opt. Precision Eng., 2017, 25(9): 2532-2540. (in Chinese)

【7】KAZEMI V, SULLIVAN J. One millisecond face alig nment with an ensemble of regression trees[C]. IEEE, Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014: 1867-1874.

【8】郑明秋, 杨帆. 改进非负矩阵分解的神经网络人脸识别 [J]. 液晶与显示, 2017, 32(3): 213-218.
ZHENG M Q, YANG F.Face recognition based on improved NMF and neural network [J]. Chinese Journal of Liquid Crystals and Displays, 2017, 32(3): 213-218. (in Chinese)

【9】KING D E. Max-margin object detection[Z]. arXiv preprint arXiv: 1502.00046, 2015.

【10】RICHARD H, ZISSERMAN A. Multiple View Geometry in Computer Vision [M]. Cambridge: Cambridge University Press, 2000.

【11】LI H, SUMNER R W, PAULY M. Global correspondence optimization for non-rigid registration of depth scans [J]. Computer Graphics Forum, 2008, 27(5): 1421-1430.

【12】LEVI Z, LEVIN D. Shape deformation via interior RBF [J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(7): 1062-1075.

【13】FASSHAUER G E. Meshfree Approximation Methods with MATLAB [M]. River Edge, NJ, USA: World Scientific, 2007.

【14】SUMNER R W, SCHMID J, PAULY M. Embedded deformation for shape manipulation [J]. ACM Transactions on Graphics (TOG), 2007, 26(3): 80.

【15】CloudCompare (version2.6.2)[EB/OL]. http: //www.cloudcompare.org/.

【16】KING D. Real-time face pose estimation[EB/OL]. (2015-12). http: //blog.dlib.net/2014/08/real-time-face-pose-estimation.html.


GAO Zhan-yu,GU Ying-ying,L Yao-yu,XU Zhen-bang,WU Qing-wen. Face reconstruction fused with generic morphable model[J]. Optics and Precision Engineering, 2018, 26(1): 184-192

高瞻宇,顾营迎,吕耀宇,徐振邦,吴清文. 融合通用形变模型信息的面部三维重建[J]. 光学 精密工程, 2018, 26(1): 184-192


【1】刘东生,陈建林,费 点,张之江. 基于深度相机的大场景三维重建. 光学 精密工程, 2020, 28(1): 234-243

【2】林国军,蒋行国,杨明中,李兆飞,解 梅. 基于叠加协同表示分类的人脸识别. 液晶与显示, 2020, 35(2): 161-166

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF