ICA 的梯度下降算法框架

罗一涵; 付承毓; 舒勤

doi:doi:10.3969/j.issn.1003-501x.2009.09.020

光电工程, 2009, 36 (9): 104, 网络出版: 2010-01-31

ICA 的梯度下降算法框架

Framework of Gradient Descent Algorithms for ICA

论文大纲

罗一涵 ^1,*付承毓 ¹舒勤 ²

作者单位

¹ 中国科学院光电技术研究所,成都 610209

² 四川大学电气信息学院,成都 610065

摘要

为了设计更多有效的独立分量分析（ICA)算法,本文提出了ICA 梯度下降算法（GDA)的一般框架,覆盖了许多目前流行的算法,如Infomax,MMI,MLE 等等。该框架由一种新的基于II 类超加（减)性函数的参比函数理论导出,并采用推广的EASI 形式作为更新规则来获得更好的性能。同时本文也展示了一个基于二次熵函数的框架使用例子,并提出了其梯度的快速计算方法,最后仿真证明了它的有效性。实验结果表明,该框架非常实用,可作为开发更多有效ICA 算法的有利工具。

Abstract

To design more effective algorithms for Independent Component Analysis (ICA), a general framework of Gradient Descent Algorithms (GDAs) for ICA was proposed, which covers many popular algorithms such as Infomax, Minimization of Mutual Information (MMI), Maximum Likelihood Estimation (MLE) and so on. This framework was derived from a new theory of the contrast functions for ICA based on the superadditive (or subadditive) function of class II. For better performances, the Equivariant Adaptive Separation via Independence (EASI) form was generalized and used as the updating rule. An example of using the framework was also shown based on the quadratic entropy. Furthermore, a fast method of computing the gradient in the example was proposed and the simulation proved its validity. The results demonstrate that this framework is a useful tool to discover more effective algorithms for ICA.

参考文献

[1] 杨福生,洪波. 独立分量分析的原理与应用 [M]. 北京:清华大学出版社,2006.

YANG Fu-sheng,HONG Bo. Theory and application of Independent Component Analysis [M]. Beijing:Tsinghua University Press,2006.

[2] Hyv rinen A,Karhunen J,Oja E. Independent Component Analysis [M]. A Wiley-Interscience Publication. JOHN WILEY&SONS INC,2001.

[3] . An information-maximization approach to blind separation and blind deconvolution[J]. Neural Computation (S0899-7667), 1995, 7(6): 1129-1159.

[4] . Independent component analysis using an extended infomax algorithm for mixed sub-Gaussian and super-Gaussian sources[J]. Neural computation (S0899-7667), 1999, 11(2): 417-441.

[5] Shriki O,Sompolinsky H,Lee D D. An Information Maximization Approach to Overcomplete and Recurrent Representations [C]// 12th Conference on Neural Information Processing Systems (NIPS 2000),Nov 27-Dec 2,2000:87-93.

[6] . Minimax Mutual Information Approach for Independent Component Analysis[J]. Neural Computation (S0899-7667), 2004, 16(6): 1235-1252.

[7] Hyv rinen A. Denoising of sensory data by maximum likelihood estimation of sparse components [C]// Proc Int Conf on Artificial Neural Networks(ICANN’98),Skovde,Sweden,Sept 2-4,1998:141-146.

[8] Pham D T. Contrast functions for ICA and sources separation [EB/OL]. 2001. http://ljk.imag.fr/membres/Dinh-Tuan.Pham/publics- triees.html.

[9] CHEN Zhe,MA Jin-wen. Contrast Functions for Non-circular and Circular Sources Separation in Complex-Valued ICA [C]//Proceedings of 2006 IEEE International Joint Conference on Neural Networks (IJCNN’06),Vancouver,BC,2006:465-472.

[10] . Natural gradient works efficiently in learning[J]. Neural Computation (S0899-7667), 1998, 10(2): 251-276.

[11] Cardoso J F. Equivariant adaptive sources separation [J]. IEEE Trans on Signal Processing (S1053-587X),1996,SP-44(12):3017-3029.

[12] Principe J,Xu D,Fisher J. Unsupervised adaptive filtering [M]. New York:Wiley,2000.

[13] . Blind source separation using Renyi’s mutual information[J]. IEEE Signal Processing Letters (S1070-9908), 2001, 8(6): 174-176.

罗一涵, 付承毓, 舒勤. ICA 的梯度下降算法框架[J]. 光电工程, 2009, 36(9): 104. LUO Yi-han, FU Cheng-yu, SHU Qin. Framework of Gradient Descent Algorithms for ICA[J]. Opto-Electronic Engineering, 2009, 36(9): 104.

ICA 的梯度下降算法框架

关于本站 Cookie 的使用提示

全站搜索

ICA 的梯度下降算法框架

相关论文

相关资讯

关于本站 Cookie 的使用提示

全站搜索