A probabilistic learning algorithm for robust modeling using neural networks with random weights

Information Sciences

Volumn 313, Issue , 2015, Pages 62-78

  Cao, Feilong ^a   Ye, Hailiang ^a   Wang, Dianhui ^b,c  

^a CHINA JILIANG UNIVERSITY   (China)

^b LA TROBE UNIVERSITY   (Australia)

^c STATE KEY LABORATORY OF SYNTHETICAL AUTOMATION FOR PROCESS INDUSTRIES   (China)

Author keywords

Expectation maximization; Laplace distribution; Neural networks with random weights (NNRWs); Outliers; Probabilistic robust learning

Indexed keywords

APPROXIMATION ALGORITHMS; CLASSIFICATION (OF INFORMATION); LEARNING SYSTEMS; MAXIMUM PRINCIPLE; NEURAL NETWORKS; PROBABILITY DISTRIBUTIONS; STATISTICS;

EXPECTATION - MAXIMIZATIONS; LAPLACE DISTRIBUTIONS; OUTLIERS; RANDOM WEIGHT; ROBUST LEARNING;

LEARNING ALGORITHMS;

EID: 84928230663     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2015.03.039     Document Type: Article

References (42)

1. D.M. Hawkins Identification of Outliers 1980 Chapman and Hall London, UK
2. P.J. Rousseeuw, and A.M. Leroy Robust Regression and Outlier Detection 2005 Wiley New York
3. F.R. Hampel, E.M. Ronchetti, P.J. Rousseeuw, and W.A. Stahel Robust Statistics: The Approach Based on Influence Functions 2011 Wiley New York
4. D.S. Chen, and R.C. Jain A robust backpropagation learning algorithm for function approximation IEEE Trans. Neural Networks 5 3 1994 467 479
5. A. Sanchez, and V. David Robustization of a learning method for RBF networks Neurocomputing 9 1 1995 85 94
6. L. Huang, B.L. Zhang, and Q. Huang Robust interval regression analysis using neural networks Fuzzy Sets Syst. 97 3 1998 337 347
7. C.C. Chuang, S.F. Su, and C.C. Hsiao The annealing robust backpropagation (ARBP) learning algorithm IEEE Trans. Neural Networks 11 2000 1067 1077
8. K. Liano Robust error measure for supervised neural network learning with outliers IEEE Trans. Neural Networks 7 1 1996 246 250
9. J.A.K. Suykens, J. De Brabanter, L. Lukas, and J. Vandewalle Weighted least squares support vector machines: robustness and sparse approximation Neurocomputing 48 1 2002 85 105
10. C.C. Chuang, S.F. Su, J.T. Jeng, and C.C. Hsiao Robust support vector regression networks for function approximation with outliers IEEE Trans. Neural Networks 13 6 2002 1322 1330
11. C.C. Chuang, J.T. Jeng, and M.L. Chan Robust least squares-support vector machines for regression with outliers IEEE Int. Conf. Fuzzy Syst. 2008 312 317
12. B. Biggio, G. Fumera, and F. Roli Multiple classifier systems for robust classifier design in adversarial environments Int. J. Mach. Learn. Cybernet. 1 1-4 2010 27 41
13. X.W. Zhao, J.Y. Liang, and F.Y. Cao A simple and effective outlier detection algorithm for categorical data Int. J. Mach. Learn. Cybernet. 5 3 2014 469 477
14. J. Li, G. Han, J. Wen, and X.B. Gao Robust tensor subspace learning for anomaly detection Int. J. Mach. Learn. Cybernet. 2 2 2011 89 98
15. S. Haykin Neural Network: A Comprehensive Foundation second ed. 2004 Prentice Hall
16. A.N. Tychonoff Solution of incorrectly formulated problems and the regularization method Soviet 4 1963 1035 1038
17. F.L. Cao, Y.P. Tan, and M.M. Cai Sparse algorithms of random weight networks and applications Expert Syst. Appl. 41 2014 2457 2462
18. l 1-norm for image analysis Pattern Recogn. Lett. 33 5 2012 537 542
19. l 1-norm solution is also the sparsest solution Commun. Pure Appl. Math. 59 2006 797 829
20. l 1-norm minimization problems when the solution may be sparse IEEE Trans. Inform. Theory 54 2008 4789 4812
21. l 1 norm factorization in the presence of outliers and missing data by alternative convex programming IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. 1 2005 739 746
22. l 1 norm IEEE Conf. Comput. Vis. Pattern Recogn. 2010 771 778
23. N.Y. Wang, T.S. Yao, J.D. Wang, and D.Y. Yeung A probabilistic approach to robust matrix factorization Computer Vision-ECCV 2012 2012 Springer Berlin, Heidelberg 126 139
24. R. Salakhutdinov, and M. Andriy Probabilistic matrix factorization Adv. Neural Inform. Process. Syst. 1 2007 1257 1264
25. D.F. Andrews, and C.L. Mallows Scale mixtures of normal distributions J. Roy. Stat. Soc. Ser. B (Methodol.) 36 1 1974 99 102
26. W.F. Schmidt, M.A. Kraaijveld, R.P.W. Duin, Feed forward neural networks with random weights, in: Proceedings of 11th IAPR International Conference on Pattern Recognition Methodology and Systems, vol. 2, 1992, pp. 1-4.
27. C.R. Rao, and S.K. Mitra Generalized Inverse of Matrices and Its Applications 1971 Wiley New York
28. Y.H. Pao, and Y. Takefuji Functional-link net computing: theory, system architecture, and functionalities IEEE Computer 25 5 1992 76 79
29. Y.H. Pao, G.H. Park, and D.J. Sobajic Learning and generalization characteristics of the random vector functional-link net Neurocomputing 6 2 1994 163 180
30. B. Igelnik, and Y.H. Pao Stochastic choice of basis functions in adaptive function approximation and the functional-link net IEEE Trans. Neural Networks 6 6 1995 1320 1329
31. P.L. Bartlett The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network IEEE Trans. Inform. Theory 44 2 1998 525 536
32. E.J. Candès, and T. Tao Decoding by linear programming IEEE Trans. Inform. Theory 51 12 2005 4203 4215
33. D.L. Donoho Compressed sensing IEEE Trans. Inform. Theory 52 4 2006 1289 1306
34. B.K. Natarajan Sparse approximate solutions to linear systems SIAM J. Comput. 24 2 1995 227 234
35. S. Kotz, T.J. Kozubowski, and K. Podgorski The Laplace Distribution and Generalizations: A Revisit With Applications to Communications, Exonomics, Engineering, and Finance 2001 Springer
36. K. Lange, and J.S. Sinsheime Normal/independent distributions and their applications in robust regression J. Comput. Graph. Stat. 2 2 1993 175 198
37. A.P. Dempster, N.M. Laird, and D.B. Rubin Maximum likelihood from incomplete data via the EM algorithm J. Roy. Stat. Soc., Ser. B (Methodol.) 39 1 1977 1 38
38. Z.H. Zhang, S.S. Wang, D.H. Liu, and M.I. Jordan EP-GIG priors and applications in Bayesian sparse learning J. Mach. Learn. Res. 13 1 2012 2031 2061
39. H. Drucker, C.J. C Burges, L. Kaufman, A. Smola, and V. Vapnik Support vector regression machines Adv. Neural Inform. Process. Syst. 9 1997 155 161
40. K. Bache, M. Lichman, UCI Machine Learning Repository, School of Information and Computer Science, University of California, Irvine, 2013. < http://archive.ics.uci.edu/ml >.
41. J. Demsar Statistical comparisons of classifiers over multiple data sets J. Mach. Learn. Res. 7 2006 1 30
42. X.Z. Wang, Y.L. He, and D.D. Wang Non-naive Bayesian classifiers for classification problems with continuous attributes IEEE Trans. Cybernet. 44 1 2014 21 39