A novel weight initialization method for the random neural network

Neurocomputing

Volumn 73, Issue 1-3, 2009, Pages 160-168

  Timotheou, Stelios ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Combinatorial optimization; Nonnegative least squares (NNLS); Projected gradient method; Random neural network (RNN); Weight initialization

Indexed keywords

EMERGENCY RESPONSE; EXECUTION TIME; FLOW EQUATIONS; INITIALIZATION METHODS; LINEAR SYSTEM OF EQUATIONS; NONNEGATIVE LEAST SQUARES; NONNEGATIVE LEAST SQUARES (NNLS); NONNEGATIVITY CONSTRAINTS; PROJECTED GRADIENT; PROJECTED GRADIENT METHOD; RANDOM NEURAL NETWORK; RANDOM NEURAL NETWORK (RNN); SOLUTION QUALITY; WEIGHT INITIALIZATION;

COMBINATORIAL OPTIMIZATION; GRADIENT METHODS; LINEAR SYSTEMS;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; CONTROLLED STUDY; LEARNING ALGORITHM; LINEAR SYSTEM; MATHEMATICAL MODEL; PRIORITY JOURNAL; PROCESS OPTIMIZATION; RANDOMIZATION; REGRESSION ANALYSIS; SIGNAL PROCESSING;

EID: 70350714259     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2009.02.023     Document Type: Article

References (30)

1. Gelenbe E. Random neural networks with negative and positive signals and product form solution. Neural Computation 1 4 (1989) 502-510
2. Gelenbe E. Stability of the random neural network. Neural Computation 2 2 (1990) 239-247
3. Gelenbe E. Learning in the recurrent random network. Neural Computation 5 (1993) 154-164
4. Gelenbe E., Ghanwani A., and Srinivasan V. Improved neural heuristics for multicast routing. IEEE Journal of Selected Areas of Communications 15 2 (1997) 147-155
5. Gelenbe E., and Kocak T. Area-based results for mine detection. IEEE Transactions on Geoscience and Remote Sensing 38 1 (2000) 1-14
6. Gelenbe E., and Kocak T. Wafer surface reconstruction from top-down scanning electron microscope images. Microelectronic Engineering 75 (2004) 216-233
7. Abdelbaki H., Gelenbe E., and Kocak T. Neural algorithms and energy measures for EMI based mine detection. Journal of Differential Equations and Dynamical Systems 13 1-2 (2005) 63-86
8. Gelenbe E., Cramer C., Sungur M., and Gelenbe P. Traffic and video quality in adaptive neural compression. Multimedia Systems 4 (1996) 357-369
9. Cramer C., Gelenbe E., and Bakircioglu H. Low bit rate video compression with neural networks and temporal subsampling. Proceedings of the IEEE 84 10 (1996) 1529-1543
10. Cramer C., and Gelenbe E. Video quality and traffic QoS in learning-based subsampled and receiver-interpolated video sequences. IEEE Journal on Selected Areas in Communications 18 2 (2000) 150-167
11. Oke G., and Loukas G. A denial of service detector based on maximum likelihood detection and the random neural network. The Computer Journal 50 6 (2007) 717-727
12. S. Timotheou, The random neural network: a survey, The Computer Journal (2009) doi:10.1093/comjnl/bxp032.
13. Gelenbe E. Product-form queueing networks with negative and positive customers. Journal of Applied Probability 28 3 (September 1991) 656-663
14. Artalejo J.R. G-networks: a versatile approach for work removal in queueing networks. European Journal of Operational Research 126 2 (2000) 233-249
15. Gelenbe E. Steady-state solution of probabilistic gene regulatory networks. Physical Review E 76 1 (2007) 031903
16. Gelenbe E., and Timotheou S. Random neural networks with synchronised interactions. Neural Computation 20 9 (2008) 2308-2324
17. Gelenbe E., and Timotheou S. Synchronised interactions in spiked neuronal networks. The Computer Journal 51 6 (2008) 723-730
18. Thimm G., and Fiesler E. High-order and multilayer perceptron initialization. IEEE Transactions on Neural Networks 8 2 (1997) 349-359
19. Wessels L., and Barnard E. Avoiding false local minima by proper initialization of connections. IEEE Transactions on Neural Networks 3 6 (1992) 899-905
20. O. Fontenla-Romero, D. Erdogmus, J.C. Principe, A. Alonso-Betanzos, E. Castillo, Linear least-squares based methods for neural networks learning, in: Artificial Neural Networks and Neural Information Processing, 2003, pp. 84-91.
21. Yam Y.F., Chow T.W.S., and Leung C.T. A new method in determining initial weights of feedforward neural networks for training enhancement. Neurocomputing 16 1 (1997) 23-32
22. E. Gelenbe, K. Hussain, H. Abdelbaki, Random neural network texture model, in: Conference on Applications of Artificial Neural Networks in Image Processing V, San Jose, CA, USA, 2000, pp. 104-111.
23. E. Gelenbe, A. Stafylopatis, A. Likas, Associative memory operations of the random neural network, in: Proceedings of the International Conference on Artificial Neural Networks, Helsinki, Finland, 1991, pp. 307-312.
24. J. Fourneau, Computing the steady state distribution of networks with positive and negative customers, in: Proceedings of 13 IMACS World Congress on Computation and Applied Mathematics, Dublin, 1991.
25. Lawson C.L., and Hanson R.J. Solving Least Squares Problems (1987), Prentice-Hall, Englewood Cliffs, NJ
26. Dax A. On computational aspects of bounded linear least squares problems. ACM Transactions on Mathematical Software 17 1 (1991) 64-73
27. Bro R., and Long S.D. A fast non-negativity-constrained least squares algorithm. Journal of Chemometrics 11 5 (1997) 393-401
28. D. Kim, S. Sra, I. Dhillon, A new projected quasi-Newton approach for the nonnegative least squares problem, Technical Report, Department of Computer Sciences, The University of Texas at Austin, December 2006.
29. Lin C.-J. Projected gradient methods for nonnegative matrix factorization. Neural Computation 19 10 (2007) 2756-2779
30. Bertsekas D. Nonlinear Programming (1995), Athena Scientific