Stochastic correlative learning algorithms

IEEE Transactions on Signal Processing

Volumn 52, Issue 8, 2004, Pages 2200-2209

  Haykin, Simon ^a   Chen, Zhe ^a   Becker, Suzanna ^a  

^a MCMASTER UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL COMPLEXITY; COMPUTER SIMULATION; CORRELATION METHODS; DATA REDUCTION; FEATURE EXTRACTION; IDENTIFICATION (CONTROL SYSTEMS); LEARNING ALGORITHMS; MONTE CARLO METHODS; NEUROLOGY; RANDOM PROCESSES; ROBOTIC ARMS; STEREO VISION;

BINOCULAR FUSION; FINANCIAL DATA PREDICTION; PARTICLE FILTERING; SEQUENTIAL MONTE CARLO ESTIMATION; STOCHASTIC CORRELATIVE LEARNING ALGORITHMS;

SIGNAL FILTERING AND PREDICTION;

EID: 3543138291     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2004.831067     Document Type: Article

References (52)

1. J. A. Anderson, "A memory storage model utilizing spatial correlation functions," Kybernetik, vol. 5, no. 3, pp. 113-119, 1969.
2. J. A. Anderson, "A simple neural network generating an interactive memory," Math. Biosci., vol. 14, pp. 197-220, 1972.
3. M. J. Anderson and E. Tzanakou, "Auditory stimulus optimization with feedback from fuzzy clustering of neuronal responses," IEEE Trans. Inform. Technol. Biomed., vol. 6, pp. 159-169, June 2002.
4. J. J. Atick, "Can information theory provide an ecological theory of sensory processing?," Network, vol. 3, pp. 213-251, 1992.
5. A. J. Bell and T. J. Sejnowski, "An information-maximization approach to blind separation and blind deconvolution," Neural Comput., vol. 7, pp. 1129-1159, 1995.
6. A. Bia, "Alopex-B: a new, simple, but yet faster version of the Alopex training algorithm," Int. J. Neural Syst., vol. 11, no. 6, pp. 497-507, 2001.
7. F. Black and M. Scholes, "The pricing of options and corporate liabilities," J. Political Econ., vol. 81, pp. 637-659, 1973.
8. B. S. Blais, N. Intrator, H. Shouval, and L. N. Cooper, "Receptive field formation in natural scene environments: comparison of single cell learning rules," Neural Comput., vol. 10, pp. 1797-1813, 1998.
9. T. Briegel and V. Tresp, "Fisher scoring and a mixture of modes approach for approximate inference and learning in nonlinear state space models," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1999, vol. 11.
10. Z. Chen, "Bayesian filtering: From Kalman filters to particle filters, and beyond," Tech. Rep., Adaptive Syst. Lab., McMaster Univ., Hamilton, ON, Canada, 2003.
11. Z. Chen, S. Haykin, and S. Becker, "Sampling-Based Alopex Algorithms for Neural Networks and Optimization," Tech. Rep., Adaptive Syst. Lab., McMaster Univ., Hamilton, ON, Canada, 2003.
12. Z. Chen, S. Haykin, and S. Becker, "Theory of Monte Carlo sampling-based Alopex algorithms for neural networks," in Proc. ICASSP, Montreal, QC, Canada, 2004.
13. J. F. G. de Freitas, "Bayesian Methods for Neural Networks," Ph.D. dissertation, Dept. Eng., Univ. Cambridge, Cambridge, U.K., 1998.
14. J. F. G. de Freitas, M. Niranjan, A. H. Gee, and A. Doucet, "Sequential Monte Carlo methods to train neural network models," Neural Comput., vol. 12, no. 4, pp. 955-993, 2000.
15. A. Doucet, N. de Freitas, and N. Gordon, Eds., Sequential Monte Carlo Methods in Practice. New York: Springer, 2001.
16. J. J. Eggermont, The Correlative Brain: Theory and Experiment in Neural Interaction. New York: Springer-Verlag, 1990.
17. D. J. Field, "What is the goal of sensory coding?," Neural Comput., vol. 6, pp. 559-601, 1994.
18. O. Fujita, "Trial-and-error correlation learning," IEEE Trans. Neural Networks, vol. 4, pp. 720-722, July 1993.
19. S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images," IEEE Trans. Pattern Anal. Machine Intell., vol. 6, pp. 721-741, 1984.
20. J. L. Hamilton, E. Tzanakou, and R. M. Lehman, "Neural networks trained with simulation data for outcome prediction in pallidotomy for Parkinson's disease," in Proc. 22nd IEEE Engr. Med. Biol., vol. 1, 2000, pp. 1-4.
21. E. Harth and E. Tzanakou, "Alopex: a stochastic method for determining visual receptive fields," Vis. Res., vol. 14, pp. 1475-1482, 1974.
22. E. Harth, K. P. Unnikrishnan, and A. S. Pandya, "The inversion of sensory processing by feedback pathways: a model of visual cognitive functions," Science, vol. 237, pp. 184-187, 1987.
23. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.
24. S. Haykin, Z. Chen, and S. Becker, "Correlation: Novel basis for statistical learning algorithms," Tech. Rep., Adaptive Syst. Lab., McMaster Univ., Hamilton, ON, Canada, 2003. [Online]: http://soma.crl.mcmaster.ca/~zhechen/download/ieee-corr.pdf.
25. D. O. Hebb, The Organization of Behavior: A Neuropsychological Theory. New York: Wiley, 1949.
26. J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," in Proc. Nat. Acad. Sci. USA, vol. 79, 1982, pp. 2554-2558.
27. J. M. Hutchinson, "A radial basis function approach to financial time series analysis," Ph.D. dissertation, Mass. Inst. Technol., Cambridge, MA, 1994.
28. J. M. Hutchinson, A. W. Lo, and T. Poggio, "A nonparametric approach to pricing and hedging derivative securities via learning networks," J. Finance, vol. 49, no. 3, pp. 851-889, 1994.
29. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York: Wiley, 2001.
30. W. James, Psychology (Briefer Course). New York: Holt, 1890, ch. XVI.
31. B. Julesz, Foundations of Cyclopean Perception. Chicago, IL: Univ. Chicago Press.
32. S. Kirkpatrick, C. G. Gelatt, and M. P. Vecchi, "Optimization by simulated annealing," Science, vol. 220, pp. 671-680, 1983.
33. T. Kohonen, "Correlation matrix memories," IEEE Trans. Comput., vol. C-21, pp. 353-359, 1972.
34. T. M. Liggett, Interacting Particle Systems. New York: Springer-Verlag, 1985.
35. J. T. Lo and D. Bassu, "Adaptive multilayer perceptrons with long-and short-term memories," IEEE Trans. Neural Networks, vol. 13, pp. 22-33, Jan. 2001.
36. D. J. C. MacKay, "Introduction to Monte Carlo methods," in Learning in Graphical Models, M. I. Jordan, Ed. Cambridge, MA: MIT Press, 1999.
37. D. Marr and T. Poggio, "Cooperative computation of stereo disparity," Science, vol. 194, pp. 283-287, 1976.
38. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, "Equations of state calculations by fast computing machines," J. Chem. Phys., vol. 21, pp. 1087-1091, 1953.
39. K. Nakano, "Associatron - a model of associative memory," IEEE Trans. Syst., Man, Cybern., vol. SMC-2, no. 3, pp. 380-388, 1972.
40. E. Oja, "A simplified neuron model as a principal component analyzer," J. Math. Biol., vol. 15, pp. 267-273, 1982.
41. E. Oja, "Neural networks, principal components, and subspaces," Int. J. Neural Syst., vol. 1, pp. 61-68, 1989.
42. T. D. Sanger, "Optimal unsupervised learning in a single-layer linear feedforward neural network," Neural Networks, vol. 2, no. 6, pp. 459-473, 1989.
43. P. S. Sastry, M. Magesh, and K. P. Unnikrishnan, "Two timescale analysis of Alopex algorithm for optimization," Neural Comput., vol. 14, no. 11, pp. 2729-2750, 2002.
44. S. Shah and P. S. Sastry, "New algorithms for learning and pruning oblique decision trees," IEEE Trans. Syst., Man, Cybern. C, vol. 29, pp. 494-505, Nov. 1999.
45. E. Tzanakou, R. Michalak, and E. Harth, "The Alopex process: visual receptive fields by response feedback," Biol. Cybern., vol. 35, pp. 161-174, 1979.
46. E. Tzanakou, "Neural networks in signal and image processing," in Proc. IEEE ELECTRO, 1996, pp. 337-346.
47. E. Tzanakou, Supervised and Unsupervised Pattern Recognition: Feature Extraction and Computational Intelligence. Roca Raton, FL: CRC, 2000.
48. K. P. Unnikrishnan and K. P. Venugopal, "Alopex: a correlation-based learning algorithm for feedforward and recurrent neural networks," Neural Comput., vol. 6, pp. 469-490, 1994.
49. R. van der Merwe, J. F. G. de Freitas, A. Doucet, and E. Wan, "The unscented particle filter," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2001, vol. 13.
50. K. P. Venugopal, A. S. Pandya, and R. Sundhakar, "A recurrent network controller and learning algorithm for the on-line learning control of autonomous underwater vehicles," Neural Networks, vol. 7, no. 5, pp. 833-846, 1994.
51. E. Wan and R. van der Merwe, "The unscented Kalman filter," in Kalman Filtering and Neural Networks, S. Haykin, Ed. New York: Wiley, 2001.
52. W. H. Wong and F. Liang, "Dynamic importance weighting in Monte Carlo and optimization," in Proc. Nat. Acad. Sci. USA, vol. 94, 1997, pp. 14 220-14 224.