Partially pre-calculated weights for the backpropagation learning regime and high accuracy function mapping using continuous input RAM-based sigma-pi nets

Neural Networks

Volumn 13, Issue 1, 2000, Pages 91-110

  Neville, R S ^a   Stonham, T J ^b   Glover, R J ^b  

^a LONDON SOUTH BANK UNIVERSITY   (United Kingdom)

^b BRUNEL UNIVERSITY   (United Kingdom)

Author keywords

Backpropagation; Higher order; n Tuple; Neural networks; RAM nets; Sigma pi; Training

Indexed keywords

BACKPROPAGATION; ERROR ANALYSIS; INTEGRATION; LEARNING SYSTEMS; MATHEMATICAL MODELS; RANDOM ACCESS STORAGE; VECTORS;

BACKPROPAGATION LEARNING REGIME; MEAN SQUARE ERROR (MSE); TIME INTEGRATION NODES (TIN);

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER; LEARNING; MATHEMATICAL MODEL; MEMORY; METHODOLOGY; MODEL; PRIORITY JOURNAL; TECHNIQUE; TRAINING;

ARTIFICIAL INTELLIGENCE; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER);

EID: 0033980299     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(99)00102-1     Document Type: Article

References (68)

1. Aleksander, I. (1989a). Canonical neural nets based on logic nodes. In: First IEE International Conference on Artificial Neural Networks (pp. 110-114).
2. Aleksander, I. (1989b). Neural computing architectures: the design of brain-like machines. In I. Aleksander (Ed.), North Oxford Academic Publishers, ISBN 0-946536-47-3, re: The logic of connectionist systems, chap. 8 (pp. 133-155) and A probabilistic logic neuron network for associative learning, chap. 9 (pp. 156-171) (see also chapters 1, 10 and 11).
3. Aleksander, I. (1989b). Neural computing architectures: the design of brain-like machines. In I. Aleksander (Ed.), North Oxford Academic Publishers, ISBN 0-946536-47-3, re: The logic of connectionist systems, chap. 8 (pp. 133-155) and A probabilistic logic neuron network for associative learning, chap. 9 (pp. 156-171) (see also chapters 1, 10 and 11).
4. Aleksander, I. (1990). Weightless neural tools: towards cognitive macrostructures. In: CAIP Neural Network Workshop, Rutgers University, New Jersey.
5. Aleksander I., Stonham T.J. Guide to pattern recognition using random-access memories. Computers and Digital Techniques. 2:1979;29-40.
6. Aleksander, I., Thomas, W.W., & Bowden, P.A. (1984). WISARD: a radical step forward in image recognition. Sensor Review, July, 120-124.
7. Austin J. Grey scale tuple processing. Kittler J. Pattern Recognition: Fourth International Conference. 1988;110-120 Springer, Berlin.
8. Austin J. A review of RAM based neural networks. Fourth International Conference on Microelectronics for Neural Networks and Fuzzy Systems, September 26-28, Turin, Italy. 1994;58-66. ISBN 0-8186-6710-9.
9. Barto A., Sutton R., Anderson C. Neuron like adaptive elements that can solve difficult learning problems. IEEE Transactions on Systems, Man, and Cybernetics. SMC-13:(5):1983;834-846.
10. Bledsoe, W.W., & Browning, I. (1959). Pattern recognition and reading machines. In: Proceedings of the Eastern Joint Computer Conference (pp. 225-232).
11. Bledsoe W.W., Blisson C.L. Improved memory matrices for the n-tuple pattern recognition method. IRE Transactions on Electronic Computers. ED11:1962;4141-4415.
12. Clarkson T., Gorse D., Taylor J., Ng C. Learning probabilistic RAM nets using VLSI structures. IEEE Transactions on Computers. 1992;1552-1561.
13. Clarkson T., Ng C.K., Guan Y. The pRAM: an adaptive VLSI chip. IEEE Transactions on Neural Networks. 4:(3):1993;408-412.
14. Crane, H.D. (1962). Neuristor - a novel device and system concept. In: Proceedings of the Institution of Radio Engineers (IRE), October (pp. 2048-2060).
15. Dedenham R.M., Gart S.C.J. Investigation into the effect of numerical resolution on the performance of back propagation. Personnaz L., Dreyfus G. Neural networks from models to applications. 1989;752-755 IDSET, Paris.
16. Ferguson, A. (1995). Learning in RAM-based artificial neural networks. PhD Thesis, Department of Electrical Engineering, University of Hertfordshire, Hatfield Campus, College Lane, Hatfield, Herts, UK.
17. Ferguson A., Dixon L.C., Bolouri H. Learning algorithms for RAM-based neural networks. Annals of Mathematics & Artificial Intelligence. 1999;. in press.
18. Goldstone J.S., Myers C.E. Traffic monitoring with WISARD and probilistic logic nodes. Aleksander I. Artificial neural networks 2. 1991;1669-1672 North-Holland, Brighton, UK.
19. Gorse D., Taylor J.G. On the equivalence and properties of noisy neural and probablistic RAM nets. Physics Letters A. 131:(6):1988;326-332.
20. Gorse, D., & Taylor, J.G. (1990a). Reinforcement training strategies for probabilistic RAMs, theoretical aspects of neurocomputing. Symposium on Neural Networks and Neurocomputing (pp. 180-184).
21. Gorse D., Taylor J. Training strategies for probabilistic RAMs. Parallel processing in neural systems and computers. 1990;161-164 North-Holland, Amsterdam.
22. Gorse D., Taylor J.G. A general model of stochastic neural processing. Biological Cybernetics. 63:1990;299-306.
23. Gorse D., Taylor J. A continuous input RAM-based stochastic neural model. Neural Networks. 4:1991;657-665.
24. Gorse D., Romano-Critchley D.A. Learning real-valued univariate time series: an application of a pulse-based reinforcement algorithm for learning continuous functions using pRAMs. Bisset D. WNNW'95, Weightless Neural Network Workshop 1995: Computing with Logical Neurons, 26-27 September. 1995;University of Kent, Canterbury, UK.
25. Gorse, D., Taylor, J., & Clarkson, T. (1993). Learning real-valued functions using hardware-implementable stochastic reinforcement algorithm. WCNN-93-PORTLAND, World Congress on Neural Networks, July 11-15, Oregon Convention Centre, Portland, Oregon, USA, Vol. III (pp. III.301-III.304).
26. Gorse, D., Taylor, J., & Clarkson, T. (1994). A pulse-based reinforcement algorithm for learning continuous functions. World Congress on Neural Networks - San Diego, 1994 International Neural Networks Society Annual Meeting, Town & County Hotel, San Diego, CA, USA, June 5-9, Vol. 2, ISBN 0-8058-1745-X (pp. II.73-II.78).
27. Grozinger, M. (1990). Feasibility of neurocomputing on the associative string processor. Masters Thesis, Department of Electrical Engineering, Brunel University, Uxbridge, UK.
28. Guan Y., Clarkson T.G., Taylor J.G., Gorse D. Noisy reinforcement training for pRAM nets. Neural Networks. 7:(3):1994;523-538.
29. Gullapalli, V. (1988). A stochastic algorithm for learning real-valued functions via reinforcement feedback. COINS Technical Report Number: 88-91, 29 September, The Department of Computer and Information Sciences, The University of Massachusetts, USA.
30. Gullapalli V. A stochastic reinforcement learning algorithm for learning real-valued functions. Neural Networks. 3:(6):1990;671-692.
31. Gurney, K. (1989). Learning in nets of structured hypercubes. PhD Thesis, Department of Electrical Engineering, Brunel University, Middlesex, UK, available as: Technical Memo CN/R/144.
32. Gurney K.N. Training nets of hardware realisable sigma-pi units. Neural Networks. 5:1992;289-303.
33. Gurney K. Weighted nodes and RAM-nets: a unified approach. Journal of Intelligent Systems. 2:(1-4):1992;155-185.
34. Gurney K. Training nets of stochastic units using system identification. Neural Networks. 6:(1):1993;133-145.
35. Kan, W.-K. (1987). A probabilistic logic neuron for associative learning. In Proceedings of the International Conference on Neural Networks, IEEE, ICCN'87, San Diego, CA, USA (pp. 541-548).
36. Kolcz A., Allinson N.M. Application of the CMAC input encoding scheme in the N-tuple approximation net. IEE Proceedings of Computers and Digital Technology. 141:(3):1994;177-183.
37. Krikelis, A., & Grozinger, M. (1990). Implementing neural networks with the associative string processor. In International Workshop on VLSI for Artificial Intelligence and Neural Networks, Oxford University, UK, 5-7 September.
38. Lenze B. How to make sigma-pi neural networks perform perfectly on regular training sets. Neural Networks. 7:(8):1994;1285-1294.
39. Milligan, D.K. (1988). Annealing in RAM-based learning networks. Brunel University Technical Memorandum CN/R/142, Middlesex, UK.
40. Morgan, P., Ferguson, A., Dixon, L.C.W., & Bolouri, A. (1995). Hardware implementation of a real-valued sigma-pi network. ENNS International Conference on Artificial Neural Networks (ICANN'95), Paris, accepted for presentation.
41. Myers, C. (1989). Output functions for probabilistic logic nodes. In First IEE International Artificial Neural Networks, UK.
42. Myers, C. (1990). Learning with delayed reinforcement in an exploratory probabilistic logic neural network. PhD Thesis, Department of Electrical Engineering, Imperial College of Science, The University of London.
43. Nagumo, J., Arimoto, S., & Yoshizawa, S. (1962). An active pulse transmission line simulating nerve axon. In: Proceedings of the Institution of Radio Engineers (IRE), October (pp. 2061-2070).
44. Neville, R.S. (1990a). Am2909 microprogram address sequencer. MSc Full Custom Design Assignment Thesis, Spring Term, Assignment Thesis for Microelectronics Systems Design MSc, Brunel University, UK.
45. Neville, R.S. (1990b). Investigate and evaluate the design of probabilistic nodes for Boolean n-cube networks. MSc Thesis, Department of Electrical Engineering, Brunel University, UK.
46. Neville, R. (1993). Augmentation of sigma-pi structures and learning regimes. PhD Thesis, Department of Electrical Engineering, Brunel University, Middlesex, UK.
47. Neville R.S., Stonham T.J. Adaptive reward-penalty for probabilistic logic nodes. Aleksander I. International Conference on Artificial Neural Networks, ICANN'92, 4-7 September. 1992;North-Holland, Brighton, UK.
48. Neville, R., & Stonham, T. (1993a). Adaptive Critic for probabilistic logic nets. In: Proceedings of the World Congress on Neural Networks, Portland, Oregon, USA (pp. III.389-III.392).
49. Neville, R., & Stonham, J. (1993b). Augmentation of generalisation of probabilistic logic nets. In: The World Congress on Neural Networks, WCNN-93, Portland, Oregon, USA (pp. IV.198-IV.201).
50. Neville R.S., Stonham T.J. Adaptive associative reward-penalty algorithms for sigma-pi networks. Neural, Parallel & Scientific Computations. 2:(2):1994;141-164.
51. Neville, R., & Stonham, J. (1994b). Unbounded reinforcement for the associative reward-penalty algorithm. In: World Congress on Neural Networks, WCNN-94, San Diego, CA, USA (pp. III.637-III.640).
52. Neville, R., & Stonham, J. (1994c). A comparison study of unbounded and real-valued reinforcement associative reward-penalty algorithms. In: International Conference on Artificial Neural Networks, ICANN-94, Sorrento, Italy (pp. 651-654).
53. Neville R.S., Stonham T.J. Generalisation in sigma-pi networks. Connection Science. 7:(1):1995;141-164.
54. Neville R.S., Stonham T.J. Adaptive critic for sigma-pi networks. Neural Networks. 9:(4):1996;603-625.
55. Neville, R.S., Glover, R.J., & Stonham, T.J. (1995a). Evaluation of training sigma-pi networks on massively parallel processor. In: World Congress on Neural Networks, WCNN'95, Renaissance Hotel, Washington, DC, USA, 17-21 July (pp II.484-II.487).
56. Neville, R., Glover, R.J., & Stonham, J. (1995b). Evaluation of training & mapping sigma-pi networks to a massively parallel processor. In: IEEE ICNN'95, International Conference on Neural Networks, Perth, Western Australia, 27 November-1 December (pp. 1042-1047).
57. Neville, R., Glover, R.J., & Stonham, J. (1995c). Mapping Neural Networks to the Associative String Processor, Department of Electrical Engineering, Brunel University, UK. Registered as Technical Memorandum N/R/150 in the Department of Electrical Engineering, Brunel University, Copyright October.
58. Neville, R., Glover, R.J., & Stonham, J. (1995d). Mapping sigma-pi networks and the associative reward-penalty training regime to a massively parallel processor. IEEE Transactions on Neural Networks. Submitted for publication.
59. Neville, R., Glover, R.J., & Stonham, J. (1995e). Mapping sigma-pi networks and the associative reward-penalty training regime to the associative string processor. IEEE Transactions on Massively Parallel Processors/Associative Processors (Special Issue). Submitted for publication.
60. Rumelhart D.E., Hinton G.E., Williams G.E. Learning internal representations by error propagation. Parallel distributed processing. 1:1986;MIT Press, Cambridge, MA.
61. Werbos P.J. The elements of intelligence. Cybernetica. 3:1968;000.
62. Werbos, P.J. (1974). Beyond regression: new tools for predictions and analysis in the behavioural sciences. PhD Dissertation, Committee on Applied Mathematics, Harvard University, Cambridge, MA, November.
63. Werbos, P. (1989). Back-propagation and neurocontrol: a review and prospectus. IEEE Proceedings of the International Joint Conference on Neural Networks (IJCNN'89), New York, USA, vol. 1 (pp. I.209-I.216).
64. Werbos P.J. Neurocontrol and fuzzy logic: connections and designs. International Journal of Approximate Reasoning. 6:1992;185-219.
65. Werbos P.J. The roots of backpropagation: from ordered derivatives to neural networks and political forecasting. 1994;Wiley, New York.
66. Weste N.H.E., Eshraghian K. Principles of VLSI design: a systems perspective. Conway L., Seitz C. VLSI Systems Series. 1988;Addison-Wesley, Reading, MA.
67. Wilkie, B.A. (1983). A stand-alone, high resolution, adaptive pattern recognition system. PhD Thesis W499, Department of Electrical Engineering, Brunel University, Middlesex, UK.
68. Zhang B., Zhang L., Zhang H. A quantitative analysis of the behaviours of the PLN network. Neural Networks. 5:(4):1992;639-644.