Discrete all-positive multilayer perceptrons for optical implementation

Optical Engineering

Volumn 37, Issue 4, 1998, Pages 1305-1315

  Moerland, Perry D ^a,c   Fiesler, Emile ^b   Saxena, Indu ^b  

^a IDIAP RESEARCH INSTITUTE   (Switzerland)

^b PHYSICAL OPTICS CORPORATION   (United States)

^c EINDHOVEN UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Activation function; Backpropagation; Liquid crystal light valve; Neural network; Non negative neural networks; Optical multilayer perceptron; Weight discretization

Indexed keywords

EID: 0006058819     PISSN: 00913286     EISSN: None     Source Type: Journal    
DOI: 10.1117/1.601963     Document Type: Article

References (30)

1. I. Saxena and P. Horan, "Optical implementations," Chap. El.3 in The Handbook of Neural Computation, E. Fiesler and R. Beale, Eds., Institute of Physics Publishers & Oxford University Press, New York (1997).
2. F. Yu, T. Lu, X. Yang, and D. Gregory, "Optical neural network with pocket-sized liquid-crystal televisions," Opt. Lett. 15(15), 863-865 (1990).
3. J.-S. Jang, S.-G. Shin, S.-W Yuk, S.-Y. Shin, and S.-Y. Lee, "Dymanic optical interconnections using holographic lenslet arrays for adaptive neural networks," Opt. Eng. 32(1), 80-87 (1993).
4. R. G. Stearns, "Optically programmed neural network capable of stand-alone operation," Appl. Opt. 32(26), 5141-5152 (1993).
5. I. Saxena and E. Fiesler, "Adaptive multilayer optical neural network with optical thresholding," Opt. Eng. 34(8), 2435-2440 (1995).
6. N. Collings, "Design considerations for a useful two-layer neural network," in Euro-American Workshop on Optical Pattern Recognition, B. Javidi and P. Refregier, Eds. PM12, 314-333, SPIE Press (1994).
7. N. Collings, A. R. Pourzand, and R. Volkel, "Construction of a programmable multilayer analogue neural network using space invariant interconnects," Proc. SPIE 2565, 40-47 (1995).
8. P. Moerland, E. Fiesler, and I. Saxena, "Incorporation of liquidcrystal light valve non-linearities in optical multilayer neural networks," Appl. Opt. 35(26), 5301-5307 (1996).
9. D. Psaltis and Y. Qiao, "Adaptive multilayer optical networks," in Progress in Optics, E. Wolf, Ed., Vol. 31, pp. 227-261, Elsevier Science Publishers, Amsterdam (1993).
10. W. Xue, "Characterization of liquid crystal light valves for neural network applications," PhD Thesis, Institute of Micro-technology, University of Neuchâtel, Switzerland (1994).
11. N. Collings and W. Xue, "Liquid crystal light valves as thresholding elements in neural networks: basic device requirements," Appl. Opt. 33(14), 2829-2833 (1994).
12. D. Rumelhart, G. Hinton, and R. Williams, "Learning internal representations by error propagation," Chap. 8 in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1: Foundations, pp. 318-362, MIT Press, Cambridge (1986).
13. G. Thimm and E. Fiesler, "Weight initialization in higher order and multi-layer perceptrons," IEEE Trans. Neural Net. 8(2), 349-359 (1997).
14. G. Thimm, P. Moerland, and E. Fiesler, "The interchangeability of learning rate and gain in backpropagation neural networks," Neural Comput. 8(2), 451-460 (1996).
15. E. Fiesler, A. Choudry, and H. J. Caulfield, "A weight discretization paradigm for optical neural networks," in Proc. SPIE 1281, 164-173 (1990).
16. I. Shariv, O. Gila, and A. A. Friesem, "All-optical bipolar neural network with polarization-modulating neurons," Opt. Lett. 16(21), 1692-1694 (1991).
17. N. Kasama, Y. Hayasaki, T. Yatagai, M. Mori, and S. Ishihara, "Experimental demonstration of optical three layer neural network," Jpn. J. Appl. Phys. 29(8), L1565-L1568 (1990).
18. M. Kranzdorf, B. Bibner, L. Zhang, and K. Johnson, "Optical connectionist machine with polarization-based bipolar weight values," Opt. Eng. 28, 844-848 (1989).
19. I. Saxena, E. Fiesler, and P. Moerland, "A method for all-positive optical multilayer perceptrons," in Proc. 3rd IEEE Int. Conf. on Electronics, Circuits, and Systems (ICECS'96), Vol. 1, pp. 448-451, Piscataway, NJ (1996).
20. R. C. Frye, E. A. Rietman, and C. C. Wong, "Back-propagation learning and nonidealities in analog neural network hardware," IEEE Trans. Neural Net., 2(1), 110-117 (1991).
21. M. G. Robinson and K. M. Johnson, "Noise analysis of polarization-based optoelectronic connectionist machines," Appl. Opt. 31(2), 263-272 (1992).
22. R. P. Gorman and T. J. Sejnowki, "Analysis of hidden units in a layered network trained to classify sonar targets," Neural Net. 1, 75-89 (1988).
23. Data made available in 1994 by librarians P. M. Murphy and D. W. Aha from the UCI Repository of Machine Learning Databases, a machine-readable data repository accessible via anonymous-ftp: ftp://ftp.ics.uci.edu/pub/machine-learning-databases.
24. M. D. Garris and R. A. Wilkinson, NIST Special Database 3, National Institute of Standards and Technology, Advanced System Division, Image Recognition Group (1992).
25. S. E. Fahlman, "An empirical study of learning speed in backpropagation networks," Technical Report CMU-CS-88-162, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA (1988).
26. L. Prechelt, "PROBEN1 - a set of benchmarks and benchmarking rules for neural network training algorithms." Technical report 21/94, Facultät für Informatik, Universität Karlsruhe, D-76128 Karlsruhe, Germany (1994). The benchmark set is available at ftp://ftp.ira.uka.de/pub/neuron/proben.tar.gz.
27. M. Moreira and E. Fiesler, "Neural networks with adaptive learning rate and momentum terms," Technical Report 95-04, IDIAP, Martigny, Switzerland. Available via anonymous ftp: ftp://ftp.idiap.ch/pub/reports/1995/95-04.ps.Z.
28. T. Lundin and P. Moerland, "Quantization and pruning of multilayer perceptions: towards compact neural networks," IDIAP-Com 97-02, IDIAP, Martigny, Switzerland. Available via anomymous ftp: ftp:// ftp.idiap.ch/pub/reports/1997/com97-02.ps.gz.
29. I. Saxena, P. Moerland, E. Fiesler, A. R. Pourzand, and N. Collings, "An optical thresholding perceptron," in Proc. Workshop on Optics and Computer Science, Geneva, Switzerland (1997).
30. I. Saxena, P. Moerland, E. Fiesler, and A. Pourzand, "Handwritten digit recognition with binary optical perceptron," in Artificial Neural Networks - ICANN'97, Lecture Notes in Computer Science, Vol. 1327, W. Gerstner, A. Germond, M. Hasler, and J.-D. Nicoud, Eds., pp. 1253-1258, Springer Verlag, Berlin (1997).