On the possibilities of the limited precision weights neural networks in classification problems

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 1240 LNCS, Issue , 1997, Pages 753-762

  Draghici, Sorin ^a   Sethi, Ishwar K ^a  

^a Wayne State University   (United States)

Author keywords

Classification problems; Entropy; Integer weights; Neural networks; Number of bits

Indexed keywords

COMPUTATION THEORY; ENTROPY; INFORMATION THEORY;

EUCLIDIAN DISTANCE; HARDWARE IMPLEMENTATIONS; INTEGER WEIGHTS; NUMBER OF LAYERS; THEORETICAL POINTS; WEIGHT RANGE;

NEURAL NETWORKS;

EID: 84902205190     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/bfb0032534     Document Type: Conference Paper

References (32)

1. [Abu-Mostafa, 1988] Abu-Mostafa Y.S., Connectivity versus Entropy, NIPS'87, D.Z. Anderson (Ed.), Amer. Inst. Of Phys., New York, 1988, 1-8.
2. [Beiu, 1994] - Beiu V., Peperstraete J.A., Vandewalle J., Lauwereins R., Area-time performances of some neural computations, Int. Syrup. On Signal Proc., Robotics and NN's, P. Borne, T. Fukuda and S.G. Tzafestas (Eds.), GERF EC, Lille, 1994, pp. 664-668.
3. [Beiu, 1997a] - Beiu V. - Neural Networks Using Threshold Gates: A Complexity Analysis of Their Area and Time Efficient VLSI Representations, Ph.D. thesis, Katholieke Universiteit Leuven, 1994. Extended version to appear as "VLSI Complexity of Discrete Neural Networks", Gordon & Breach, 1997 (in press).
4. [Beiu, 1995] - Beiu, V., Optimal VLSI Implementations of Neural Networks, Chap. 18 in J.G. Taylor (ed.): Neural Networks and Their Applications, John Wiley & Sons, Chichester, UK, 255-276, 1996.
5. [Beiu, 1995a] - Beiu, V., Taylor, J.G., VLSI optimal neural network learning algorithm, Artif. NN's and Genetic Algs., D.W. Pearson, N.C. Steele and R.F. Albrecht (Eds.), Springer-Verlag, Vienna, 1995, pp. 61-64.
6. [Beiu, 1995b] Beiu, V., Taylor, J.G., Area efficient constructive learning algorithms, Proc. 10 th Intl.Conf. on Control Sys. and Comp. Set., PU Bucharest, Bucharest, 1995, pp. 293-310.
7. [Beiu, 1996] - Beiu, V., Entropy bounds for classification algorithms, Neural Network World, Vol. 6, No. 4, pp. 497-505, IDG Press, 1996.
8. [Beiu, 1997] - Beiu, V., T. De Pauw, Tight bounds on the size of neural networks for classification problems, submitted for IWANN'97.
9. [Bruck, 1990] Bruck J., Goodman J.W., On the power of neural networks for solving hard problems, NIPS'87, D.Z. Anderson (Ed.), Amer. Inst. Of Phys., NY, 1988, 137-143.
10. Bruck J., Goodman J.W J. of Complexity, 6, 1990, 129-135.
11. [Coggins, 1994] Coggins R., M. Jabri, Wattle: A Trainable Gain Analogue VLSI Neural Network, Advances in NIPS 6 (NIPS*93, Denver, CO), Morgan Kaufman, San Mateo, CA, 874-881, 1994.
12. [Denker, 1988] Denker J.S., Wittner B.S., Network Generality, Training Required and Precision Required, NIPS'88, D.Z. Anderson (Ed.), Amer. Inst of Phys., New York, 1988, 219-222.
13. [Dundar, 1995] Dundar G., K. Rose, The Effect of Quantization on Multilayer Neural Networks, IEEE Transactions on Neural Networks 6(6), pp. 1446-1451, 1995.
14. [Hammerstrom, 1988] - Hammerstrom D., The connectivity analysis of simple associations - or - How many connections do you need?, NIPS'87, D.Z. Anderson (Ed.), Amer. Inst. Of Phys., New York, 1988, 338-347.
15. [Hohfeld, 1991a] - Hohfeld M., S.E. Fahlman, Learning with limited numerical precision using the Cascade-Correlation Algorithm, Tech.Rep. CMU-CS-91-130, School of Comp. Sci. Carnegie Mellon, May 1991.
16. Hohfeld M., S.E. Fahlman in IEEE Transactions on Neural Networks, 3(4), 602- 611, 1992.
17. [Hohfeld, 1991b] - HohfeId M., S.E. Fahlman, Probabilistic rounding in neural networks with limited precision. In U. Ruckert and J.A. Nossek (eds.): Microelectronics for Neural Networks (Proc. MicroNeuro'91 - Munich, Germany), Kyrill & Method Verlag, 1-8, October 1991.
18. Neurocomputing, 4, 291-299, 1992.
19. [Khan, 1994] - Khan A.H., E.L. Hines, Integer weight neural networks, Electronics Letters, 30(15), pp. 1237-1238, 1994.
20. [Khan, 1996] Khan A.H., R.G. Wilson, Integer weight approximation of continuous-weight multilayer feedforward nets, Proc. IEEE Int. Conf. on Neural Networks, vol. 1, pp. 392-397, Washington DC, June 1996, IEEE Press, New York, NY.
21. [Klaggers, 1993] - Klaggers H., Soegtrop M., Limited fan-in random wired cascade-correlation learning architecture, MicroNeuro'93, D.J. Myers and A.F.Murray (Eds.), Univ.Ed Tech. Ltd. Edinburgh, 1993, pp. 79-82.
22. [Kwan, 1992 ]- Kwan H.K., Tang C.Z., Designing Multilayer Feedforward Neural Networks Using Simplified Activation Functions and One-Power-of-Two Weights. Electronic Letters, 28(25), pp. 2343-2344, 1992.
23. [Kwan, 1993] - Kwan H.K., Tang C.Z., Multiplierless Multilayer Feedforward Neural Networks Design Suitable for Continuous Input-Output Mapping, Electronic Letters, 29(14), pp. 1259-1260, 1993.
24. [Mason, I995] - Mason R.D., Robertson W., Mapping Hierarchical Neural Networks to VLSI Hardware, Neural Networks, vol. 8, 6, 1995, 905-913.
25. [Marchesi, 1990] Marchesi M., G. Orlandi, F. Piazza, L. Pollonara, A. Uncini, Multilayer Perceptrons with Discrete Weights, Proc. Int. Joint Conf. on Neural Networks IJCNN'90, San Diego, Vol. 2, pp. 623-630, June 1990.
26. [Marchesi, 1990] Marchesi M., G. Orlandi, F. Piazza, A. Uncini, Fast Neural Networks without Multipliers, IEEE Transactions on Neural Networks, 4(1), pp. 53-62, 1993.
27. [Phatak, 1994] - Phatak D.S., Koren I., Connectivity and performance tradeoffs in the cascadecorrelation learning architecture, IEEE Trans. NN's, 5, 6, 1994, 930-935.
28. [Tang, 1993] Tang C.Z., H.K. Kwan, Multilayer Feedforward Neural Networks with Single Power-of-Two Weights. IEEE Trans. On Signal Processing, 41(8), 2724-2727, 1993.
29. [Vincent, 1992] Vincent J.M., D.J.Myers, Weight Dithering and Wordlength Selection for Digital Backpropagation Networks, BT TechnoIogy J., 10(3), pp. 124-133, 1992.
30. [Williamson, 1991] Williamson R.C., Entropy and the complexity of feedforward neural networks, NIPS'90, R.P. Lippmann, J.E.Moody and D.S. Touretzky (Eds.), Morgan Kaufmann, San Mateo, 1991, pp. 946-952.
31. [Xie, 1991] - Xie Y., M.A. Jabri, Training Algorithms for Limited Precision Feedforward Neural Networks, SEDAL TR 1991-8-3, School of EE, University of Sydney, Australia, 1991.
32. Proc. of the Australian Conference on Neural Networks, Canberra, Australia, 67-71, 1992.