On the capabilities of neural networks using limited precision weights

Neural Networks

Volumn 15, Issue 3, 2002, Pages 395-414

  Draghici, Sorin ^a  

^a Wayne State University   (United States)

Author keywords

Integer weights; VC complexity; VLSI implementation

Indexed keywords

ENTROPY; PRECISION BALANCES; PROBLEM SOLVING; STATISTICAL METHODS; VLSI CIRCUITS;

WORST-CASE ANALYSIS;

NEURAL NETWORKS;

ACCURACY; ANALYTIC METHOD; ARTICLE; ARTIFICIAL NEURAL NETWORK; ENTROPY; INFORMATION PROCESSING; MATHEMATICAL ANALYSIS; MATHEMATICAL COMPUTING; PRIORITY JOURNAL; PROBLEM SOLVING; STATISTICS; TRAINING; WEIGHT;

NEURAL NETWORKS (COMPUTER);

EID: 0035989190     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(02)00032-1     Document Type: Article

References (140)

1. Limited interconnectivity in synthetic neural systems
2. On lower bounds for the depth of thresh-old circuits with weights from (-1,0,+1)
3. Explicit constructions of depth-2 majority circuits for comparison and addition
4. Growing layers of perceptrons: Introducing the extentron algorithm
5. What size net gives good generalization?
6. On ACC
7. Digital integrated circuits implementations
8. Entropy bounds for classification algorithms
9. Constant fan-in digital neural networks are VLSI-optimal
10. On the circuit and VLSI complexity of threshhold gate comparison
11. High-speed noise robust threshold gates
12. Ultra-fast noise immune CMOS threshold gates
13. On higher order noise immune perceptrons
14. On VLSI-optimal neural computations
15. Limited weights neural networks: Very tight entropy based bounds
16. Deeper sparser nets can be optimal
17. On the circuit complexity of sigmoid feedforward neural networks
18. Simpler neural networks by fan-in reduction
19. On the circuit complexity of feedforward neural networks
20. On limited fan-in optimal neural networks
21. A constructive approach to calculating lower bounds
22. Harmonic analysis of polynomial threshold functions
23. On the power of neural networks for solving hard problems
24. Polynomial threshold functions. ACO functions, and spectral norms
25. A fast stochastic error-descent algorithm for supervised learning and optimisation
26. Neuromorphic learning VLSI systems: A survey
27. Constant depth reducibility
28. Approximation by superpositions of a sigmoidal function
29. Neural networks in analog hardware-design and implementation issues
30. The constraint based decomposition (CBD)
31. Adapting theoretical constructive algorithms to hardware implementations for classification problems
32. A VLSI optimal constructive algorithm for classification problems
33. The effect of quantization on multilayer neural networks
34. Cascade error projection: A learning algorithm for hardware implementation
35. Design of parallel hardware neural network systems from custom analog VLSI building block chips
36. A weight discretization paradigm for optical neural networks
37. Majority gates vs. general weighted threshold gates
38. Simulating threshold circuits by majority circuits
39. A reconfigurable CMOS neural network
40. VLSI implementation of a neural network memory with several hundreds of neurons
41. VLSI implementation of a neural network model
42. Capacity of neural networks with discrete synaptic couplings
43. Threshold circuits of bounded depth
44. The connectivity analysis of simple associations: How many connections do you need?
45. Almost optimal lower bounds for small depth circuits
46. On the power of small-depth threshold circuits
47. 0
48. Bounds on the sample complexity of bayesian learning using information theory and the VC dimension
49. Kolmogorov's mapping neural network existence theorem
50. Hardware implementation of neural networks in Japan
51. Learning with limited numerical precision using the Cascade-Correlation algorithm
52. Some notes on threshold circuits, and multiplication in depth 4
53. An electrically trainable artificial neural network (ETANN) with 10240 floating gate synapses
54. Artificial neural networks using MOS analog multiplier
55. A neural network learning algorithm tailored for VLSI implementation
56. The effects of precision constraints in a backpropagation learning network
57. On the node complexity of neural networks
58. Approximation capabilities of multiplayer feed forward networks
59. Multilayered feed forward networks are universal approximators
60. Universal approximation of an unknown mapping and its derivatives using multilayer feed forward networks
61. Size-depth tradeoffs for threshold circuits
62. Weight perturbation: An optimal architecture and learning technique for analog VLSI feed forward and recurrent multilayer networks
63. On the representation of continuous functions of many variables by superpositions of continuous functions of one variable and addition
64. Survey and current status of neural network hardware
65. Designing multilayer feedforward neural networks using simplified activation functions and one-power-of-two weights
66. Multiplierless multiplayer feedforward neural networks design suitable for continuous input-output mapping
67. Special issue on neuromorphic engineering
68. Connectionist quantization functions
69. Multilayer perceptrons with discrete weights
70. Fast neural networks without multipliers
71. On the power of networks of majority functions
72. On the precision constraints of threshold elements
73. Vapnik-Chervonenkis dimension of neural networks with binary weights
74. Linear input logic
75. Discrete all positive multilayer perceptrons for optical implementation
76. Generation of self-dual threshold functions and lower bound of the number of threshold functions and a maximum weight
77. Lower bounds on the number of threshold functions and a maximum weight
78. A digital multilayer neural network with limited binary expressions
79. Parallel computation with threshold functions
80. Learning in threshold networks
81. On small depth threshold circuits (invited)
82. Synthesis of threshold circuits for certain classes of boolean functions
83. An analysis on the performance of silicon implementations of backpropagation algorithms for artificial neural networks
84. Special issue on neural network hardware
85. Computation beyond the Turing limit
86. Backpropagation without multiplications
87. Neural computation of arithmetic functions
88. On the power of threshold circuits with small weights
89. Optimal depth neural networks for multiplication and related problems
90. On optimal depth threshold circuits for multiplication and related problems
91. Depth-size tradeoffs for neural computation
92. Computing with almost optimal size neural networks
93. On the structure of continuous functions of several variables
94. A universal mapping for Kolmogorov's superposition theorem
95. A numerical implementation of Kolmogorov's superpositions
96. A numerical implementation of Kolmogorov's superpositions II
97. Information theoretical approach to the storage capacity of neural networks with binary weights
98. New information theoretical approach to the storage capacity of neural networks with binary weights
99. A quantized backpropagation learning rule and its applications to optical neural networks
100. Multilayer feedforward neural networks with single power-of-two weights
101. On the uniform convergence of relative frequencies of events to their probabilities
102. Weight dithering and wordlength selection for digital backpropagation networks
103. Optimal lower bounds on the depth of polynomial-size threshold circuits for some arithmetic functions
104. 30 years of adaptive neural networks: perceptron, madaline and backpropagation
105. Bounds on threshold gate realizability