A Statistical Approach to Neural Networks for Pattern Recognition

A Statistical Approach to Neural Networks for Pattern Recognition

Volumn , Issue , 2006, Pages 1-268

  Dunne, Robert A ^a  

^a CSIRO   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84889332768     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9780470148150     Document Type: Book

References (260)

1. Adby, P. R. and Dempster, M. A. H. (1974). Introduction to Optimization Methods. Chapman and Hall, London.
2. Ahmed, S. and Tesauro, G. (1988). Scaling and generalization in neural networks: a case study. In Touretzky et al. (1988), pp. 3-9.
3. Aires, F., Prigent, C, and Rossow, W. B. (2004). Neural network uncertainty assessment using bayesian statistics: A remote sensing application. Neural Computation, 16(11):2415-2458.
4. Aitchison, J. and Dunsmore, I. R. (1975). Statistical Prediction Analysis. Cambridge University Press, Cambridge.
5. Almeida, L. and Wellekens, C, editors (1990). Lecture Notes in Computer Science 412, Neural Networks EURASIP Workshop Sesimbra, Portugal. Springer-Verlag, Berlin.
6. Alpaydin, E. (1991). Gal: Networks that grow when they learn and shrink when they forget. Preprint, International Computer Science Institute, Berkeley, CA.
7. Amari, S., Murata, N., Miiller, K.-R., Finke, M., and Yang, H. (1997). Asymptotic statistical theory of overtraining and cross-validation. IEEE Transactions on Neural Networks, 8(5). Previously, University of Tokyo Techreport: METR 95-06, 1995.
8. Anderson, J. A. and Rosenfeld, E., editors (1988). Neurocomputing: Foundations of Research. MIT Press, Cambridge, MA.
9. Anderson, T. W. (1984). An Introduction to Multivariate Statistical Analysis. Wiley, New York, second edition.
10. Anderson, T. W. and Bahadur, R. R. (1962). Classification into two multivariate normal distributions with different covariance matrices. Annals of Mathematical Statistics, 33:420-431.
11. Apostol, T. (1967). Calculus, Vol. I & Vol II. Wiley, New York.
12. Baldi, P. (1991). Computing with arrays of bell-shaped and sigmoid functions. In Lippmann et al. (1991), pp. 735-742.
13. Baldi, P. and Hornik, K. (1995). Learning in linear neural networks: A survey. IEEE Transactions on Neural Networks, 6(4):837-858.
14. Barnard, E. and Cole, R. A. (1989). A neural-net training program based on conjugate-gradient optimization. Technical Report CSE 89-014, Oregon Graduate Institute of Science and Technology.
15. Barron, A. R. (1984). Predicted squared error: A criterion for automatic model selection. In Farlow, S. J., editor, Self-organizing Methods in Modeling. Marcel Decker, New York.
16. Bartlett, P. L. (1998). The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525-536.
17. Barton, S. A. (1991). A matrix method for optimizing a neural network. Neural Computation, 3:450-459.
18. Battiti, R. (1992). First- and second-order methods for learning: between steepest descent and Newton's method. Neural Computation, 4:141-166.
19. Baum, E. and Haussler, D. (1989). What size net gives valid generalization? Neural Computation, 1:151-160.
20. Baum, E. B. (1990). When are k-nearest neighbor and back propagation accurate for feasible sized sets of examples? In Almeida and Wellekens (1990), pp. 2-25.
21. Becker, R. A., Chambers, J. M, and Wilks, A. R. (1988). The new S Language. Wads worth and Brooks, Monteray, CA.
22. Becker, S. and le Cun, Y. (1989). Improving the convergence of back-propagation learning with second order methods. In Touretzky et al. (1988), pp. 29-37.
23. Benediktsson, J. A., Sveinsson, J. R., and Arnason, K. (1995). Classification and feature extraction of AVIRIS data. IEEE Transactions on Geoscience and Remote Sensing, 33(5):1194-1205.
24. Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems (with discussion). Journal of the Royal Statistical Society B, 36(2):192-236.
25. Besag, J. (1986). On the statistical analysis of dirty pictures (with discussion). Journal of the Royal Statistical Society B, 48(3):259-302.
26. Billingsley, P. (1995). Probability and Measure. Wiley, New York, 3rd edition.
27. Birch, M. W. (1963). Maximum likelihood in three-way contingency tables. Journal of the Royal Statistical Society B, 25:220-233.
28. Bischof, H., Schneider, W., and Pinz, A. (1992). Multispectral classification of Landsat images using neural networks. IEEE Transactions on Geoscience and Remote Sensing, 30(3):482-490.
29. Bishop, C. M. (1993). Curvature-driven smoothing: a learning algorithm for feedforward networks. IEEE Transactions on Neural Networks, 4:882-884.
30. Bishop, C. M. (1995a). Neural Networks for Pattern Recognition. Oxford University Press, Oxford.
31. Bishop, C. M. (1995b). Training with noise is equivalent to Tikohonov regulariza-tion. Neural Computation, 7:108-116.
32. Breiman, L. (2001). Statistical modelling: The two cultures (with discussion). Statistical Science, 16(3):199-231.
33. Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. (1984). Classification and Regression Trees. Wadsworth and Brooks/Cole, Monterey, CA.
34. Breiman, L. and Ihaka, R. (1984). Nonlinear discriminant analysis via ACE and scaling. Technical Report 40, Dept. of Statistics, University of California, Berkeley.
35. Bridle, J. S. (1990). Training stochastic model recognition algorithms as networks can lead to maximum mutual information estimation of parameters. In Touretzky (1990), pp. 211-217.
36. Brown, M. P. S., Grundy, W. N., Lin, D., Cristianini, N., Sugnet, C, Furey, T. S., Manuel Ares, J., and Haussler, D. (2000). Knowledge-based analysis of mi-croarray gene expression data using support vector machines. Proceedings of the National Academy of Sciences, 97(1):62-267.
37. Buja, A., Hastie, T., and Tibshirani, R. (1987). Linear smoothers and additive models. Statistical Research Reports 50, AT&T Bell Laboratories.
38. Burges, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(1):121-167.
39. Campbell, N. A. (1978). The influence function as an aid in outlier detection in discriminant analysis. Journal of the Royal Statistical Society Series B, 27(3):251-258.
40. Campbell, N. A. and Atchley, W. R. (1981). The geometry of canonical variate analysis. Systematic Zoology, 30(3):268-280.
41. Campbell, N. A. and Kiiveri, H. T. (1988). Neighbour relations and remotely sensed data. Internal report, Division of Mathematics and Statistics, CSIRO, Western Australia.
42. Campbell, N. A. and Kiiveri, H. T. (1993). Canonical variate analysis with spatially-correlated data. Australian Journal of Statistics, 35(3):333-344.
43. Campbell, N. A. and Mahon, R. J. (1974). A multivariate study of variation in two species of rock crab of the genus leptographus. Australian Journal of Zoology, 22:417-425.
44. Charalambous, C. (1990). A conjugate gradient algorithm for the efficient training of artificial neural networks. Technical Report 90/06, Centre for Research into Sensory Technology, University of Western Australia.
45. Chauvin, Y. (1990). Generalized performance of overtrained back-propagation networks. In Almeida and Wellekens (1990), pp. 46-56.
46. Chen, D. S. and Jain, R. C. (1994). robust backpropagation learning algorithm for function approximation. Neural Networks, 5(3):467-479.
47. Cheng, B. and Titterington, D. M. (1994). Neural networks: A review from a statistical perspective (with discussion). Statistical Science, 9(1):2-54.
48. Clark, R., Swayze, G., Gallagher, A., Gorelick, N., and Kruse, F. (1991). Mapping with imaging spectrometer data using the complete band shape least-squares algorithm simultaneously fit to multiple spectral features from multiple materials. In Proceedings of the Third Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Workshop, pp. 2-3.
49. Clarke, B. R. (1983). Uniqueness and Frechet differentiability of functional solutions to maximum likelihood type equations. Annals of Statistics, 11 (4): 1196-1205.
50. Clunies-Ross, C. W. and Riffenburgh, R. H. (1960). Geometry and linear discrimination. Biometrika, 47:185-189.
51. Cook, R. D. and Weisberg, S. (1982). Residuals and Influence in Regression. Monographs on Statistics and Applied Probability. Chapman and Hall, London.
52. Cooke, T. and Peake, M. (2002). The optimal classification using a linear discriminant for two point classes having known mean and covariance. Published electronically January 11, 2002, Journal of Multivariate Analysis.
53. Courant, D. and Hilbert, D. (1953). Methods of Mathematical Physics. Wiley-Interscience Publications, New York.
54. Cox, D. R. (1958). Two further applications of a model for binary regression. Biometrika, 45:562-565.
55. Cox, T. F. and Ferry, G. (1991). Robust logistic discrimination. Biometrika, 7(4):841-849.
56. Cox, T. F. and Pearce, K. F. (1997). A robust logistic discrimination model. Statistics and Computing, 7:155-161.
57. Cybenko, G. (1990). Complexity theory of neural networks and classification problems. In Almeida and Wellekens (1990), pp. 46-55.
58. Day, N. E. and Kerridge, D. F. (1967). A general maximum likelihood discriminant. Biometrics, 23:313-323.
59. de Boor, C. (1978). A Practical Guide to Splines, volume 27 of Applied Mathematical Sciences. Springer-Verlag, Berlin.
60. Diaconis, P. and Freedman, D. (1984). Asymptotics of graphical projection pursuit. Annals of Statistics, 12(3):793-815.
61. Dietterich, T. G. (1996). Statistical tests for comparing supervised classification learning algorithms. OR 97331, Oregan State University.
62. Dobson, A. J. (1990). An Introduction to Generalized Linear Models. Chapman and Hall, London.
63. Donoho, D. (2000). High-dimensional data analysis: The curses and blessings of dimensionality. Available at www-stat.stanford.edu/~donoho/Lectures/AMS2000.
64. Dunne, Ft. (1997). Evaluating contextural information models. Unpublished CMIS Remote Sensing and Monitoring Task Report, CSIRO, Floreat Park, Western Australia.
65. Dunne, R. (2000). A graphical method of selecting starting values for an MLP model. Unpublished CMIS Remote Sensing and Monitoring Task Report, CSIRO, Floreat Park, Western Australia.
66. Dunne, R. and Campbell, N. A. (1994). A suite of programs for fitting multi-layer perceptrons in Fortran and S-PLUS - Version 2.1. Technical Report 94/05, Murdoch University, Western Australia.
67. Dunne, R., Campbell, N. A., and Kiiveri, H. T. (1992). Task-based pruning. In Proceedings of the Third Australian Conference on Neural Networks, pp. 166-170. Sydney University Electrical Engineering Department.
68. Efron, B. (1975). The efficiency of logistic regression compared to normal discriminant analysis. Journal of the American Statistical Association, 70:892-898.
69. Efron, B. (1983). Estimating the error rate of a prediction rule, improvements on cross-validation. Journal of the American Statistical Association, 78:316-331.
70. Fahlman, S. E. (1989). Fast-learning variations on back-propagation: An empirical study. In Touretzky (1989), pp. 38-51.
71. Fahlman, S. E. (1990). Readme.sonar. Text file accompanying the distribution of the sonar data set of Gorman and Sejnowski.
72. Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7:179-188.
73. Francis, B., Green, M., and Payne, C. (1993). The GLIM System, Release 4 Manual. Clarendon Press, Oxford.
74. Frank, I. E. and Friedman, J. H. (1993). A statistical view of some chemometrics regression tools (with discussion). Technometrics, 35(2): 109-148.
75. Freund, Y. and Schapire, R. E. (1996). Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference, pp. 148-156.
76. Friedman, J., Hastie, T., and Tibshirani, R. (1998). Additive logistic regression: a statistical view of boosting. Technical report, Department of Statistics, Stanford University.
77. Friedman, J. H. (1984). A variable span smoother. Tech. Rep. No. 5, Laboratory for Computational Statistics, Dept. of Statistics, Stanford Univ., California.
78. Friedman, J. H. (1989). Regularized discriminant analysis. Journal of the American Statistical Association, 84:165-175.
79. Friedman, J. H. (1991a). Multivariate adaptive regression splines (with discussion). Annals of Statistics, 19:1-141.
80. Friedman, J. H. (1991b). Unpublished lecture notes, CSIRO, Sydney, Australia.
81. Friedman, J. H. (1996). On bias, variance, 0/1-loss, and the curse-of-dimensionality. Available on http://www-stat.stanford.edu/~jhf.
82. Friedman, J. H. (1999). Stochastic gradient boosting. Available at http:// www-stat.stanford.edu/~jhf/ftp/stobst.ps.
83. Friedman, J. H. (2000). Greedy function approximation: A gradient boosting machine. The 1999 IMS Reitz Lecture. Available at http://www-stat.Stanford.edu/~jhf/ftp/trebst.ps.
84. Friedman, J. H. and Silverman, B. W. (1989). Flexible parsimonious smoothing and additive modeling. Technometrics, 31(1):3-21.
85. Friedman, J. H. and Stuetzle, W. (1981). Projection pursuit regression. Journal of the American Statistical Association, 76:817-823.
86. Fukumizu, K. (1996). A regularity condition of the information matrix of a multilayer perceptron network. Neural Networks, 9(5):871-879.
87. Fukunaga, K. (1990). Introduction to Statistical Pattern Recognition. Academic Press, Boston, second edition. [First edition, 1972].
88. Funahashi, K. (1989). On the approximate realization of continuous mappings by neural networks. Neural Networks, 2:183-192.
89. Furby, S., Kiiveri, H., and Campbell, N. (1990). The analysis of high dimensional spectral curves. In Proceedings of the Fifth Australian Remote Sensing Conference, pp. 175-184.
90. Gallinari, P., Thiria, S., and Fogelman-Soulie, F. (1988). Multilayer perceptrons and data analysis. In IEEE Annual International Conference on Neural Networks, volume 1, pp. 391-399.
91. Geman, S., Bienenstock, E., and Doursar, R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4(1):1-58.
92. Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6):721-741.
93. Gibson, G. J. and Cowan, C. F. N. (1990). On the decision regions of multilayer perceptrons. Proceedings of the IEEE, 78(10):1590-1594.
94. Girosi, F., Jones, M., and Poggio, T. (1993). Priors, stabilizers and basis functions: from regularization to radial, tensor and additive splines. A. I. Memo No. 1430 C.B.C.L. Paper No. 75, Massachusetts Institute of Technology.
95. Gish, H. (1990). A probabilistic approach to the understanding and training of neural network classifiers. In IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1361-1364.
96. Golub, G. H. and van Loan, C. F. (1982). Matrix Computations. Johns Hopkins University Press, Baltimore.
97. Golub, G. H. and Wilkinson, J. H. (1966). Note on iterative refinement of least squares solution. Numer. Math, 9:139-148.
98. Goodacre, R., Kell, D. B., and Bianchi, G. (1992). Neural networks and olive oil. Nature, 359:594.
99. Goodall, C. (1983). M-estimators of location: An outline of the theory. In Hoaglin, D. C., Mosteller, F., and Tukey, J. W., editors, Understand Robust and Exploratory Data Analysis, pp. 339-403. Wiley, New York.
100. Gorman, R. and Sejnowski, T. (1988). Analysis of hidden units in a layered network trained to classify sonar targets. Neural Networks, 1:75-89.
101. Goutte, C. and Hansen, L. K. (1997). Regularization with a pruning prior. Neural Networks, 10(6):1053-1059.
102. Halmos, P. R. (1974). Measure Theory. Springer-Verlag, New York.
103. Hamamoto, Y., Uchimura, S., and Tomita, S. (1996). On the behavior of artificial neural network classifiers in high-dimensional spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(5):571-574.
104. Hampel, F. (1974). The influence curve and its role in robust estimation. Journal of the American Statistical Association, 69:383-393.
105. Hampel, F. R. (1985). The breakdown points of the mean combined with some rejection rules. Technometrics, 27(2):95-107.
106. Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., and Stahel, W. A. (1986). Robust Statistics: The Approach Based on Influence Functions. Wiley, New York.
107. Hampshire, II, J. B. and Perlmutter, B. A. (1990). Equivalence proofs for multilayer perceptron classifiers and the Bayesian discriminant function. In Touretzky, D., Elman, A., Sejnowski, T., and Hinton, G., editors, Proceedings of the 1990 Connectionist Models Summer School, San Mateo, CA. Morgan Kaufmann.
108. Hand, D. J. (1997). Construction and Assessment of Classification Rules. Wiley, Chichester.
109. Hand, D. J. (2006). Classifier technology and the illusion of progress. Statistical Science, 21(1):1-14. with discussion.
110. Hassibi, B. and Stork, D. G. (1991). Second order derivatives for network pruning: Optimal brain surgeon. In Lippmann et al. (1991), pp. 164-171.
111. Hastie, T. (1994). Nonparametric discriminant analysis. Unpublished lecture notes, AT&T Bell Laboratories.
112. Hastie, T., Buja, A., and Tibshirani, Ft. (1995). Penalized discriminant analysis. Annals of Statistics, 23:73-102.
113. Hastie, T. and Tibshirani, R. (1996). Discriminant analysis by Gaussian mixtures. Journal of the Royal Statistical Society B, 58:158-176.
114. Hastie, T., Tibshirani, R., and Buja, A. (1994). Flexible discriminant analysis by optimal scoring. Journal of the American Statistical Association, 89:1255-1270.
115. Hastie, T., Tibshirani, R., and Buja, A. (1999). Flexible discriminant and mixture models. In Kay, J. and Titterington, D. M., editors, Statistics and Neural Networks: Recent Advances at the Interface. Oxford University Press.
116. Hastie, T., Tibshirani, R., and Friedman, J. (2001). The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer.
117. Hastie, T., Tibshirani, R., Leisch, F., Hornik, K., and Ripley, B. D. (2006). mda: Mixture and flexible discriminant analysis. S original by Trevor Hastie and Robert Tibshirani. R port by Friedrich Leisch and Kurt Hornik and Brian D. Ripley. R package version 0.3-2.
118. Hastie, T. J. and Pregibon, D. (1992). Generalized linear models. In Chambers, J. M. and Hastie, T. J., editors, Statistical Models in S, chapter 9. Wadsworth and Brooks, Pacific Grove, CA.
119. Haykin, S. (1999). Neural Networks. A Comprehensive Foundation. Prentice-Hall, Engelwood Cliffs, NJ.
120. Hebb, D. O. (1949). The Organization of Behavior. Wiley, New York.
121. Hestenes, M. R. and Stiefel, E. (1952). Methods of conjugate gradients for solving linear systems. Jour, of research of the national bureau of standard, 49(6):409-436.
122. Himmelblau, D. M. (1990). Introducing efficient second order effects into back propagation learning. In International Joint Conference on Neural Networks, volume 1, pp. 631-634, Hillsdale, NJ. Lawrence Erlbaum.
123. Hogg, R. V. (1979). An introduction to robust estimation. In Launer, R. L. and Wilkinson, G. N., editors, Robustness In Statistics. Academic Press, Boston.
124. Hornik, K., Sinchcombe, M., and White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359-365.
125. Huber, P. J. (1981). Robust Statistics. Wiley, New York.
126. Huber, P. J. (1985). Projection pursuit. Annals of Statistics, 13(2):435-475.
127. Hunt, B., Qi, Y., and DeKruger, D. (1992). A generalization method for back-propagation using fuzzy sets. In Proceedings of the Third Australian Conference on Neural Networks, pp. 12-16. Sydney University Electrical Engineering Department.
128. Ingrassia, S. and Morlini, I. (2005). Neural network modeling for small datasets. Technometrics, 47(3):297-311.
129. Jervis, T. T. and Fitzgerald, W. J. (1993). Optimization schemes for neural networks. Technical Report CUED/F-INFENG/TR 114, Cambridge University, Electrical Engineering Department.
130. Jordan, M. I. (1995). Why the logistic function? A tutorial on probabilities and neural networks. Computational Cognitive Science Technical Report 9603, Massachusetts Institute of Technology.
131. Karkkainen, T. and Heikkola, E. (2004). Robust formulations for training multilayer perceptrons. Neural Computation, 16:837-862.
132. Kiiveri, H. T. (1992). Canonical variate analysis of high-dimensional spectral data. Technometrics, 34(3):321-331.
133. Kiiveri, H. T. and Caccetta, P. (1996). Some statistical models for remotely sensed data. Unpublished report, CSIRO, Floreat Park, Western Australia.
134. Kiiveri, H. T. and Campbell, N. A. (1992). Allocation of remotely sensed data using Markov models for image data and pixel labels. Australian Journal of Statistics, 34(3):361-374.
135. Knerr, S., Personnaz, L., and Dreyfus, G. (1990). Single layer learning revisited: A stepwise procedure for building and training a neural network. In Fogelman-Soulie, F. and Herault, J., editors, Neurocomputing: Algorithms, Architectures, and Applications, volume F 68. NATO ASI Series, Springer-Verlag, Berlin.
136. Knerr, S., Personnaz, L., and Dreyfus, G. (1992). Handwritten digit recognition by neural networks with single-layer training. IEEE Transactions on Neural Networks, 3(6):962-968.
137. Krzanowski, W. J. and Hand, D. J. (1997). Assessing error rate estimators: the leave-one-out method reconsidered. Australian Journal of Statistics, 39:35-46.
138. Krzanowski, W. J. and Marriott, F. H. C. (1994). Multivariate Analysis Part 2: Classification, Covariance Structures and Repeated Measurements. Edward Arnold, London.
139. Le Cun, Y., Denker, J. S., and Solla, S. A. (1990). Optimal brain damage. In Touretzky (1990), pp. 598-605.
140. Lee, Y. and Lippmann, R. P. (1990). Practical characteristics of neural network and conventional classifiers on artificial speech problems. In Touretzky (1990), pp. 168-177.
141. Li, S. Z. (1995). Markov Random Field Modeling in Computer Vision. Springer-Verlag, New York.
142. Liano, K. (1996). Robust error measure for supervised neural network learning with outliers. IEEE Transactions on Neural Networks, 7(1):246-250.
143. Liestol, K., Andersen, P. K., and Andersen, U. (1994). Survival analysis and neural networks. Statistics in Medicine, 13:1189-1200.
144. Lippmann, R. P. (1987). An introduction to computing with neural networks. IEEE Transactions on Acoustics, Speech, and Signal Processing, 4(2):4-22.
145. Lippmann, R. P., Moody, J. E., and Touretzky, D. S., editors (1991). Advances in Neural Information Processing Systems 3. Proceedings of the 1990 Conference. Morgan Kaufmann, San Mateo, CA.
146. Liu, Y. (1994). Robust parameter estimation and model selection for neural network regression. In Cowan, J. D., Tesauro, G., and Alspector, J., editors, Advances in Neural Information Processing Systems 6. Proceedings of the 1993 Conference, pp. 192-199, San Francisco. Morgan Kaufmann.
147. Lui, H. C. (1990). Analysis of decision contour of neural network with sigmoid nonlinearity. In International Joint Conference on Neural Networks, volume 1, pp. 655-659, Hillsdale, NJ. Lawrence Erlbaum.
148. Mahon, R. (1974). A study of rock crabs of the genus Leptographus Milne Edwards. PhD thesis, Department of Zoology, University of Western Australia.
149. Mardia, F., Kent, T., and Bibby, J. (1979). Multivariate Analysis. Academic Press, New York.
150. Marquardt, D. W. (1970). Generalized inverses, ridge regression, biased linear estimation, and non-linear estimation. Technometrics, 12(3):591-612.
151. Marx, B. D. and Eilers, P. H. C. (1999). Generalized linear regression on sampled signals and curves: a p-spline approach. Technometrics, 41(1):1-13.
152. Mathieson, M. (1997). Ordered classes and incomplete examples in classification. In Mozer, M. C., M. I, J., and Petsche, T., editors, Advances in Neural Information Processing Systems, volume 9. MIT Press, Cambridge, MA.
153. McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models. Chapman and Hall, London.
154. McCulloch, W. S. and Pitts, W. (1943). A logical calculus of ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5:115-133. Reprinted in Anderson and Rosenfeld (1988).
155. McLachlan, G. J. (1992). Discriminant Analysis and Statistical Pattern Recognition. Wiley, New York.
156. Mielniczuk, J. and Tyrcha, J. (1993). Consistency of multilayer preceptron regression estimators. Neural Networks, 6:1019-1022.
157. Minsky and Papert (1969). Perceptrons. MIT Press, Cambridge, MA.
158. Mitchie, D., Spiegelhalter, D. J., and Taylor, C. C. (1994). Machine Learning, Neural and Statistical Classification. Ellis Horwood. The book is now out of print and has been made available at http://www.amsta.leeds.ac.uk/~charles/statlog.
159. Moller, M. F. (1993). A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks, 6:525-533.
160. Moody, J. and Utans, J. (1995). Architecture selection strategies for neural networks: application to corporate bond rating prediction. In Refenes, A.-P., editor, Neural Networks in the Capital Markets, pp. 277-300, Chichester. Wiley.
161. Moody, J. E. (1992). The effective number of parameters: An analysis of generalization and regularization in nonlinear learning systems. In Moody, J. E., Hanson, S. J., and Lippmann, R. P., editors, Advances in Neural Information Processing Systems J,. Proceedings of the 1991 Conference. Morgan Kaufmann, San Mateo, CA.
162. Morgan, J. N. and Sonquist, J. A. (1963). Problems in the analysis of survey data, and a proposal. Journal of the American Statistical Association, 58:415-434.
163. Morgan, N. and Bourland, H. (1991). Generalization and parameter estimation in feedforward nets: some experiments. In Lippmann et al. (1991), pp. 630-637.
164. Movellan, J. R. (1990). Error functions to improve noise resistance and generalization in backpropagation networks. In International Joint Conference on Neural Networks, volume 1, pp. 557-560, Hillsdale, NJ. Lawrence Erlbaum.
165. Mozer, M. C. and Smolensky, P. (1989). Skeletonization: A technique for trimming the fat from a network via relevance assessment. In Touretzky (1989), pp. 107-115.
166. Murata, N., Yoshizawa, S., and Amari, S. (1994). Network information criterion- determining the number of hidden units for artificial neural network models. IEEE Transactions on Neural Networks, 5:865-872.
167. Nash, J. (1990). Compact Numerical Methods For Computers. Linear Algebra and Function Minimization. IOP Publishing, Bristol.
168. Neal, R. M. (1995). Bayesian Learning for Neural Networks. PhD thesis, Department of Computer Science, University of Toronto.
169. Nelder, J. A. and Mead, R. (1965). A simplex algorithm for function minimization. Computer Journal, 7:308-313.
170. Noble, B. and Daniel, J. W. (1988). Applied Linear Algebra. Prentice-Hall International, Englewood Cliffs, NJ.
171. Nowlan, S. J. and Hinton, G. E. (1992). Simplifying neural networks by soft weight-sharing. Neural Computation, 4:473-493.
172. O'Neill, R. (1971). Algorithm AS 47 - function minimization using a simplex procedure. Applied Statistics, 20:338-345.
173. Osborne, M., Presnell, B., and Turlach, B. (1998). Knot selection for regression splines via the lasso. In Weisberg, S., editor, Dimension Reduction, Computational Complexity, and Information, volume 30 of Computing Science and Statistics, Interface Foundation of North America, pp. 44-49. Fairfax Station.
174. Pao, Y. (1989). Adaptive Pattern Recognition and Neural Networks. Addison-Wesley, Reading, MA.
175. Paola, J. D. and Schowergerdt, R. A. (1995). A review and analysis of backpropaga-tion neural networks for classification of remotely-sensed multi-spectral imagery. International Journal of Remote Sensing, 16(16):3033-3058.
176. Perni'a-Espinoza, A. V., Ordieres-Mera, J. B., de Pisón, F. J. M., and Gonzalez-Marcos, A. (2005). Tao-robust backpropagation learning algorithm. Neural Networks, 18(2):191-204.
177. Piegorsch, W. W. and Casella, G. (1989). The early use of matrix diagonal increments in statistical problems. SIAM Review, 31(3):428-434.
178. Plaut, D. C, Nowlan, S. J., and Hinton, G. E. (1986). Experiments on learning by backpropagation. Tech. Rep. CMU-CS-86-126, Carnegie Mellon University, Pittsburg.
179. Poggio, T. and Girosi, F. (1990a). Networks for approximation and learning. Proceedings of the IEEE, 78(9):1481-1497.
180. Poggio, T. and Girosi, F. (1990b). Regularization algorithms for learning that are equivalent to multilayer networks. Science, 247:978-982.
181. Polak, E. (1971). Computational Methods in Optimization. Academic Press, New York.
182. Pregibon, D. (1982). Resistant fits for some commonly used logistic models with medical aplications. Biometrics, 38:485-498.
183. Press, W. H., Flannery, B. P., Teukolsky, S. A., and Vetterling, W. T. (1992). Numerical Recipes in C. Cambridge University Press, Cambridge, second edition.
184. R Development Core Team (2006). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0.
185. Ramsay, J. O. and Silverman, W. B. (1997). Functional Data Analysis. Springer-Verlag, New York.
186. Ramsay, J. O. and Silverman, W. B. (1999). S-PLUS Functions for FDA. Manual accompanying FDA software.
187. Rao, C. R. (1948). The utilization of multiple measurements in problems of biological classification (with discussion). Journal of the Royal Statistical Society B, 10:159-203.
188. Rao, C. R. (1973). Linear Statistical Inference and Its Applications. Wiley, New York, second edition.
189. Reed, R. (1993). Pruning algorithms - a survey. IEEE Transactions on Neural Networks, 4(5):740-747.
190. Reed, R., Marks II, R. J., and Oh, S. (1995). Similarities of error regularization, sigmoid gain scaling, target smoothing, and training with jitter. IEEE Transactions on Neural Networks, 6(3):529-538.
191. Reinsch, C. H. (1967). Smoothing by spline functions. Numerische Mathematik, 10:177-183.
192. Rice, J. A. and Silverman, B. W. (1991). Estimating the mean and covariance structure nonparametrically when the data are curves. Journal of the Royal Statistical Society B, 53(1):233-243.
193. Richard, M. D. and Lippmann, R. P. (1991). Neural network classifiers estimate Bayesian a posteriori probabilities. Neural Computation, 3:461-483.
194. Richards, J. A. (1986). Remote Sensing Digital Image Analysis: An Introduction. Springer-Verlag, Berlin.
195. Richards, J. A. and Jia, X. (2006). Remote Sensing Digital Image Analysis: An Introduction. Springer, Berlin, 4th edition.
196. Ripley, B. D. (1986). Statistics, images and pattern recognition (with discussion). Canadian Journal of Statistics, 14:83-111.
197. Ripley, B. D. (1994a). Flexible non-linear approaches to classification. In Cherkassky, V., Friedman, J. H., and Wechsler, H., editors, From Statistics to Neural Networks: Theory and Pattern Recognition, pp. 105-126, Berlin. ASI Proceedings, subseries F, Computer and Systems Sciences, Springer-Verlag.
198. Ripley, B. D. (1994b). multinom. Software for fitting multinomial models via an MLP. Available from statlib at http://lib.stat.cmu.edu/.
199. Ripley, B. D. (1994c). Neural networks and related methods for classification (with discussion). Journal of the Royal Statistical Society, B, 56:409-456.
200. Ripley, B. D. (1995). Statistical ideas for selecting network architectures. In Kap-pen, B. and Gielen, S., editors, Neural Networks: Artificial Intelligence and Industrial Applications, pp. 183-190. Springer, London.
201. Ripley, B. D. (1996). Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge.
202. Rosenblatt, F. (1962). Principles of Neurodynamics. Spartan Books, Washington, DC.
203. Rousseeuw, P. J. and Leroy, A. M. (1987). Robust Regression and Outlier Detection. Wiley, New York.
204. Rousseeuw, P. J. and van Zomeren, B. C. (1990). Unmasking multivariate outliers and leverage points (with discussion). Journal of the American Statistical Association, 85:633-651.
205. Rowan, T. (1990). Functional Stability Analysis of Numerical Algorithms. PhD thesis, Department of Computer Sciences, University of Texas at Austin.
206. Rubenstein, D. and Hastie, T. (1997). Discriminative vs informative learning, available on http://www-stat.stanford.edu/~trevor/.
207. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323:533-536. Reprinted in Anderson and Rosenfeld (1988).
208. Sankar, A. and Mammone, R. J. (1990a). A fast learning algorithm for tree neural networks. In Proceedings of the 1990 Conference on Information Sciences and Systems, pp. 638-642, Princeton, NJ.
209. Sankar, A. and Mammone, R. J. (1990b). Tree structured neural networks. Technical Report CAIP-TR-122, Center for Computer Aids for Industrial Productivity, Rutgers University.
210. Sankar, A. and Mammone, R. J. (1991a). Optimal pruning of neural tree networks for improved generalization. In Proceedings of the International Joint Conference on Neural Networks, Seattle, volume 2, pp. 219-224.
211. Sankar, A. and Mammone, R. J. (1991b). Speaker independent vowel recognition using neural tree networks. In Proceedings of the International Joint Conference on Neural Networks, Seattle, volume 2, pp. 809-814.
212. Sarle, W. S. (1994). Neural networks and statistical models. In Proceedings of the Nineteenth Annual SAS Users Group International Conference, pp. 1538-1550.
213. Sarle, W. S. (1995). Stopped training and other remedies for overfitting. In Proceedings of the 27th Symposium on the Interface, pp. 352-360.
214. Sarle, W. S. (1996). Faq2.html. The Frequently Asked Questions (FAQ) list for the comp.ai.neural-nets newsgroup, June.
215. Saul, L. K. and Jordan, M. I. (1996). Exploiting tractable substructures in intractable networks. In Touretzky, D. S., Mozer, M. C, and Hasselmo, M. E., editors, Advances in Neural Information Processing Systems, volume 8, pp. 486-492, Cambridge, MA. The MIT Press.
216. Scarselli, F. and Tsoi, A. C. (1998). Universal approximation using feedforward neural networks: a survey of some existing methods and some new results. Neural Networks, 11(1):15-37.
217. Schowengerdt, R. A. (1997). Remote Sensing. Academic Press, San Diego.
218. Scott, D. W. (1992). Multivariate Density Estimation. Wiley, New York.
219. Serra, R. and Zanarini, G. (1990). Complex Systems and Cognitive Processes. Springer Verlag, Berlin.
220. Sethi, I. (1990). Entropy nets: from decision trees to neural networks. Proceedings of the IEEE, 78(10):1605-1613.
221. Shanno, D. F. and Phua, K. H. (1980). Remark on "Algorithm 500: Minimization of unconstrained multivariable functions". ACM Transactions on Mathematical Software, 6:618-622.
222. Shewchuk, J. R. (1994). An introduction to the conjugate gradient method without the agonizing pain. Technical Report CMU-CS-94-125, Carnegie Mellon University, School of Computer Science.
223. Sietsma, J. and Dow, R. J. F. (1991). Creating artificial neural networks that generalize. Neural Networks, 4:67-79.
224. Silva, F. M. and Almeida, L. B. (1990). Acceleration techniques for the backprop-agation algorithm. In Almeida and Wellekens (1990), pp. 110-119.
225. Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman &: Hall, London.
226. Silvey, S. D. (1975). Statistical Inference. Chapman and Hall, London.
227. Smagt, van der, P. P. (1994). Minimization methods for training feedforward neural networks. Neural Networks, 7(1):1-11.
228. Smith, P. L. (1979). Splines: As a useful and convenient statistical tool. American Statistician, 33(2):57-62.
229. Sneath, P. H. A. and Sokal, R. R. (1973). Numerical taxonomy. Freeman, San Francisco.
230. Specht, D. F. (1990a). Probabilistic neural networks. Neural Networks, 3:109-118.
231. Specht, D. F. (1990b). Probabilistic neural networks and the polynomial adaline as complementary techniques for classification. IEEE Transactions on Neural Networks, 1:111-121.
232. Specht, D. F. (1991). A general regression neural network. IEEE Transactions on Neural Networks, 2:568-576.
233. Staudte, R. G. and Sheather, S. J. (1990). Robust Estimation and Testing. Wiley, New York.
234. Stewart, G. W. (1987). Collinearity and least squares regression. Statistical Science, 2:68-100.
235. Strang, G. (1988). Linear Algebra and its Applications. Harcourt Brace Jovanovich, San Diego, third edition.
236. Su, J. Q. and Liu, J. S. (1993). Linear combinations of multiple diagnostic markers. Journal of the American Statistical Association, 88(424):1350-1355.
237. Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society B, 58(1):267-288.
238. Titterington, D. M., Murray, G. D., Murray, L. S., Spiegelhalter, D. J., Skene, A. M., Habbema, J. D. F., and Gelpka, G. J. (1981). Comparison of discrimination techniques applied to a complex data set of head injured patients (with discussion). Journal of the Royal Statistical Society A, 144:145-174.
239. Touretzky, D., Hinton, G., and Sejnowski, T., editors (1988). Proceedings of the 1988 Connectionist Models Summer School, Pittsberg. Morgan Kaufmann, San Mateo, CA.
240. Touretzky, D. S., editor (1989). Advances in Neural Information Processing Systems. Proceedings of the 1988 Conference. Morgan Kaufmann, San Mateo, CA.
241. Touretzky, D. S., editor (1990). Advances in Neural Information Processing Systems 2. Proceedings of the 1989 Conference. Morgan Kaufmann, San Mateo, CA.
242. Turing, A. M. (1948). Rounding-off errors in matrix processes. Quart. J. Mech, pp. 287-308.
243. Utans, J. and Moody, J. (1991). Selecting neural network architectures via the prediction risk: application to corporate bond rating prediction. In Proc. of the First Int. Conf on AI Applications on Wall Street, Los Alamos, CA. IEEE Computer Society.
244. Vapnik, V. N. (1995). The Nature of Statistical Learning Theory. Springer, New York.
245. Venables, W. N. and Ripley, B. D. (1994). Modern Applied Statistics with S-Plus. Springer, New York, first edition.
246. Venables, W. N. and Ripley, B. D. (1999). Modern Applied Statistics with S-Plus. Springer, New York, third edition.
247. Venables, W. N. and Ripley, B. D. (2000). S Programming. Springer, New York.
248. Venables, W. N. and Ripley, B. D. (2002). Modern Applied Statistics with S. Springer, New York, fourth edition.
249. Wahba, G. (1990). Spline Models for Observational Data, volume 59 of Regional Conference in Applied Mathematics. S.I.A.M.
250. Wand, M. P. (1999). A comparison of regression spline smoothing procedures. Unpublished manuscript available at http://www.biostat.harvard.edu/~mwand/papers.html.
251. Webb, A. and Lowe, D. (1990). The optimized internal representation of multilayer classifier networks performs nonlinear discriminant analysis. Neural Networks, 3:367-375.
252. Webb, A., Lowe, D., and Bedworth, M. (1988). A comparison of nonlinear optimization strategies for feed-forward adaptive layered networks. Technical Report 4157, Royal Signal and Radar Establishment.
253. Weigend, A. S. (1994). On overfitting and the effective number of hidden units. In Mozer, M. C, editor, Proceedings of the 1993 Connectionist Models Summer School, pp. 335-342, Hillsdale, NJ. Lawrence Erlbaum.
254. Weigend, A. S., Rumelhart, D. E., and Huberman, B. A. (1991). Generalization by weight-elimination with application to forecasting. In Lippmann et al. (1991), pp. 875-882.
255. Werbos, P. (1974). Beyond Regression. PhD thesis, Harvard University.
256. White, H. (1989). Learning in artificial neural networks: A statistical perspective. Neural Computation, 1:425-464.
257. Widrow, B. and Lehr, M. A. (1990). 30 years of adaptive networks: perceptron, madaline and backpropagation. Proceedings of the IEEE, 78(9): 1415-1442.
258. Wilkinson, G. G., Fierens, F., and Kanellopoulos, I. (1995). Integration of neural and statistical approaches in spatial data classification. Geographical Systems, 2:1-20.
259. Williams, P. (1995). Bayesian regularization and pruning using a Laplacian prior. Neural Computation, 7:117-143.
260. Wilson, J. (1992). A comparison of procedures for classifying remotely-sensed data using simulated data sets incorporating autocorrelations between spectral responses. Int. J. Remote Sensing, 13(14):2701-2725.