DistAl: An inter-pattern distance-based constructive learning algorithm

Intelligent Data Analysis

Volumn 3, Issue 1, 1999, Pages 55-73

  Yang, Jihoon ^a,b   Parekh, Rajesh ^a,c   Honavar, Vasant ^a  

^a Iowa State University   (United States)

^b Hughes Aircraft Co   (United States)

^c Allstate Res and Planning Center   (United States)

Author keywords

Constructive learning algorithms; Neural networks; Pattern classification

Indexed keywords

ITERATIVE METHODS; NETWORK LAYERS; NEURAL NETWORKS; NEURONS; PATTERN RECOGNITION; PATTERN RECOGNITION SYSTEMS;

CONSTRUCTIVE LEARNING ALGORITHM; CONSTRUCTIVE NEURAL NETWORK; MULTI-LAYER NETWORK; REAL-WORLD DATASETS; THRESHOLD NEURON; THRESHOLD-LOGIC UNITS; TRAINING PATTERNS; WEIGHT MODIFICATIONS;

LEARNING ALGORITHMS;

EID: 0006406095     PISSN: 1088467X     EISSN: 15714128     Source Type: Journal    
DOI: 10.3233/IDA-1999-3105     Document Type: Article

References (81)

1. U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, R. Uthurusamy, Advances in KnowledgeDiscovery and Data Mining, MIT Press, Cambridge, MA, 1996.
2. V. Honavar, Machine learning: Principles and applications, in: J. Webster (Ed.), Encyclopedia of Electrical and Electronics Engineering, Wiley, New York, to appear.
3. J. Bradshaw, Software Agents, MIT Press, Cambridge, MA, 1997.
4. V. Honavar, Intelligent agents, in: J. Williams, K. Sochats (Eds.), Encyclopedia ofInformation Technology, Marcel Dekker, New York, to appear.
5. K. Balakrishnan, V. Honavar, Intelligent diagnosis systems, International Journal of Intelligent Systems (1998).
6. R. Duda, P. Hart, Pattern Classification and Scene Analysis, Wiley, New York, 1973.
7. T. Mitchell, Machine Learning, McGraw-Hill, New York, 1997.
8. P. Langley, Elements of Machine Learning, Morgan Kaufmann, Palo Alto, CA, 1995.
9. V. Honavar, Toward learning systems that integrate multiple strategies and representations, in: V. Honavar, L. Uhr (Eds.), Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, Academic Press: New York, 1994, pp. 615-644.
10. C-H. Chen, R. Parekh, J. Yang, K. Balakrishnan, V. Honavar, Analysis of decision boundaries generated by constructive neural network learning algorithms, in: Proceedings ofWCNN95, July 17-21, Washington DC, vol. I, 1995, pp. 628-635.
11. S. Gallant, Neural Network Learning and Expert Systems, MIT Press, Cambridge, MA, 1993.
12. R. Parekh, J. Yang, V. Honavar, Constructive neural network learning algorithms for multi-category real-valued pattern classification, Technical Report ISU-CS-TR97-06, Department of Computer Science, Iowa State University, 1997.
13. R. Parekh, J. Yang, V. Honavar, Mupstart-a constructive neural network learning algorithm for multi-category pattern classification, in: Proceedings of the IEEEIINNS International Conference on Neural Networks, ICNN'97, 1997, pp. 1924-1929.
14. J. Yang, R. Parekh, V. Honavar, MTiling-a constructive neural network learning algorithm for multi-category pattern classification, in: Proceedings of the World Congress on Neural Networks'96, San Diego, 1996, pp. 182-187.
15. V. Honavar, Structural learning, in: J. Webster (Ed.), Encyclopediaof Electrical and Electronics Engineering, Wiley, New York, to appear.
16. D. Rumelhart, G. Hinton, R. Williams, Learning internal representations by error propagation, in: Parallel Distributed Processing: Explorations into the Microstructure of Cognition, vol. I (Foundations). MIT Press, Cambridge, MA, 1986.
17. P. Werbos, Beyond regression: New tools for prediction and analysis in behavioral sciences, Ph.D. Thesis, Harvard University, 1974.
18. F. Rosenblatt, The perceptron: A probabilistic model for information storage and organization in the brain, Psychological Rev. 65 (1958) 386-408.
19. N. Nilsson, The Mathematical Foundations of Learning Machines, McGraw-Hill, New York, 1965.
20. W. Krauth, M. Mezard, Learning algorithms with optimal stability in neural networks, J. Phys. A: Math. Gen. 20 (1987) L745-L752.
21. J. Anlauf, M. Biehl, Properties of an adaptive perceptron algorithm, in: Parallel Processing in Neural Systems and Computers. 1990, pp. 153-156.
22. M. Frean, Small nets and short paths: Optimizing neural computation, Ph.D. Thesis, Center for Cognitive Science, Edinburgh University, UK, 1990.
23. H. Poulard, Barycentric correction procedure: A fast method of learning threshold units, in: Proceedings of WCNN'95, July 17-21, Washington DC 1 (1995) 710-713.
24. B. Raffin, M. Gordon, Learning and generalization with minimerror, a temperature-dependen learning algorithm, Neural Comput. 7 (1995) 1206-1224.
25. R. Reed, Pruning algorithms-a survey, IEEE Trans. Neural Networks 4 (5) (1993) 740-747.
26. R. Parekh, J. Yang, V. Honavar, Pruning strategies for constructive neural network learning algorithms. in: Proceedings of the IEEE/INNS InternationalConference on Neural Networks, ICNN'97, 1997. pp. 1960-1965.
27. V. Honavar, Generative Learning structures and processes for generalized connectionist networks, Ph.D. Thesis, Univel'sityof Wisconsin, Madison, 1990.
28. V. Honavar, L. Uhr, Generative learning structures for generalized connectionist networks, Inform. Sci. 70 (1-2) (1993) 75-108.
29. R. Parekh, V. Honavar, Constructive theory refinement in knowledge based neural networks, in: Proceedings of the International Joint Conference on Neural Networks, Anchorage, Alaska, 1998, pp. 2318-2323.
30. J. Nadal, Study of a growth algorithm for a feedforward network, Internat. J. Neural Systems I (I) (1989) 55-59.
31. S. Gallant, Perceptron based learning algorithms, IEEE Trans. Neural Networks 1 (2) (1990) 179-191.
32. M. Mezard, J. Nadal, Learning feed-forward networks: The tiling algorithm, J. Phys. A: Math. Gen. 22 (1989) 2191-2203.
33. M. Frean, The upstart algorithm: A method for constructing and training feedforward neural networks, Neural Comput. 2 (1990) 198-209.
34. N. Burgess, A constructive algorithm that converges for real-valued input patterns, Internat. J. Neural Systems 5 (1) (1994) 59-66.
35. M. Marchand, M. Golea, P. Rujan, A convergence theorem for sequential learning in two-layer perceptrons, Europhys, Lett. 11 (6) (1990) 487-492.
36. J. Yang, V. Honavar, Feature subset selection using a genetic algorithm. in: Proceedings of the Genetic Programming Conference, GP'()7, Stanford University, CA, 1997, pp. 380-385.
37. J. Yang, V. Honavar, Feature subset selection using a genetic algorithm, IEEE Intel!. Systems 13 (2) (1998) 44-49.
38. J. Yang, V. Honavar, Feature subset selection using a genetic algorithm, in: Feature Extraction, Construction and Selection-A Data Mining Perspective. Kluwer Academic, New York, 1998.
39. D. Aha, Incremental constructive induction: An instance-based approach, in: Proceedings of the Eighth International Workshop on Machine Learning, Evanston, IL, Morgan Kaufmann, 1991, pp. 117-121.
40. D. Aha, D. Kibler, M. Albert, Instance-based learning algorithms, Mach. Learning 6 (1991) 37-66.
41. P. Turney, Theoretical analyses of cross-validation error and voting in instance-based learning. J. Experimental and Theoretical Artificial Intell. 1994, pp. 331-360.
42. P. Domingos, Rule induction and instance-based learning: A unified approach, in: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-95), 1995.
43. T. Cover, P. Hart, Nearest neighbor pattern classification, IEEE Trans. Information Theory 13 (1967) 21-27.
44. E. Diday, Recent progress in distance and similarity measures in pattern recognition, in: Proceedings of the Second International Joint Conference on Pattern Recognition, 1974, pp. 534-539.
45. B. Dasarathy, Nearest Neighbor (NN) Norms: NN Pattern Classification Techiniques, IEEE Computer Society Press, Los Alamitos, CA, 1991.
46. C. Stanfill, D. Waltz, Toward memory-based reasoning, Commun. ACM 29 (12) (1986) 1213-1228.
47. S. Cost, S. Salzberg, A weighted nearest neighbor algorithm for learning with symbolic features, Mach. Learning 10 (1) (1993) 57-78.
48. J. Kolodner, Case-Based Reasoning, Morgan Kaufmann, San Francisco, 1993.
49. G. Carpenter, S. Grossberg, Pattern Recognition by Self-Organizing Neural Networks, MIT Press, Cambridge, MA, 1991.
50. A. Tversky, Features of similiarity, Psychological Rev. 84 (1977) 327-352.
51. R. Nosofsky, Attention, similarity, and the identification-categorization relationship, J. Experimental Psychology: General 115 (1986) 39-57.
52. G. Salton, M. McGill, Introduction to modem information retrieval, McGraw-Hill, New York, 1983.
53. D. Broomhead, D. Lowe, Multivariable functional interpolation and adaptive networks, Complex Systems 2 (1988) 321-355.
54. M. Powell, Radial basis functions for multivariable interpolation: A review, in: J. Mason, M. Cox (Eds.), Algorithms for Approximation, Clarendon Press, Oxford, 1987, pp. 143-167.
55. F. Girosi, M. Jones, T. Poggio, Regularization theory and neural networks architectures, Neural Computation 7 (1995) 219-269.
56. R. Michalski, I. Mozetic, J. Hong, H. Lavrac, The multi-purpose incremental learning system aq15 and its testing application to three medical domains, in: Proceedings of the Fifth National Conference on AI. Morgan Kaufmann, 1986, pp. 1041-1045.
57. P. Clark, R. Niblett, The cn2 induction algorithm, Mach. Learning 3 (1989) 261-284.
58. B. Batchelor, Pattern Recognition: Ideas in Practice, Plenum Press, New York, 1978.
59. D. Wilson, T. Martinez, Improved heterogeneous distance functions, J. Artificial Intel!. Res. 6 (1997) 1-34.
60. B. Natarajan, Machine Learning: A Theoretical Approach, Morgan Kauffman, San Mateo, CA, 1991.
61. D. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, New York, 1989.
62. M. Mitchell, An Introduction to Genetic algorithms, MIT Press, Cambridge, MA, 1996.
63. Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs, 3rd ed., Springer, New York, 1996.
64. P. Murphy, D. Aha, Repository of machine learning databases, Department ofInformation and Computer Science, University of California, Irvine, CA, 1994.
65. G. Salton, Developments in automatic text retrieval, Science 53 (1991) 974-979.
66. J. Yang, P. Pai, V. Honavar, L. Miller, Mobile intelligent agents for document classification and retrieval: A machine learning approach, in: 14th European Meeting on Cybernetics and Systems Research, Symposium on Agent Theory to Agent Implementation, Vienna, Austria, 1998.
67. D. Koller, M. Sahami, Hierarchically classifying documents using very few words, in: International Conference on Machine Learning, 1997, pp. 170-178.
68. H. Andersen, A. Tsoi, A constructive algorithm for the training of a multilayer perceptron based on the genetic algorithm, Complex Systems 7 (1993) 249-268.
69. S. Fahlman, C. Lebiere, The cascade correlation learning algorithm, in: D. Touretzky (Ed.), Neural Information Systems 2, Morgan-Kauffman, 1990, pp. 524-532.
70. M. Golea, M. Marchand, A growth algorithm for neural network decision trees, Europhysics Letters 12 (3) (1990) 205-210.
71. S. Fahlman, Faster-learning variations on backpropagation: an empirical study, in: D. Touretzky, G. Hinton, T. Sejnowsky (Eds.), Proceedings of the 1988 Connectionist Models Summer School, Morgan-Kauffman, 1988, pp. 38-51.
72. R. Kohavi, Feature subset selection as search with probabilistic estimates, in: AAAI Fall Symposium on Relevance, 1994.
73. M. Richeldi, P. Lanzi. Performing effective feature selection by investigating the deep structure of the data, in: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining. AAAI Press, 1996, pp. 379-383.
74. J. Yang, V. Honavar, Experiments with the cascade-correlation algorithm, in: Proceedings of the Fourth UNB AI Symposium, Frederiction, Canada, 1991, pp. 369-380.
75. R. Parekh, Constructive learning: Inducing grammars and neural networks, Ph.D. Thesis, Department of Computer Science, Iowa State University, Ames, lA, 1998.
76. T. Andersen, T. Martinez The effect of decision surface fitness on dynamic multi-layer perceptron networks (dmpl), in: Proceedings of the World Congress on Neural Networks'96, pp. 177-181, San Diego, 1996, pp. 369-380.
77. D. Lowe, Similarity metric learning for a variable-kernel classifier, Neural Computation 7 (1995) 72-85.
78. S. Weiss, L. Kapouleas, An empirical comparison of pattern recognition, neural nets, and machine learning classification methods, in: Proceedings of the 11th International Joint Conference on Artificial Intelligence, 1989.
79. C. Merz, P. Murphy, UCI Repository of Machine Learning Databases, 1998.
80. S. Thrun, Lifelong learning: A case study, Technical Report CMU-CS-95-208, Carnegie Mellon University, 1995.
81. J.W. Shavlik, A framework for combining symbolic and neural learning, in: Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, Academic Press, Boston, 1994.