Integrated instance-based learning algorithm

Computational Intelligence

Volumn 16, Issue 1, 2000, Pages 1-28

  Wilson, D Randall ^a   Martinez, Tony R ^b  

^a Brigham Young University   (United States)

^b NONE

Author keywords

[No Author keywords available]

Indexed keywords

FUNCTIONS; INTERPOLATION; LEARNING ALGORITHMS; MATHEMATICAL MODELS; NEURAL NETWORKS; OPTIMIZATION; PARAMETER ESTIMATION; SET THEORY;

DISTANCE FUNCTION; DISTANCE WEIGHTING; INDUCTIVE LEARNING; INSTANCE BASED LEARNING; PRUNING;

LEARNING SYSTEMS;

EID: 0033901276     PISSN: 08247935     EISSN: None     Source Type: Journal    
DOI: 10.1111/0824-7935.00103     Document Type: Article

References (80)

1. AHA, DAVID W. 1992. Tolerating noisy, irrelevant and novel attributes in instance-based learning algorithms. International Journal of Man-Machine Studies, 36:267-287.
2. AHA, DAVID W., and ROBERT L. GOLDSTONE. 1992. Concept learning and flexible weighting. In Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society, Bloomington, IN. Lawrence Erlbaum, New Haven/Hillsdale/Hove, pp. 534-539.
3. AHA, DAVID W., DENNIS KIBLER, and MARC K. ALBERT. 1991. Instance-based learning algorithms. Machine Learning, 6:37-66.
4. ATKESON, CHRIS, ANDREW MOORE, and STEFAN SCHAAL. 1997. Locally weighted learning. Artificial Intelligence Review, 11:11-73.
5. BATCHELOR, BRUCE G. 1978. Pattern Recognition: Ideas in Practice. Plenum Press, New York.
6. BIBERMAN, YORAM. 1994. A context similarity measure. In Proceedings of the European Conference on Machine Learning (ECML-94). Catalina, Italy, Springer Verlag, New York, pp. 49-63.
7. BREIMAN, LEO. 1996. Bagging predictors. Machine Learning, 24:123-140.
8. BREIMAN, LEO, JEROME H. FRIEDMAN, RICHARD A. OLSHEN, and CHARLES J. STONE. 1984. Classification and Regression Trees, Wadsworth International Group, Belmont, CA.
9. BRODLEY, CARLA E. 1993. Addressing the selective superiority problem: Automatic algorithm/model class selection. In Proceedings of the Tenth International Machine Learning Conference, Amherst, MA, pp. 17-24.
10. BROOMHEAD, D. S., and D. LOWE. 1988. Multi-variable functional interpolation and adaptive networks. Complex Systems, 2:321-355.
11. BUNTINE, WRAY. 1992. Learning classification trees. Statistics and Computing, 2:63-73.
12. CAMERON-JONES, R. M. 1995. Instance selection by encoding length heuristic with random mutation hill climbing. In Proceedings of the Eighth Australian Joint Conference on Artificial Intelligence, pp. 99-106.
13. CARPENTER, GAIL A., and STEPHEN GROSSBERG. 1987. A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing, 37:54-115.
14. CHANG, CHIN-LIANG. 1974. Finding prototypes for nearest neighbor classifiers. IEEE Transactions on Computers, 23(11):1179-1184.
15. CLARK, PETER, and TIM NIBLETT. 1989. The CN2 induction algorithm. Machine Learning, 3:261-283.
16. CONOVER, W. J. 1971. Practical Nonparametric Statistics. John Wiley, New York, pp. 206-209, 383.
17. COST, SCOTT, and STEVEN SALZBERG. 1993. A weighted nearest neighbor algorithm for learning with symbolic features. Machine Learning, 10:57-78.
18. COVER, T. M., and P. E. HART. 1967. Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13(1):21-27.
19. DASARATHY, BELUR V. 1991. Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos, CA.
20. DASARATHY, BELUR V., and BELUR V. SHEELA. 1979. A composite classifier system design: Concepts and methodology. Proceedings of the IEEE 67(5):708-713.
21. DEGROOT, M. H. 1986. Probability and Statistics (2nd ed.). Addison-Wesley, Reading, MA.
22. DENG, KAN, and ANDREW W. MOORE. 1995. Multiresolution instance-based learning. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI'95).
23. DIDAY, EDWIN. 1974. Recent progress in distance and similarity measures in pattern recognition. Second International Joint Conference on Pattern Recognition, pp. 534-539.
24. DIETTERICH, THOMAS G. 1989. Limitations on inductive learning. In Proceedings of the Sixth International Conference on Machine Learning. Morgan Kaufmann, San Mateo, CA, pp. 124-128.
25. DOMINGOS, PEDRO 1995. Rule induction and instance-based learning: A unified approach. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI-95). Morgan Kaufmann, San Mateo, CA, pp. 1226-1232.
26. DUDANI, SAHIBSINGH A. 1976. The distance-weighted k-nearest-neighbor rule. IEEE Transactions on Systems, Man and Cybernetics, 6(4):325-327.
27. FISHER, R. S. 1936. The use of multiple measurements in taxonomic problems. Annual Eugenics, 7(part II):179-188.
28. GATES, G. W. 1972. The reduced nearest neighbor rule. IEEE Transactions on Information Theory, IT-18(3):431-433.
29. GIRAUD-CARRIER, CHRISTOPHE, and TONY MARTINEZ. 1995. An efficient metric for heterogeneous inductive learning applications in the attribute-value language. In Intelligent Systems, vol. 1. Edited by E. A. Yfantis. Kluwer, pp. 341-350.
30. HART, P. E. 1968. The condensed nearest neighbor rule. IEEE Transactions on Information Theory, 14:515-516.
31. HECHT-NIELSEN, R. 1987. Counterpropagation networks. Applied Optics, 26(23):4979-4984.
32. KELLER, JAMES M., MICHAEL R. GRAY, and JAMES A. GIVENS, JR. 1985. A fuzzy k-nearest neighbor algorithm. IEEE Transactions on Systems, Man, and Cybernetics, 15(4):580-585.
33. KOHAVI, RON. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI'95).
34. KOHONEN, TEUVO. 1990. The self-organizing map. Proceedings of the IEEE, 78(9):1464-1480.
35. LANGLEY, PAT, WAYNE, IBA, and KEVIN THOMPSON. 1992. An analysis of Bayesian classifiers. In Proceedings of the 10th National Conference on Artificial Intelligence (AAAI-92), AAAI Press/MIT Press, Cambridge, MA, pp. 223-228.
36. LEBOWITZ, MICHAEL. 1985. Categorizing numeric information for generalization. Cognitive Science, 9:285-308.
37. LOWE, DAVID G. 1995. Similarity metric learning for a variable-kernel classifier. Neural Computation, 7(1):72-85.
38. MERZ, C. J., and P. M. MURPHY. 1996. UCI Repository of Machine Learning Databases. University of California, Irvine, Department of Information and Computer Science. Internet: http://www.ics.uci.edu/~mlearn/MLRepository.html.
39. MICHALSKI, RYSZARD S. 1969. On the quasi-minimal solution of the general covering problem. Proceedings of the Fifth International Symposium on Information Processing, Bled, Yugoslavia, pp. 12-128.
40. MICHALSKI, RYSZARD S., ROBERT E. STEPP, and EDWIN DIDAY. 1981. A recent advance in data analysis: Clustering objects into classes characterized by conjunctive concepts. In Progress in Pattern Recognition, vol. 1. Edited by Laveen N. Kanal and Azriel Rosenfeld. North-Holland, New York, pp. 33-56.
41. MICHIE, D., D. SPIEGELHALTER, and C. TAYLOR. 1994. Machine Learning, Neural and Statistical Classification, Book 19. Ellis Horwood, Hertfordshire, England.
42. MITCHELL, TOM M. 1980. The need for biases in learning generalizations. In Readings in Machine Learning. Edited by J. W. Shavlik and T. G. Dietterich. Morgan Kaufmann, San Mateo, CA, pp. 184-191.
43. MOHRI, TAKAO, and HIDEHIKO TANAKA. 1994. An optimal weighting criterion of case indexing for both numeric and symbolic attributes. In Case-Based Reasoning: Papers from the 1994 Workshop, Technical Report WS-94-01. Edited by D. W. Aha. AIII Press, Menlo Park, CA, pp. 123-127.
44. MOORE, ANDREW W., and MARY S. LEE. 1993. Efficient algorithms for minimizing cross validation error. In Machine Learning: Proceedings of the Eleventh International Conference, Morgan Kaufmann, San Mateo, CA.
45. NADLER, MORTON, and ERIC P. SMITH. 1993. Pattern Recognition Engineering. Wiley, New York.
46. NOSOFSKY, ROBERT M. 1986. Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115(1):39-57.
47. PAPADIMITRIOU, CHRISTOS H., and JON LOUIS BENTLEY. 1980. A worst-case analysis of nearest neighbor searching by projection. Lecture Notes in Computer Science, vol. 85, Automata Languages and Programming, pp. 470-482.
48. PAPADIMITRIOU, C. H., and STEIGLITZ, K. 1982. Combinatorial Optimization: Algorithms and Complexity. Prentice-Hall, Englewood Cliffs, NJ.
49. QUINLAN, J. R. 1986. Induction of decision trees, Machine Learning, 1:81-106.
50. QUINLAN, J. R. 1989. Unknown attribute values in induction. In Proceedings of the 6th International Workshop on Machine Learning. Morgan Kaufmann, San Mateo, CA, pp. 164-168.
51. QUINLAN, J. R. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA.
52. RACHLIN, JOHN, SIMON KASIF, STEVEN SALZBERG, and DAVID W. AHA. 1994. Towards a better understanding of memory-based and bayesian classifiers. In Proceedings of the Eleventh International Machine Learning Conference, New Brunswick, NJ. Morgan Kaufmann, San Mateo, CA, pp. 242-250.
53. RENALS, STEVE, and RICHARD ROHWER. 1989. Phoneme classification experiments using radial basis functions. Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN'89), 1:461-467.
54. RITTER, G. L., H. B. WOODRUFF, S. R. LOWRY, and T. L. ISENHOUR. 1975. An algorithm for a selective nearest neighbor decision rule. IEEE Transactions on Information Theory, 21(6):665-669.
55. ROSENBLATT, FRANK. 1959. Principles of Neurodynamics. Spartan Books, New York.
56. RUMELHART, D. E., and J. L. MCCLELLAND. 1986. Parallel Distributed Processing, MIT Press, Cambridge, MA.
57. SALZBERG, STEVEN. 1991. A nearest hyperrectangle learning method. Machine Learning, 6:277-309.
58. SCHAFFER, CULLEN. 1993. Selecting a classification method by cross-validation. Machine Learning, 13(1).
59. SCHAFFER, CULLEN. 1994. A conservation law for generalization performance. In Proceedings of the Eleventh International Conference on Machine Learning (ML'94). Edited by W. W. Cohen and H. Hirsh. Morgan Kaufmann, San Mateo, CA, pp. 259-265.
60. SCHAPIRE, ROBERT E., YOAV FREUND, PETER BARTLETT, and WEE SUN LEE. 1997. Boosting the margin: A new explanation for the effectiveness of voting methods. In Machine Learning: Proceedings of the Fourteenth International Conference (ICML'97). Edited by D. Fisher. Morgan Kaufmann Publishers, San Francisco, CA, pp. 322-330.
61. SKALAK, DAVID B. 1994. Prototype and feature selection by sampling and random mutation hill climbing algorithms. In Proceedings of the Eleventh International Conference on Machine Learning (ML'94). Morgan Kaufmann, San Mateo, CA, pp. 293-301.
62. SPROULL, ROBERT F. 1991. Refinement to nearest-neighbor searching in k-dimensional trees. Algorithmica, 6:579-589.
63. STANFILL, C., and D. WALTZ. 1986. Toward memory-based reasoning. Communications of the ACM, 29:1213-1228.
64. TOMEK, IVAN. 1976. An experiment with the edited nearest-neighbor rule. IEEE Transactions on Systems, Man, and Cybernetics, 6(6):448-452.
65. TVERSKY, AMOS. 1977. Features of similarity. Psychological Review, 84(4):327-352.
66. WASSERMAN, PHILIP D. 1993. Advanced Methods in Neural Computing. Van Nostrand Reinhold, New York.
67. WATSON, I., and F. MARIR. 1994. Case-based reasoning: A review. The Knowledge Engineering Review, 9(4):327-354.
68. WESS, STEFAN, KLAUS-DIETER ALTHOFF, and GUIDO DERWAND. 1994. Using k-d Trees to improve the retrieval step in case-based reasoning. In Topics in Case-Based Reasoning. Edited by Stefan Wess, Klaus-Dieter Althoff, & M. M. Richter. Springer-Verlag, Berlin, pp. 167-181.
69. WETTSCHERECK, DIETRICH. 1994. A hybrid nearest-neighbor and ncarest-hyperrectangle algorithm. In Proceedings of the 7th European Conference on Machine Learning (ECML'94), LNAI-784. Edited by F. Bergadano and L. de Raedt. Springer, Berlin/Heidelberg/New York/Tokyo, pp. 323-338.
70. WETTSCHERECK, DIETRICH, DAVID W. AHA, and TAKAO MOHRI. 1995. A review and comparative evaluation of feature weighting methods for lazy learning algorithms. Technical Report AIC-95012. Naval Research Laboratory, Navy Center for Applied Research in Artificial Intelligence, Washington, D.C.
71. WETTSCHERECK, DIETRICH, and THOMAS G. DIETTERICH. 1995. An experimental comparison of nearest-neighbor and nearest-hyperrectangle algorithms. Machine Learning, 19(1):5-28.
72. WILSON, DENNIS L. 1972. Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man, and Cybernetics, 2(3):408-421.
73. WILSON, D. RANDALL, and TONY R. MARTINEZ. 1996. Instance-based learning with genetically derived attribute weights. International Conference on Artificial Intelligence, Expert Systems and Neural Networks (AIE'96), pp. 11-14.
74. WILSON, D. RANDALL, and TONY R. MARTINEZ. 1997a. Improved heterogeneous distance functions. Journal of Artificial Intelligence Research, 6(1):1-34.
75. WILSON, D. RANDALL, and TONY R. MARTINEZ. 1997b. Instance pruning techniques. In Machine Learning: Proceedings of the Fourteenth International Conference (ICML'97). Edited by D. Fisher. Morgan Kaufmann Publishers, San Francisco, CA, pp. 403-411.
76. WILSON, D. RANDALL, and TONY R. MARTINEZ. 1997c. Bias and the probability of generalization. In Proceedings of the 1997 International Conference on Intelligent Information Systems (IIS'97), pp. 108-114.
77. WILSON, D. RANDALL, and TONY R. MARTINEZ. 2000. Reduction techniques for exemplar-based learning algorithms. To appear in Machine Learning Journal, 38(3).
78. WOLPERT, DAVID H. 1993. On overfitting avoidance as bias. Technical Report SFI TR 92-03-5001. The Santa Fe Institute, Santa Fe, NM.
79. ZARNDT, FREDERICK. 1995. A comprehensive case study: An examination of connectionist and machine learning algorithms. Master's thesis, Brigham Young University.
80. ZHANG, JIANPING, 1992. Selecting typical instances in instance-based learning. In Proceedings of the Ninth International Conference on Machine Learning.