Bootstrapping rule induction to achieve rule stability and reduction

Journal of Intelligent Information Systems

Volumn 27, Issue 1, 2006, Pages 49-77

  Waitman, Lemuel R ^a   Fisher, Douglas H ^a   King, Paul H ^a  

^a VANDERBILT UNIVERSITY   (United States)

Author keywords

Algorithm stability; Bootstrapping; Decision rules; Multidimensional scaling; Perioperative medicine; Rule abstraction; Rule induction; Rule similarity; Rule visualization

Indexed keywords

ALGORITHMS; ASYMPTOTIC STABILITY; BOUNDARY CONDITIONS; DECISION THEORY; EXPERT SYSTEMS; PERTURBATION TECHNIQUES;

ALGORITHM STABILITY; BOOTSTRAPPING; DECISION BOUNDARIES; DECISION RULES; MULTIDIMENSIONAL SCALING; PERIOPERATIVE MEDICINE; RULE ABSTRACTION; RULE INDUCTION; RULE SIMILARITY; RULE VISUALIZATION;

LEARNING SYSTEMS;

EID: 33747879040     PISSN: 09259902     EISSN: 15737675     Source Type: Journal    
DOI: 10.1007/s10844-006-1626-z     Document Type: Article

References (48)

1. Bay, S. D. (1999). The UCI KDD Archive, http://kdd.ics.uci.edu. University of California, Department of Information and Computer Science, Irvine, CA.
2. Borg, I., & Groenen, P. J. F. (1997). Modern multidimensional scaling: theory and applications. New York: Springer.
3. Bothner, U., Georgieff, M., & Schwilk, B. (2000). Building a large-scale perioperative anaesthesia outcome-tracking database: Methodology, implementation, and experiences from one provider within the German quality project. British Journal of Anaesthesia, 85, 271-280.
4. Bousquet, O., & Elisseeff, A. (2002). Stability and generalization. Journal of Machine Learning Research, 2, 499-526.
5. Breiman, L. (1996a). Bagging predictors. Machine Learning, 24, 123-140.
6. Breiman, L. (1996b). Heuristics of instability and stabilization in model selection. Annals of Statistics, 24, 2350-2383.
7. Busing, F. (2001). PROXSCAL. http://www.fsw.leidenuniv.nl/www/w3_ment/ medewerkers/busing/ PROXSCAL.HTM. Leiden, The Netherlands: Leiden University.
8. Clark, P., & Boswell, R. (1991). Rule induction with CN2: Some recent improvements. In Machine Learning - Proceedings of the Fifth European Conference (EWSL-91), (pp. 151-163). Berlin: Springer-Verlag.
9. Clark, P., & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3, 261-283.
10. Cohen, W. (1995). Fast effective rule induction. In Proceedings of the Twelfth International Conference on Machine Learning, (pp. 115-123). San Mateo, CA: Morgan Kaufmann.
11. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7, 1-26.
12. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York, NY: Chapman and Hall.
13. Elisseeff, A., Evgeniou, T., & Pontil, M. (2005). Stability of randomized learning algorithms. Journal of Machine Learning Research, 6, 55-79.
14. Evans, B., & Fisher, D. (1994). Overcoming process delays with decision tree induction. IEEE Expert, 9, 60-66.
15. Evans, B., & Fisher, D. (2002). Using decision tree induction to minimize process delays in the printing industry. In Handbook of data mining and knowledge discovery, (pp. 874-881). Oxford, UK: Oxford University Press.
16. Everitt, B. (1993). Cluster analysis, 3rd ed. London: E. Arnold.
17. Evgeniou, T., Pontil, M., & Elisseeff, A. (2004). Leave-one-out error, stability, and generalization of voting combinations of classifiers. Machine Learning, 55, 71-97.
18. Forrest, W. Y., & Harris, D. F. (1993). Multidimensional scaling. In SPSS For Windows: Professional Statistics, Release 6.0, (pp. 155-222). Chicago, IL: SPSS Inc.
19. Forrest, J. B., Cahalan, M. K., Rehder, K., & Goldsmith, C. H. (1990). Multicenter study of general anesthesia. II. Results. Anesthesiology, 72, 262-268.
20. Forrest, J. B., Rehder, K., Cahalan, M. K., & Goldsmith, C. H. (1992). Multicenter study of general anesthesia III. Predictors of severe perioperative adverse outcomes. Anesthesiology, 76, 3-15.
21. Freidman, J. H., & Popescu, B. E. (2005). Predictive Learning via Rule Ensembles. Stanford University, Department of Statistics, http://www.stat.stanford.edu/people/faculty/friednian/ index.html.
22. Freund, Y., & Schapire, R. E. (1996). Experiments with a new boosting algorithm. In Proceedings of the Thirteenth International Conference on Machine Learning, (pp. 148-156). San Mateo, CA: Morgan Kaufmann.
23. Gago, P., & Bentos, C. (1998). A metric for selection of the most promising rules. In Proceedings of the Second European Conference on the Principles of Data Mining and Knowledge Discovery (PKDD'98), (pp. 19-27). France: Nantes.
24. Good, I. J. (1965). The Estimation of Probabilities: An Essay on Modern Bayesian Methods. Research Monograph 30. Cambridge, MA: MIT Press.
25. Gower, J. C. (1971). A general coefficient of similarity and some of its properties. Biometrics, 27, 857-872.
26. Higgins, M. S., Beattie, C., St. Jaques, P., Patel, N., Yarborough, J., & Tsai, Y.S. (1997). The Vanderbilt Perioperative Information Management System Forms Foundation for Outcomes Management. In AMIA Annual Fall Symposium, Nashville, TN: (pp. 948). Philadelphia, PA: Hanley and Belfus Inc.
27. Johnson, R. A., & Wichern, D. W. (1992). Clustering. In Applied Muttivariate Statistical Analysis. 3rd ed, (pp. 573-609). Upper Saddle River, NJ: Prentice Hall, Inc.
28. Kearns, M., & Ron, D. (1999). Algorithmic stability and sanity check bounds for leave-one-out cross validation bounds. Neural Computation, 11, 1427-1453.
29. Klösgen, W (2002). Decision rules. In Handbook of Data Mining and Knowledge Discovery, (pp. 277-282). Oxford, UK: Oxford University Press.
30. Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection, (pp. 1137-1145). San Mateo, CA: Morgan Kaufmann.
31. Kubat, M., Holte, R. C., & Matwin, S. (1998). Machine learning for the detection of oil spills in satellite radar images. Machine Learning, 30, 195-215.
32. Kutin, S., & Niyogi, P. (2002). Almost-everywhere algorithmic stability and generalization error. In Proceedings of the 18th Conference on Uncertainty in AI, (pp. 275-282). San Francisco, CA: Morgan Kaufmann.
33. Lent, B., Swami, A., & Widom, J. (1997). Clustering association rules. In Proceedings of the Thirteenth International Conference on Data Engineering (IDCE '97), (pp. 220-231). Birmingham, England: IEEE Computer Society Press.
34. Michalski, R. S., & Chilausky, R. (1980). Learning by being told and learning from examples: An experimental comparison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis. International Journal of Policy Analysis and Information Systems, 4, 125-160.
35. Mitchell, T. M. (1997). Machine learning. New York, NY: McGraw Hill.
36. Neter, J., Wasserman, W., & Kutner, M. H. (1990). Applied linear statistical models, 3rd ed. Homewood, IL: Richard D. Irwin, Inc.
37. Riddle, P., & Fresnedo, R. N. D. (1996). Framework for a generic knowledge discovery toolkit. In Learning From Data: AI and Statistics V, (pp. 341-353). Berlin: Springer-Verlag.
38. Riddle, P., Segal, R., & Etzioni, O. (1994). Representation design and brute-force induction in a Boeing manufacturing domain. Applied Artificial Intelligence, 8, 125-147.
39. Segal, R. B. (1997). Machine Learning as Massive Search. PhD Dissertation. University of Washington, Department of Computer Science, Seattle, WA.
40. Segal, R., & Etzioni, O. (1994). Learning decision lists using homogeneous rules. In Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI-94), (pp. 619-625). Menlo Park, CA: AAAI Press/MIT Press.
41. Silberschatz, A., & Tuzhilin, A. (1995). On subjective measures of interestingness in knowledge discovery. In Proceedings of the First International Conference on Knowledge Discovery and Data Mining (KDD-95), (pp. 275-281). Menlo Park, CA: AAAI Press.
42. Smyth, P., & Goodman, R. M. (1991). Rule induction using information theory. In Knowledge Discovery in Databases, (pp. 159-176). Cambridge, MA: MIT Press.
43. Strait, P. T. (1983). Some standard tests of statistical hypotheses. In A First Course in Probability and Statistics with Applications, (pp. 410-422). New York, NY: Harcourt Brace Jovanovich, Inc.
44. Taylor, P. (1999). Statistical methods. In Intelligent Data Analysis: an Introduction, (pp. 113-118). Berlin: Springer-Verlag.
45. Torgerson, W. S. (1958). Theory and methods of scaling. New York: John Wiley and Sons.
46. Turney, P. (1995). Bias and the quantification of stability. Machine Learning, 20, 23-33.
47. van den Eijkel, G. C. (1999). Rule induction. In Intelligent Data Analysis: An Introduction, (pp. 195-216). Berlin: Springer-Verlag.
48. Waitman, L. R., Fisher, D., & King, P. (2003). Bootstrapping rule induction. In Proceedings of the IEEE International Conference on Data Mining, (pp. 677-680). Los Alamitos, CA: IEEE Computer Society.