Large Datasets Lead to Overly Complex Models: an Explanation and a Solution

Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining, KDD 1998

Volumn , Issue , 1998, Pages 294-298

  Oates, Tim ^a   Jensen, David ^a  

^a Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA MINING; DECISION TREES; LEAD COMPOUNDS; PARAMETER ESTIMATION;

COMPACT MODEL; COMPLEX MODEL; COMPLEXITY RESULTS; CONSTRUCTION ALGORITHMS; KDD COMMUNITY; LARGE DATASETS; MODEL CONSTRUCTION; MODEL SIZE; PARAMETERS ESTIMATES; PRUNING METHODS;

LARGE DATASET;

EID: 85166340496     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (10)

1. Cohen, P. R. 1995a. Ernpirical Methods jor Artificial Intelligence. The MIT Press.
2. Cohen, W. W. 1995b. Fast effective rule induction. In Proceedings of the Twelfth International Conference on Machine Learning.
3. Edgington, E. S. 1995. Raiidomization Tests. Marcel Dekker.
4. Jensen, D., and Cohen, P. R. 1998. Multiple comparisons in induction algorithms. To appear in Machine Learning.
5. Jensen, D. 1991. Knowledge discovery through induction with randomization testing. In Proceedirigs of the 1991 Knowledge Discovery in Databases Workshop, 148-159.
6. Jensen, D. 1992. Induction with Randomization Testing: Decision-Oriented Analysis of Large Data Sets. Ph.D. Dissertation, Washington University.
7. Oates, T., and Jensen, D. 1997. The effects of training set size on decision tree complexity. In Proceedings of The Fourteenth Inteiiiational Conference on Machine Learning, 254-262.
8. Quinlan, J. R., and Rivest, R. 1989. Inferring decision trees using the minimum description length principle. Information and Computation 80:227-248.
9. Quinlan, J. R. 1987. Simplifying decision trees. International Journal of Man-Machine Studies 27:221-234.
10. Quinlan, J. R. 1993. C4-5: Programs for Machine Learning. Morgan Kaufmann.