Assessment of classification models with small amounts of data

Informatica

Volumn 18, Issue 3, 2007, Pages 343-362

  Brumen, Boštjan ^a   Jurič, Matjaž B ^a   Welzer, Tatjana ^a   Rozman, Ivan ^a   Jaakkola, Hannu ^b   Papadopoulos, Apostolos ^c  

^a UNIVERSITY OF MARIBOR   (Slovenia)

^b TAMPERE UNIVERSITY OF TECHNOLOGY   (Finland)

^c ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Accuracy; Assessment; Classification; Learning curve; Sampling

Indexed keywords

EID: 34948911451     PISSN: 08684952     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (23)

1. Anderson, J.R., and L.J. Schooler (1991). Reflections of the environment in memory. Psychological Science, 2(6), 396-408.
2. Berry, M.A.J., and G. Linoff (1997). Data Mining Techniques for Marketing, Sales, and Customer Support. John Wiley & Sons, Inc., New York.
3. Blake, C.L., and C.J. Merz (1998). UCI Repository of Machine Learning Databases. Department of Information and Comp. Sci., University of California, Irvine, CA, USA. http://www.ics.uci.edu/~mlearn/LRepository.html.
4. Bronshtein, I.N., and K.A. Semendjajev (1997). Handbook of Mathematics, 3rd edition. Springer Verlag.
5. Brumen, B., T. Welzer, I. Golob and H. Jaakkola (2001a). Convergence detection in classification task of knowledge discovery process. In PICMET '01, Portland International Conference on Management of Engineering and Technology. Proceedings, vol. 1. Book of Summaries at PICMET'01 (IEEE Cat. No.01CH37199). pp. 66.
6. Brumen, B., H. Jaakkola and T. Welzer (2001b). Prediction sample size in data mining tasks using additive incremental approach. Frontiers in Artificial Intelligence and Applications, 67, 246-270.
7. Brumen, B., I. Golob, T. Welzer, I. Rozman, M. Družovec and H. Jaakkola (2002a). Data protection for out-sourced data mining. Informatica (Ljublj.), 26(2).
8. Brumen, B., T. Welzer, H. Jaakkola, I. Golob and I. Rozman (2002b). Towards cost-effective construction of classification methods. Frontiers in Artificial Intelligence and Applications, 73, 301-306.
9. Brumen, B., I. Golob, T. Welzer, I. Rozman, M. Družovec, H. Jaakkola (2003). An algorithm for protecting knowledge discovery data. Informatica (Lithuania), 14(3).
10. Brumen, B. (2004). Empirical procedure for assessment of classification algorithms. Doctoral Dissertation, University of Maribor, Slovenia.
11. Brumen, B., I. Golob, H. Jaakkola, T. Welzer and I. Rozman (2004). Early assessment of classification performance. Australasian CS Week Frontiers, 32, 91-96.
12. Brumen, B., T. Welzer, M. Družovec, I. Golob, H. Jaakkola, I. Rozman and J. Kubalík (2005a). Protecting medical data for decision-making analyses. Journal of Medical Systems, 29(1), 65-80.
13. Brumen, B., T. Welzer, I. Rozman, M. Hölbl and H. Jaakkola (2005b). Convergence detection criteria for classification based on final error rate. In: I.J. Rudas (Ed.), Proceedings of the IEEE 3rd International Conference on Computational Cybernetics, pp. 41-46.
14. Cabena, P., P. Hadjinijan, R. Stadler, J. Verhees and A. Zanasi (1997). Discovering Data Mining: From Concept to Implementation. Prentice Hall Ptr, New Jersey, USA.
15. Cohen, P.R. (1995). Empirical Methods for Artificial Intelligence. Mit Press, Cambridge, MA, USA.
16. Coleman, T., M.A. Branch and A. Grace (1999). Optimization Toolbox for Use with Matlab®. User's Guide, Version 2. Mathworks Inc.
17. Frey, L.J., and D.H. Fisher, Jr. (1999). Modeling decision tree performance with the power law. In D. Heckerman and J. Whittaker (Eds.), Proceedings of the Seventh International Workshop on Artificial Intelligence and Statistics. San Francisco, Morgan Kaufmann, Inc., USA.
18. Harris-Jones, C., and T.L. Haines (1997). Sample size and misclassification: Is more always better? Working Paper, AMSCAT-WP-97-118, AMS Center for Advanced Technologies.
19. Lehnert, W.G., J. McCarthy, S. Soderland, E. Riloff, C. Cardie, J. Peterson, F.F. Feng, C. Dolan and S. Gold-man. (1993). Umass/Hughes: description of the CIRCUS system as used for MUC-5. In Proceedings of 5th Message Understanding Conference. MKP Inc., San Mateo, USA. pp. 277-291.
20. Provost, F., D. Jensen and T. Oates (1999). Efficient progressive sampling. In S. Chaudhuri and D. Madigan (Eds.), Proceedings of the Fifth International Conference on Knowledge Discovery and Data Mining, August 15-18, San Diego. ACM Press, pp. 23-32.
21. Quinlan, R.J. (2004). Data Mining Tools See5 & C5.0. http://www.rulequest.com/ (last visited 8/2004).
22. Vapnik, V.N. (1982). Estimation of Dependences Based on Empirical Data. Springer-Verlag, New York, USA.
23. Witten, I.H., and E. Frank (2000). Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann Publishers, San Francisco, USA.