Alleviating Naive Bayes attribute independence assumption by attribute weighting

Journal of Machine Learning Research

Volumn 14, Issue , 2013, Pages 1947-1988

  Zaidi, Nayyar A ^a   Cerquides, Jesús ^b   Carman, Mark J ^a   Webb, Geoffrey I ^a  

^a MONASH UNIVERSITY   (Australia)

^b ARTIFICIAL INTELLIGENCE RESEARCH INSTITUTE IIIA CSIC   (Spain)

Author keywords

Attribute independence assumption; Classification; Naive Bayes; Weighted naive Bayes classification

Indexed keywords

ATTRIBUTE INDEPENDENCE ASSUMPTION; ATTRIBUTE WEIGHTING; CONDITIONAL INDEPENDENCE ASSUMPTION; LOGISTIC REGRESSIONS; MACHINE LEARNING COMMUNITIES; NAIVE BAYES; NAIVE BAYES CLASSIFIERS; WEIGHTED NAIVE BAYES;

CLASSIFICATION (OF INFORMATION); DECISION TREES; LEARNING ALGORITHMS; LEARNING SYSTEMS;

CLASSIFIERS;

EID: 84884223082     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (46)

1. R. Bellman. Dynamic Programming. Princeton University Press, 1957.
2. B. Bjerregaard, S. Brynitz, J. Holst-Christensen, E. Kalaja, J. Lund-Kristensen, J. Hilden, F. T. deDombal, and J. C. Horrocks. Computer-aided diagnosis of acute abdomen: a system from leeds used on copenhagen patients. In In Decision Making and Medical Care: Can Information Science Help. North-Holland Publishing Company, 1976.
3. L. Breiman. Random forests. Machine Learning, 45:5-32, 2001.
4. A. Carvalho, T. Roos, A. Oliveira, and P. Myllymaki. Discriminative learning of Bayesian networks via factorized conditional log-likelihood. Journal of Machine Learning Research, 2011.
5. J. Cerquides and R. L. De Mántaras. Tan classifiers based on decomposable distributions. Machine Learning, 59(3):323-354, 2005a.
6. J. Cerquides and R. L. De Mántaras. Robust Bayesian linear classifier ensembles. In Proceedings of the Sixteenth European Conference on Machine Learning, pages 70-81, 2005b.
7. B. Cestnik, I. Kononenko, and I. Bratko. Assistant 86: A knowledge-elicitation tool for sophisticated users. In Proceedings of the Second European Working Session on Learning, pages 31-45. Wulmslow, UK: Sigma Press, 1987.
8. B. Damien and G. I. Webb. The need for low bias algorithms in classification learning from small data sets. In Proceedings of the Sixth European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD) 2002, pages 62-73, Berlin, 2002. Springer-Verlag.
9. M. H. DeGroot and S. E. Fienbert. The comparison and evaluation of forecasters. Statistician, 32(1):12-22, 1982.
10. J. Demšar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7:1-30, 2006.
11. P. Domingos and M. J. Pazzani. Beyond independence: Conditions for the optimality of the simple Bayesian classifier. In Proceedings of the Thirteenth International Conference on Machine Learning, pages 105-112. Morgan Kaufmann, 1996.
12. R. Duda, P. Hart, and D. Stork. Pattern Classification. John Wiley and Sons, 2006.
13. U. M. Fayyad and B. I. Keki. On the handling of continuous-valued attributes in decision tree generation. Machine Learning, 8(1):87-102, 1992.
14. J. T. A. S. Ferreira, D. G. T. Denison, and D. J. Hand. Weighted naive Bayes modelling for data mining, 2001.
15. A. Frank and A. Asuncion. UCI machine learning repository, 2010. URL http://archive.ics. uci.edu/ml.
16. J. Friedman. Flexible metric nearest neighbor classification. Technical report, Department of Statistics, Stanford University, 1994.
17. M. Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. The American Statistical Association, 32(200):675-701, 1937.
18. M. Friedman. A comparison of alternative tests of significance for the problem of m rankings. The American Statistical Association, 11(1):86-92, 1940.
19. N. Friedman and M. Goldszmidt. Building classifiers using Bayesian networks. In AAAI-96, pages 1277-1284, 1996.
20. N. Friedman, D. Geiger, and M. Goldszmidt. Bayesian network classifiers. Machine Learning, 29(2):131-163, 1997.
21. R. Greiner, W. Zhou, X. Su, and B. Shen. Structural extensions to logistic regression: Discriminative parameter learning of belief net classifiers. Journal of Machine Learning Research, 2004.
22. D. Grossman and P. Domingos. Learning Bayesian network classifiers by maximizing conditional likelihood. In International Conference on Machine Learning, 2004.
23. I. Guyon, S. Gunn, M. Nikravesh, and L. Zadeh. Feature Extraction, Foundation and Applications. Springer, 2004.
24. M. A. Hall. Correlation-based feature selection for discrete and numeric class machine learning. In Proceedings of the Seventeenth International Conference on Machine Learning, pages 359-366. Morgan Kaufmann, 2000.
25. M. A. Hall. A decision tree-based attribute weighting filter for naive Bayes. Knowledge-Based Systems, 20:120-126, March 2007.
26. D. J. Hand and K. Yu. Idiot's Bayes - not so stupid after all? International Statistical Review, 69(3):385-398, 2001.
27. J. Hilden and B. Bjerregaard. Computer-aided diagnosis and the atypical case. In In Decision Making and Medical Care: Can Information Science Help, pages 365-378. North-Holland Publishing Company, 1976.
28. R. Kohavi and D. Wolpert. Bias plus variance decomposition for zero-one loss functions. In Proceedings of the Thirteenth International Conference on Machine Learning, pages 275-283. San Francisco: Morgan Kaufmann, 1996.
29. I. Kononenko. Comparison of inductive and naive Bayesian learning approaches to automatic knowledge acquisition. Current Trends in Knowledge Acquisition, 1990.
30. P. Langley and S. Sage. Induction of selective Bayesian classifiers. In Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, pages 399-406. Morgan Kaufmann, 1994.
31. P. Langley, W. Iba, and K. Thompson. An analysis of Bayesian classifiers. In Proceedings of the Tenth National Conference on Artificial Intelligence, pages 223-228. AAAI Press and MIT Press, 1992.
32. D. D. Lewis. Naive Bayes at forty: The independence assumption in information retrieval. In ECML-98: Proceedings of the Tenth European Conference on Machine Learning, pages 4-15, Berlin, April 1998. Springer.
33. A. Murphy and R. Winkler. Reliability of subjective probability forecasts of precipitation and temperature. Applied Statistics, 26(1):41-47, 1977.
34. F. Pernkopf and J. A. Bilmes. Efficient heuristics for discriminative structure learning of Bayesian network classifiers. Journal of Machine Learning Research, 2010.
35. C. A. Ratanamahatana and D. Gunopulos. Feature selection for the naive Bayesian classifier using decision trees. Applied Artificial Intelligence, pages 475-487, 2003.
36. T. Roos, H. Wettig, P. Grünwald, P. Myllymäki, and H. Tirri. On discriminative Bayesian network classifiers and logistic regression. Machine Learning, 59(3):267-296, 2005.
37. G. I. Webb. Multiboosting: A technique for combining boosting and wagging. Machine Learning, 40(2):159-196, 2000.
38. G. I. Webb, J. Boughton, and Z. Wang. Not so naive Bayes: Averaged one-dependence estimators. Machine Learning, 58(1):5-24, 2005.
39. G. I. Webb, J. Boughton, F. Zheng, K. M. Ting, and H. Salem. Learning by extrapolation from marginal to full-multivariate probability distributions: decreasingly naive Bayesian classification. Machine Learning, pages 1-40, 2011.
40. J. Wu and Z. Cai. Attribute weighting via differential evolution algorithm for attribute weighted naive Bayes (wnb). Journal of Computational Information Systems, 7(5):1672-1679, 2011.
41. B. Zadrozny and C. Elkan. Transforming classifier scores into accurate multiclass probability estimates. In ACM SIGKDD, International Conference on Knowledge Discovery and Data Mining, 2002.
42. H. Zhang and S. Sheng. Learning weighted naive Bayes with accurate ranking. In Proceedings of the Fourth IEEE International Conference on Data Mining (ICDM-04), pages 567-570, 2004.
43. F. Zheng, G. I. Webb, P. Suraweera, and L. Zhu. Subsumption resolution: An efficient and effective technique for semi-naive Bayesian learning. Machine Learning, 87(1):93-125, 2012.
44. Z. Zheng and G. I. Webb. Lazy learning of Bayesian rules. Machine Learning, 41(1):53-84, 2000.
45. Z. Zheng, G. I. Webb, and K. M. Ting. Lazy Bayesian Rules: A lazy semi-naive Bayesian learning technique competitive to boosting decision trees. In Proceedings of the Sixteenth International Conference on Machine Learning (ICML-99), pages 493-502. Morgan Kaufmann, 1999.
46. C. Zhu, R. H. Byrd, and J. Nocedal. L-bfgs-b: Algorithm 778: L-bfgs-b, fortran routines for large scale bound constrained optimization. ACM Transactions on Mathematical Software, 23(4):550-560, 1997.