Unsupervised training of Bayesian networks for data clustering

Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences

Volumn 465, Issue 2109, 2009, Pages 2927-2948

  Pham, Duc Truong ^a   Ruz, Gonzalo A ^a,b  

^a CARDIFF UNIVERSITY   (United Kingdom)

^b UNIVERSIDAD ADOLFO IBÁÑEZ   (Chile)

Author keywords

Bayesian networks; Classification expectation maximization algorithm; Clustering; Machine learning; Unsupervised training

Indexed keywords

BAYESIAN NETWORKS; BLIND SOURCE SEPARATION; CLASSIFIERS; CLUSTER ANALYSIS; DISTRIBUTED PARAMETER NETWORKS; EDUCATION; IMAGE PROCESSING; INFERENCE ENGINES; INTELLIGENT NETWORKS; LEARNING ALGORITHMS; MAXIMUM LIKELIHOOD ESTIMATION; MAXIMUM PRINCIPLE; OPTIMIZATION; ROBOT LEARNING; SPEECH RECOGNITION;

BAYESIAN NETWORK CLASSIFIERS; BENCHMARK DATASETS; CLASSIFICATION EXPECTATION-MAXIMIZATION ALGORITHM; CLUSTERING; CLUSTERING ACCURACY; DATA CLUSTERING; EM ALGORITHMS; EXPECTATION-MAXIMIZATION ALGORITHMS; IMAGE APPLICATIONS; LEARNING METHODS; MACHINE LEARNING; MAXIMUM LIKELIHOOD CRITERIA; MULTINETS; NAIVE BAYES; NEW APPROACHES; NEW MODEL; REAL-WORLD IMAGE; STRUCTURE-LEARNING; THREE MODELS; TRADITIONAL CLUSTERING; UNSUPERVISED TRAINING; WOOD BOARDS;

CLUSTERING ALGORITHMS;

EID: 70349426182     PISSN: 13645021     EISSN: 14712946     Source Type: Journal    
DOI: 10.1098/rspa.2009.0065     Document Type: Article

References (46)

1. Acid, S., de Campos, L. M. & Castellano, J. G. 2005 Learning Bayesian network classifiers: search in a space of partially directed acyclic graphs. Mach. Learn. 59, 213-235. (doi:10.1007/s10994-005-0473-4)
2. Asuncion, A. & Newman, D. J. 2007 UCI machine learning repository. Irvine, CA: University of California, School of Information and Computer Science. See http://www.ics.ci.edu/~mlearn/MLRepository.html.
3. Barash, Y. & Friedman, N. 2002 Context-specific Bayesian clustering for gene expression data. J. Comput. Biol. 9, 169-191. (doi:10.1089/ 10665270252935403)
4. Bezdek, J. C. 1981 Pattern recognition with fuzzy objective function algorithms. New York, NY: Plenum Press.
5. Bezdek, J. C. & Pal, N. R. 1998 Some new indexes of cluster validity. IEEE Trans. Syst. Man. Cybern. B 28, 301-315. (doi: 10.1109/3477.678624)
6. Biba, M., Esposito, F., Ferilli, S., Di Mauro, N., Basile, T. M. A. 2007 Unsupervised discretization using kernel density estimation. In Proc. Int. Joint Conf. on ArtificialIntelligence (IJCAI2007), Hyderabad, India.
7. Carpenter, G. & Grossberg, S. 1987 A massively parallel architecture for a self-organizing neural pattern recognition machine. Comput. Vision Graph. Image Understanding 37, 54-115. (doi:10.1016/S0734-189X(87)80014-2)
8. Celeux, G. & Govaert, G. 1992 A classification EM algorithm for clustering and two stochastic versions. Comput. Stat. Data Anal. 14, 315-332. (doi:10.1016/0167-9473(92)90042-E)
9. Celeux, G. & Govaert, G. 1995 Gaussian parsimonious clustering models. Pattern Recognit. 28, 781-793. (doi:10.1016/0031-3203(94)00125-6)
10. Cerquides, J. & Lopez de Mantaras, R. 2005 TAN classifiers based on decomposable distributions. Mach. Learn. 59, 323-354. (doi:10.1007/s10994-005- 0470-7)
11. Chow, C. K. & Liu, C. N. 1968 Approximating discrete probability distributions with dependence trees. IEEE Trans. Inform. Theory IT-14, 462-467. (doi:10.1109/TIT.1968.1054142)
12. Dash, D. & Cooper, G. F. 2002 Exact model averaging with naive Bayesian classifiers. In Proc. 19th Int. Conf. Machine Learning. San Francisco, CA: Morgan Kauffman.
13. Dempster, A. N., Laird, N. M. & Rubin, D. B. 1977 Maximum likelihood from incomplete data via the EM algorithm. J. R. Statist. Soc. Ser. 39, 1-38.
14. Dougherty, J., Kohavi, R. & Sahami, M. 1995 Supervised and unsupervised discretization of continuous features. In Proc. 12th Int. Conf. Machine Learning. San Francisco, CA:Morgan Kaufmann.
15. Duda, R. O. & Hart, P. E. 1973 Pattern classification and scene analysis. New York, NY: John Wiley & Sons.
16. Estévez, P. A., Perez, C. A. & Goles, E. 2003 Genetic input selection to a neural classifier for defect classification of radiata pine boards. Forest Prod. J. 53, 87-94.
17. Fayyad, U. M. & Irani, K. B. 1993 Multi-interval discretization of continuous-valued attributes for classification learning. In Proc. 13th Int. Joint Conf. on Artificial Intelligence. San Francisco, CA: Morgan Kaufmann.
18. Friedman, N. 1998 The Bayesian structural EM algorithm. In 14th Proc. Conf. on Uncertainty in Artificial Intelligence. San Francisco, CA: Morgan Kaufmann.
19. Friedman, N., Geiger, D. & Goldszmidt, M. 1997 Bayesian network classifiers. Mach. Learn. 29, 131-163. (doi:10.1023/A:1007465528199)
20. Grossman, D. & Domingos, P. 2004 Learning Bayesian network classifiers by maximizing conditional likelihood. In Proc. 21st Int. Conf. on Machine Learning. New York, NY: ACM.
21. Gurwicz, Y. & Lerner, B. 2006 Bayesian class-matched multinet classifier. Lect. Notes Comput. Sci. 4109, 145-153. (doi:10.1007/11815921)
22. Hwang, K. & Zhang, B. 2005 Bayesian model averaging of Bayesian network classifiers over multiple node-orders: application to sparse datasets. IEEE Trans. Syst. Man. Cybern. B, Cybern. 35, 1302-1310. (doi:10.1109/TSMCB. 2005.850162)
23. Jing, Y., Pavlović, V. & Rehg J. M. 2008 Boosted Bayesian network classifiers. Mach. Learn. 73, 155-184. (doi:10.1007/s10994-008-5065-7)
24. Kerr, G., Ruskin, H. J., Crane, M. & Doolan, P. 2008 Techniques for clustering gene expression data. Comput. Biol. Med. 38, 283-293. (doi:10.1016/j.compbiomed.2007.11.001)
25. Kohonen, T. 1995 Self-organizing maps. Berlin, Germany: Springer-Verlag.
26. Kruskal, J. B. 1956 On the shortest spanning subtree of a graph and the traveling salesman problem. Proc. Am. Math. Soc. 7, 48-50. (doi:10.2307/2033241)
27. Langley, P., Iba, W. & Thompson, K. 1992 An analysis of Bayesian classifiers. In Proc. 10th National Conf. on Artificial Intelligence. Menlo Park, CA: AAAI Press.
28. Li, J., Zhang, C., Wang, T. & Zhang Y. 2007 Generalized additive Bayesian network classifiers. In Proc. Int. Joint Conf. on Artificial Intelligence (IJCAI 2007), Hyderabad, India.
29. MacKay, D. J. C. 2003 Information theory, inference, and learning algorithms. Cambridge, UK: Cambridge University Press.
30. MacQueen, J. 1967 Some methods for classification and analysis of multivariate observations. In Proc. 5th Berkeley Symp. on Mathematical Statistics and Probability. Berkeley: University of California Press.
31. Meilǎ, M. & Jordan, M. I. 2000 Learning with mixtures of trees. J. Mach. Learn. Res. 1, 1-48. (doi:10.1162/153244301753344605)
32. Neapolitan, R. E. 2004 Learning Bayesian networks. Upper Saddle River, NJ: Pearson Prentice Hall.
33. Pearl, J. 1988 Probabilistic reasoning in intelligent systems: networks of plausible inference. San Francisco, CA: Morgan Kaufmann.
34. Peña, J. M., Lozano, J. M. & Larrañaga, P. 1999 Learning Bayesian networks for clustering by means of constructive induction. Pattern Recogn. Lett. 20, 1219-1230. (doi:10.1016/S0167-8655(99)00089-6)
35. Peña, J. M., Lozano, J. M. & Larrañaga, P. 2000 An improved Bayesian structural EM algorithm for learning Bayesian networks for clustering. Pattern Recogn. Lett. 21, 779-786. (doi:10.1016/S0167-8655(00)00038- 6)
36. Peña, J. M., Lozano, J. M. & Larrañaga, P. 2004 Unsupervised learning of Bayesian networks via estimation of distribution algorithms: an application to gene expression data clustering. 12(Suppl. 1), 63-82.
37. Pham, D. T. & Afify, A. A. 2007 Clustering techniques and their applications in engineering. Proc. IMechE C, J. Mech. Eng. Sci. 221, 1445-1459. (doi:10.1243/09544062JMES508)
38. Pham, D. T. & Alcock R. J. 2003 Smart inspection systems. London, UK: Academic Press.
39. Roberts, S. J., Everson, R. & Rezek I. 2000 Maximum certainty data partitioning. Pattern Recogn. 33, 833-839. (doi:10.1016/S0031-3203(99)00086-2)
40. Ruz, G. A., Estévez, P. A. & Perez, C. A. 2005 A neurofuzzy color image segmentation method for wood surface defect detection. Forest Prod. J. 55, 52-58.
41. Ruz, G. A., Estévez, P. A. & Ramírez, P. A. 2009 Automated visual inspection system for wood defect classification using computational intelligence techniques. Int. J. Syst. Sci. 40, 163-172. (doi:10.1080/00207720802630685)
42. Santafé G., Lozano, J. A. & Larrañaga, P. 2006a Bayesian model averaging of naive Bayes for clustering. IEEE Trans. Syst. Man. Cybern. B 36, 1149-1161. (doi:10.1109/TSMCB.2006.874132)
43. Santafé G., Lozano, J. A. & Larrañaga, P. 2006b Bayesian model averaging of TAN models for clustering. In Proc. European Workshop on Probabilistic Graphical Models (PGM 2006), Prague, Czech Republic.
44. Schmidberger, G. & Frank, E. 2005 Unsupervised discretization using tree-based density estimation. In Proc. 9th European Conf. on Principles and Practice of Knowledge Discovery in Databases, Porto, Portugal. Berlin, UK: Springer. (doi:10.1007/11564126-26)
45. Simpson P. K. 1993 Fuzzy min-max neural networks-part 2: clustering. IEEE Trans. Fuzzy Syst. 1, 32-45. (doi:10.1109/TFUZZ.1993.390282)
46. Ueda, N. & Nakano, R. 1998 Deterministic annealing EM algorithm. Neural Netw. 11, 271-282. (doi:10.1016/S0893-6080(97)00133-0)