Machine learning using Bernoulli mixture models: Clustering, rule extraction and dimensionality reduction

Neurocomputing

Volumn 119, Issue , 2013, Pages 366-374

  Saeed, Mehreen ^a   Javed, Kashif ^b   Atique Babri, Haroon ^b  

^a NATIONAL UNIVERSITY OF COMPUTER AND EMERGING SCIENCES   (Pakistan)

^b UNIVERSITY OF ENGINEERING AND TECHNOLOGY   (Pakistan)

Author keywords

Bernoulli mixture models; Binary data; Clustering; Dimensionality reduction; Feature transformation; Rule extraction

Indexed keywords

BERNOULLI MIXTURES; BINARY DATA; CLUSTERING; DIMENSIONALITY REDUCTION; FEATURE TRANSFORMATIONS; RULE EXTRACTION;

EXTRACTION; LEARNING SYSTEMS; METADATA; PATTERN RECOGNITION;

MIXTURES;

ACCURACY; ARTICLE; BERNOULLI MIXTURE MODEL; CLASSIFICATION; CLUSTER ANALYSIS; LEARNING ALGORITHM; MACHINE LEARNING; MULTIFACTOR DIMENSIONALITY REDUCTION; PATTERN RECOGNITION; PRIORITY JOURNAL; PROBABILITY; PROBLEM SOLVING; STATISTICAL MODEL; THEORETICAL STUDY;

EID: 84881552661     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2013.03.021     Document Type: Article

References (47)

1. McLachlan G., Peel D. Finite Mixture Models 2000, John Wiley & Sons, Inc.
2. H. Li, K. Yamanishi, Document classification using a finite mixture model, in: Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics, ACL '98, Association for Computational Linguistics, 1997, pp. 39-47,. http://dx.doi.org/10.3115/976909.979623.
3. Figueiredo M.A., Jain A.K. Unsupervised learning of finite mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 2002, 24(3):381-396.
4. Law M.H., Figueiredo M.A., Jain A.K. Simultaneous feature selection and clustering using mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 2004, 26(9):1154-1166.
5. Alpaydin E. Introduction to Machine Learning 2005, Prentice-Hall of India Private Limited.
6. Bishop C.M. Pattern Recognition and Machine Learning 2006, Springer.
7. Bouguila N. On multivariate binary data clustering and feature weighting. Comput. Stat. Data Anal. 2010, 54(1):120-134.
8. Carreira-Perpinán M.A., Renals S.A. Practical identifiability of finite mixtures of multivariate Bernoulli distributions. Neural Comput. 2000, 12(1):141-152.
9. A. Patrikainen, H. Mannila, Subspace clustering of high-dimensional binary data-a probabilistic approach, in: Workshop on Clustering High-dimensional Data and Its Applications, SIAM International Conference on Data Mining, 2004, pp. 57-65.
10. Juan A., Vidal E. On the use of Bernoulli mixture models for text classification. Pattern Recognition 2002, 35(12):2705-2710.
11. J. González, A. Juan, P. Dupont, E. Vidal, F. Casacuberta, A Bernoulli mixture model for word categorization, in: Proceedings of the IX Spanish Symposium on Pattern Recognition and Image Analysis, vol. 1, Benicassim, Spain, 2001, pp. 165-170.
12. J. Novovičová, A. Malík, Application of multinomial mixture model to text classification, in: Iberian Conference on Pattern Recognition and Image Analysis, IbPRIA 2003, Lecture Notes in Computer Science, vol. 2652, Springer-Verlag, Berlin, Heidelberg, 2003, pp. 646-652.
13. S. Zhu, I. Takigawa, S. Zhang, H. Mamitsuka, A probabilistic model for clustering text documents with multiple fields, in: Advances in Information Retrieval, 29th European Conference on IR Research (ECIR 2007), Lecture Notes in Computer Science, vol. 4425, Springer-Verlag, Berlin, Heidelberg, 2007, pp. 331-342.
14. J. Grim, P. Pudil, P. Somol, Multivariate structural Bernoulli mixtures for recognition of handwritten numerals, in: Proceedings of International Conference on Pattern Recognition (ICPR'00), 2000.
15. A. Juan, E. Vidal, Bernoulli mixture models for binary images, in: Proceedings of 17th International Conference on Pattern Recognition (ICPR 04), 2004.
16. V. Romero, A. Giménez, A. Juan, Explicit modelling of invariances in Bernoulli mixtures for binary images, in: 3rd Iberian Conference on Pattern Recognition and Image Analysis, IbPRIA 2007, Lecture Notes in Computer Science, vol. 4477, Springer-Verlag, Berlin, Heidelberg, 2007, pp. 539-546.
17. Sun Z., Rosen O., Sampson A. Multivariate Bernoulli mixture models with application to postmortem tissue studies in schizophrenia. Biometrics 2007, 63:901-909.
18. J. Tikka, J. Hollmen, S. Myllykangas, Mixture modelling of DNA copy number amplification patterns in cancer, in: Proceedings of the 9th International Work-Conference on Artificial Neural Networks (IWANN' 2007), Springer-Verlag, Berlin, Heidelberg, 2007, pp. 972-979.
19. J. Hollmen, J. Tikka, Compact and understandable descriptions of mixture of Bernoulli distributions, in: Proceedings of the 7th International Symposium on Intelligent Data Analysis (IDA 2007), Springer-Verlag, Berlin, Heidelberg, 2007, pp. 1-12.
20. Myllykangas S., Tikka J., Böhling T., Knuutila S., Hollmén J. Classification of human cancers based on DNA copy number amplification modelling. BMC Med. Genomics 2008, 1(15):1-13.
21. Calinon S., Guenter F., Billard A. On learning, representing, and generalizing a task in humanoid robot. IEEE Trans. Syst. Man Cybern. B. Cybern. 2007, 37(2):286-298.
22. Pavlov D., Mannila H., Smyth P. Beyond independence: Probabilistic models for query approximation on binary transaction data. IEEE Trans. Knowl. Data Eng. 2003, 15(6):1409-1421.
23. Sajama, A. Orlitsky, Supervised dimensionality reduction using mixture models, in: Proceedings of the 22nd International Conference on Machine Learning, Bonn, Germany, 2005, pp. 768-775.
24. M. Saeed, Bernoulli mixture models for Markov blanket filtering and classification, in: JMLR Workshop and Conference Proceedings, WCCI 2008 Workshop on Causality, Hong Kong, vol. 3, 2008.
25. M. Saeed, Hybrid learning using mixture models and artificial neural networks. Hands-on Pattern Recognition Challenges in Machine Learning, vol. 1, Microtome Publishing, USA, 2011.
26. M. Saeed, The use of Bernoulli mixture models for identifying corners of a hypercube and extracting Boolean rules from data, in: JMLR Workshop and Conference Proceedings, NIPS 2008 Workshop on Causality: Objectives and Assessment, Canada, vol. 6, 2010, see 〈〉. http://jmlr.csail.mit.edu/proceedings/papers/v6/.
27. DeGroot M.H., Schervish M.J. Probability and Statistics 2002, Addison-Wesley.
28. Dempster A., Laird N., Rubin D. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. 1977, 39(1):1-38.
29. Schwarz G. Estimating the dimension of a model. Ann. Stat. 1978, 6(2):461-464.
30. An Y., Hu X., Xu L. A comparative investigation on model selection in independent factor analysis. J. Math. Model. Algorithms 2006, 5:447-473.
31. Xu L. Bayesian Ying-Yang system, best harmony learning, and five action circling: a special issue on Emerging themes on information theory and Bayesian approach. J. Front. Electr. Electron. Eng. China 2010, 5(3):281-328.
32. Xu L. On essential topics of BYY harmony learning. Current status, challenging issues, and gene analysis applications. J. Front. Electr. Electron. Eng. China 2012, 7(1):147-196.
33. H. Li, K. Zhang, T. Jiang, The regularized EM algorithm, in: Proceedings of the 20th National Conference on Artificial Intelligence, 2005, pp. 807-812.
34. Li S., Assmann1 S.M., Albert R. Predicting essential components of signal transduction networks. a dynamic model of guard cell abscisic acid signaling. PLoS Biol. 2006, 4(10):1732-1748.
35. J. Jenkins, SIGNET: Boolean rule determination for abscisic acid signaling, in: JMLR Workshop and Conference Proceedings, NIPS 2008 Workshop on Causality: Objectives and Assessment, Canada, 2010, see 〈〉. http://jmlr.csail.mit.edu/proceedings/papers/v6/.
36. I. Guyon, D. Janzing, B. Schölkopf, Causality: objectives and assessment, in: JMLR Workshop and Conference Proceedings, NIPS 2008 Workshop on Causality, vol. 6, 2008, pp. 1-38.
37. C. Zheng, Z. Geng, Reverse engineering of asynchronous Boolean networks via minimum explanatory set and maximum likelihood, in: JMLR Workshop and Conference Proceedings, NIPS 2008 Workshop on Causality: Objectives and Assessment, Canada, 2010, see 〈〉. http://jmlr.csail.mit.edu/proceedings/papers/v6/.
38. Javed K., Babri H.A., Saeed M. Feature selection based on class-dependent densities for high-dimensional binary data. IEEE Trans. Knowl. Data Eng. 2012, 24(3):465-477.
39. D. Koller, M. Sahami, Toward optimal feature selection, in: Machine Learning: Proceedings of the 13th International Conference, Morgan Kaufman, 1996.
40. I. Guyon, C. Aliferis, G. Cooper, A. Elissee, J.-P. Pellet, P. Spirtes, A. Statnikov, Design and analysis of the causation and prediction challenge, in: JMLR Workshop and Conference Proceedings, WCCI 2008 Workshop on Causality, Hong Kong, vol. 3, 2008.
41. I. WCCI, Causality Challenge #1: Causation and Prediction, see 〈〉 (2008). http://www.causality.inf.ethz.ch/challenge.php.
42. G. Cawley, Causal and non-causal feature selection for ridge regression, in: JMLR Workshop and Conference Proceedings, WCCI 2008 Workshop on Causality, Hong Kong, vol. 3, 2008.
43. I. Guyon, A. Saffari, G. Dror, G. Cawley, Agnostic learning vs. prior knowledge challenge, in: Proceedings of International Joint Conference on Neural Networks, 2007.
44. Guyon I., Saffari A., Dror G., Cawley G. Analysis of the IJCNN 2007 agnostic learning vs. prior knowledge challenge. Neural Networks 2008, 21(2-3):544-550.
45. M. Saeed, Hybrid Approach for Learning, 2007, fact sheet available at: 〈〉. http://clopinet.com/isabelle/Projects/agnostic/.
46. R.W. Lutz, Logitboost with trees applied to the WCCI 2006 performance prediction challenge datasets, in: Proceedings of International Joint Conference on Neural Networks, Vancouver, Canada, 2006, pp. 2966-2969, available at: 〈〉. http://stat.ethz.ch/~lutz/publ.
47. V. Franc, Modified Multi-class SVM Formulation; Efficient LOO Computation, 2007, fact sheet available at: 〈〉. http://clopinet.com/isabelle/Projects/agnostic/.