Algorithms for discovery of multiple Markov boundaries

Journal of Machine Learning Research

Volumn 14, Issue 1, 2013, Pages 499-566

  Statnikov, Alexander ^a   Lytkin, Nikita I ^a   Lemeire, Jan ^b,c   Aliferis, Constantin F ^a  

^a NEW YORK UNIVERSITY   (United States)

^b VRIJE UNIVERSITEIT BRUSSEL   (Belgium)

^c Interdisciplinary Institute for Broadband Technology   (Belgium)

Author keywords

Information equivalence; Markov boundary discovery; Variable feature selection; Violations of faithfulness

Indexed keywords

EXTENSIVE BENCHMARKING; INFORMATION EQUIVALENCE; MARKOV BOUNDARY; SELECTION PROBLEMS; STATE-OF-THE-ART ALGORITHMS; VARIABLE/FEATURE SELECTION; VIOLATIONS OF FAITHFULNESS;

BENCHMARKING;

ALGORITHMS;

EID: 84875199703     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (65)

1. C. F. Aliferis, I. Tsamardinos, and A. Statnikov. Large-scale feature selection using markov blanket induction for the prediction of protein-drug binding. Technical Report DSL 02-06, 2002.
2. C. F. Aliferis, I. Tsamardinos, and A. Statnikov. Hiton: a novel markov blanket algorithm for optimal variable selection. AMIA 2003 Annual Symposium Proceedings, pages 21-25, 2003a.
3. C. F. Aliferis, I. Tsamardinos, A. Statnikov, and L. E. Brown. Causal explorer: a causal probabilistic network learning toolkit for biomedical discovery. Proceedings of the 2003 International Conference on Mathematics and Engineering Techniques in Medicine and Biological Sciences (METMBS), 2003b.
4. C. F. Aliferis, A. Statnikov, I. Tsamardinos, S. Mani, and X. D. Koutsoukos. Local causal and markov blanket induction for causal discovery and feature selection for classification. part i: Algorithms and empirical evaluation. Journal of Machine Learning Research, 11:171-234, 2010a.
5. C. F. Aliferis, A. Statnikov, I. Tsamardinos, S. Mani, and X. D. Koutsoukos. Local causal and markov blanket induction for causal discovery and feature selection for classification. part ii: Analysis and extensions. Journal of Machine Learning Research, 11:235-284, 2010b.
6. T. W. Anderson. An Introduction to Multivariate Statistical Analysis, volume 3rd of Wiley Series in Probability and Statistics. Wiley-Interscience, Hoboken, N. J, 2003.
7. Y. Aphinyanaphongs, A. Statnikov, and C. F. Aliferis. A comparison of citation metrics to machine learning filters for the identification of high quality medline documents. J. Am. Med. Inform. Assoc., 13(4):446-455, 2006. (Pubitemid 43947624)
8. L. Breiman. Statistical modeling: the two cultures. Statistical Science, 16(3):199-215, 2001.
9. L. E. Brown, I. Tsamardinos, and D. Hardin. To feature space and back: Identifying top-weighted features in polynomial support vector machine models. Intelligent Data Analysis, 16(4), 2012.
10. T. P. Conrads, V. A. Fusaro, S. Ross, D. Johann, V. Rajapakse, B. A. Hitt, S. M. Steinberg, E. C. Kohn, D. A. Fishman, G. Whitely, J. C. Barrett, L. A. Liotta, III Petricoin, E. F., and T. D. Veenstra. High-resolution serum proteomic features for ovarian cancer detection. Endocr. Relat Cancer, 11(2):163-178, 2004. (Pubitemid 38832824)
11. T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley New York, 1991.
12. N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, Cambridge, 2000.
13. E. R. De Long, D. M. De Long, and D. L. Clarke-Pearson. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44(3):837-845, 1988.
14. E. Dougherty and M. Brun. On the number of close-to-optimal feature sets. Cancer Informatics, 2:189-196, 2006. (Pubitemid 44440832)
15. L. Ein-Dor, I. Kela, G. Getz, D. Givol, and E. Domany. Outcome signature genes in breast cancer: is there a unique set? Bioinformatics, 21(2):171-178, 2005. (Pubitemid 40202018)
16. L. Ein-Dor, O. Zuk, and E. Domany. Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer. Proc. Natl. Acad. Sci. U. S. A., 103(15):5923-5928, 2006.
17. R. E. Fan, P. H. Chen, and C. J. Lin. Working set selection using second order information for training support vector machines. Journal of Machine Learning Research, 6(1889):1918, 2005.
18. T. Fawcett. Roc graphs: Notes and practical considerations for researchers. Technical Report, HPL-2003-4, HP Laboratories, 2003.
19. F. Fleuret. Fast binary feature selection with conditional mutual information. Journal of Machine Learning Research, 5:1531-1555, 2004.
20. D. P. Foster and R. A. Stine. Variable selecion in data mining: Building a predictive model for bankruptcy. Journal of the American Statistical Association, 99(466):303-314, 2004.
21. C. N. Glymour and G. F. Copper. Computation, Causation and Discovery. AAAI Press, Menlo Park, Calif, 1991.
22. P. I. Good. Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses, volume 2nd of Springer series in statistics. Springer, New York, 2000.
23. A. Gopnik and L. Schulz. Causal Learning: Psychology, Philosophy, and Computation. Oxford University Press, Oxford, 2007.
24. I. Guyon and A. Elisseeff. An introduction to variable and feature selection. Journal of Machine Learning Research, 3(1):1157-1182, 2003.
25. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik. Gene selection for cancer classification using support vector machines. Machine Learning, 46(1):389-422, 2002. (Pubitemid 34129977)
26. I. Guyon, S. Gunn, M. Nikravesh, and L. A. Zadeh. Feature Extraction: Foundations and Applications. Studies in fuzziness and soft computing. Springer-Verlag, Berlin, 2006.
27. B. Hammer and K. Gersmann. A note on the universal approximation capability of support vector machines. Neural Processing Letters, 17(1):43-53, 2003.
28. M. Hollander and D. Wolfe. Nonparametric statistical methods, volume 2nd of Wiley Series in Probability and Statistics. Wiley, New York, NY, USA, 1999.
29. T. Joachims. Learning to Classify Text Using Support Vector Machines. Kluwer international series in engineering and computer science. Kluwer Academic Publishers, Boston, 2002.
30. R. Kohavi and G. H. John. Wrappers for feature subset selection. Artificial Intelligence, 97(1-2):273-324, 1997. (Pubitemid 127401107)
31. J. Lemeire. Learning Causal Models of Multivariate Systems and the Value of It for the Performance Modeling of Computer Programs. PhD thesis, 2007.
32. J. Lemeire, S. Meganck, and F. Cartella. Robust independence-based causal structure learning in absence of adjacency faithfulness. Proccedings of the Fifth European Workshop on Probabilistic Graphical Models (PGM 2010), 2010.
33. H. Liu, L. Liu, and H. Zhang. Ensemble gene selection by grouping for microarray data classification. J. Biomed. Inform., 43(1):81-87, 2010.
34. H. Liu, L. Liu, and H. Zhang. Ensemble gene selection for cancer classification. Pattern Recognition, 43(8):2763-2772, 2010b.
35. S. Mani and G. F. Cooper. A study in causal discovery from population-based infant birth and death records. Proceedings of the AMIA Annual Fall Symposium, 319, 1999.
36. S. Mani and G. F. Cooper. Causal discovery using a bayesian local causal discovery algorithm. Medinfo 2004, 11(Pt 1):731-735, 2004.
37. S. Michiels, S. Koscielny, and C. Hill. Prediction of cancer outcome with microarrays: a multiple random validation strategy. Lancet, 365(9458):488-492, 2005. (Pubitemid 40215731)
38. G. Natsoulis, Ghaoui L. El, G. R. Lanckriet, A. M. Tolley, F. Leroy, S. Dunlea, B. P. Eynon, C. I. Pearson, S. Tugendreich, and K. Jarnagin. Classification of a large microarray data set: algorithm comparison and analysis of drug signatures. Genome Res., 15(5):724-736, 2005. (Pubitemid 41043643)
39. R. E. Neapolitan. Learning Bayesian Networks. Prentice Hall series in artificial intelligence. Pearson Prentice Hall, Upper Saddle River, NJ, 2004.
40. J. Peña, R. Nilsson, J. Björkegren, and J. Tegnér. Towards scalable and data efficient learning of markov boundaries. International Journal of Approximate Reasoning, 45(2):211-232, 2007. (Pubitemid 46880195)
41. J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. The Morgan Kaufmann series in representation and reasoning. Morgan Kaufmann Publishers, San Mateo, California, 1988.
42. J. Pearl. Causality: Models, Reasoning, and Inference, volume 2nd. Cambridge University Press, Cambridge, U. K, 2009.
43. J. P. Pellet and A. Elisseeff. Using markov blankets for causal structure learning. The Journal of Machine Learning Research, 9:1295-1342, 2008.
44. A. Pinkus. Approximation theory of the mlp model in neural networks. Acta Numerica, 8:143-195, 1999.
45. J. Ramsey, J. Zhang, and P. Spirtes. Adjacency-faithfulness and conservative causal inference. Proceedings of the 22nd Annual Conference on Uncertainty in Artificial Intelligence (UAI-2006), pages 401-408, 2006.
46. T. Richardson and P. Spirtes. Automated Discovery of Linear Feedback Models. MIT Press, Menlo Park, CA, 1999.
47. T. S. Richardson and P. Spirtes. Ancestral graph markov models. Annals of Statistics, 30(4):962-1030, 2002. (Pubitemid 37095336)
48. P. Roepman, P. Kemmeren, L. F. Wessels, P. J. Slootweg, and F. C. Holstege. Multiple robust signatures for detecting lymph node metastasis in head and neck cancer. Cancer Res., 66(4):2361-2366, 2006.
49. A. Rosenwald, G. Wright, W. C. Chan, J. M. Connors, E. Campo, R. I. Fisher, R. D. Gascoyne, H. K. Muller-Hermelink, E. B. Smeland, J. M. Giltnane, E. M. Hurt, H. Zhao, L. Averett, L. Yang, W. H. Wilson, E. S. Jaffe, R. Simon, R. D. Klausner, J. Powell, P. L. Duffey, D. L. Longo, T. C. Greiner, D. D. Weisenburger, W. G. Sanger, B. J. Dave, J. C. Lynch, J. Vose, J. O. Armitage, E. Montserrat, A. Lopez-Guillermo, T. M. Grogan, T. P. Miller, M. Le Blanc, G. Ott, S. Kvaloy, J. Delabie, H. Holte, P. Krajci, T. Stokke, and L. M. Staudt. The use of molecular profiling to predict survival after chemotherapy for diffuse large-b-cell lymphoma. N. Engl. J. Med., 346(25):1937-1947, 2002. (Pubitemid 34651353)
50. F. Scarselli and A. Chung Tsoi. Universal approximation using feedforward neural networks: A survey of some existing methods, and some new results. Neural Networks, 11(1):15-37, 1998. (Pubitemid 28123730)
51. B. Schölkopf, C. J. C. Burges, and A. J. Smola. Advances in Kernel Methods: Support Vector Learning. MIT Press, Cambridge, Mass, 1999.
52. J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge, UK, 2004.
53. R. L. Somorjai, B. Dolenko, and R. Baumgartner. Class prediction and discovery using gene microarray and proteomics mass spectroscopy data: curses, caveats, cautions. Bioinformatics, 19(12):1484-1491, 2003. (Pubitemid 37045318)
54. P. Spirtes, C. N. Glymour, and R. Scheines. Causation, Prediction, and Search, volume 2nd of Adaptive computation and machine learning. MIT Press, Cambridge, Mass, 2000.
55. A. Statnikov and C. F. Aliferis. Analysis and computational dissection of molecular signature multiplicity. PLoS Computational Biology, 6(5):e1000790, 2010a.
56. A. Statnikov and C. F. Aliferis. TIED: An Artificially Simulated Dataset with Multiple Markov Boundaries. Journal of Machine Learning Research Workshop and Conference Proceedings, Volume 6: Causality: Objectives and Assessment (NIPS 2008), 6:249-256, 2010b.
57. A. Statnikov, C. F. Aliferis, I. Tsamardinos, D. Hardin, and S. Levy. A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis. Bioinformatics, 21(5):631-643, 2005. (Pubitemid 40424789)
58. A. Statnikov, I. Tsamardinos, L. E. Brown, and C. F. Aliferis. Causal Explorer: A Matlab Library of Algorithms for Causal Discovery and Variable Selection for Classification. In Challenges in Machine Learning. Volume 2: Causation and Prediction Challenge. Edited by Guyon, I. and Aliferis, C. F. and Cooper, G. F. and Elisseeff, A. and Pellet, J. P. and Spirtes, P. and Statnikov, A. Microtome Publishing, Bookline, Massachusetts, 2010.
59. I. Tsamardinos and C. F. Aliferis. Towards principled feature selection: relevancy, filters and wrappers. Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics (AI and Stats), 2003.
60. I. Tsamardinos and L. E. Brown. Markov blanket-based variable selection in feature space. Technical Report DSL-08-01, 2008.
61. I. Tsamardinos, C. F. Aliferis, and A. Statnikov. Algorithms for large scale markov blanket discovery. Proceedings of the Sixteenth International Florida Artificial Intelligence Research Society Conference (FLAIRS), pages 376-381, 2003a.
62. I. Tsamardinos, C. F. Aliferis, and A. Statnikov. Time and sample efficient discovery of markov blankets and direct causal relations. Proceedings of the Ninth International Conference on Knowledge Discovery and Data Mining (KDD), pages 673-678, 2003b.
63. V. N. Vapnik. Statistical Learning Theory. Adaptive and learning systems for signal processing, communications, and control. Wiley, New York, 1998.
64. Y. Wang, J. G. Klijn, Y. Zhang, A. M. Sieuwerts, M. P. Look, F. Yang, D. Talantov, M. Timmermans, M. E. Meijer-Van Gelder, J. Yu, T. Jatkoe, E. M. Berns, D. Atkins, and J. A. Foekens. Geneexpression profiles to predict distant metastasis of lymph-node-negative primary breast cancer. Lancet, 365(9460):671-679, 2005. (Pubitemid 40260888)
65. S. M. Weiss and C. A. Kulikowski. Computer Systems that Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning, and Expert Systems. M. Kaufmann Publishers, San Mateo, Calif, 1991.