Evaluation of BIC and Cross Validation for model selection on sequence segmentations

International Journal of Data Mining and Bioinformatics

Volumn 4, Issue 6, 2010, Pages 675-700

  Haiminen, Niina ^a   Mannila, Heikki ^a  

^a UNIVERSITY OF HELSINKI   (Finland)

Author keywords

Bayesian information criterion; BIC; Binary; Categorical; Cross validation; CV; Genome; Likelihood; Model selection; Segmentation; Sequence

Indexed keywords

EID: 78650470934     PISSN: 17485673     EISSN: 17485681     Source Type: Journal    
DOI: 10.1504/IJDMB.2010.037547     Document Type: Article

References (46)

1. Akaike, H. (1973) 'Information theory and an extension of the maximum likelihood principle', Proc. 2nd Intl. Symposium on Information Theory, Akadémiai Kiadó, Budapest, Hungary, pp.267-281.
2. Andersson, S.G.E., Zomorodipour, A., Andersson, J.O., Sicheritz-Pontén, T., Cecilia, U., Alsmark, M., Podowskil, R.M., Näslund, A.K., Eriksson, A-S., Winkler, H.H. and Kurland, C.G. (1998) 'The genome sequence of Rickettsia prowazekii and the origin of mitochondria', Nature, Vol. 396, No. 6707, pp.133-140.
3. Barry, D. and Hartigan, J.A. (1993) 'A Bayesian analysis for change point problems', Journal of the American Statistical Association, Vol. 88, No. 421, pp.309-319.
4. Bellman, R. (1961) 'On the approximation of curves by line segments using dynamic programming', Communications of the ACM, Vol. 4, No. 6, p.284.
5. Berger, J.O., Ghosh, J.K. and Mukhopadhyay, N. (2003) 'Approximations and consistency of Bayes factors as model dimension grows', Journal of Statistical Planning and Inference, Vol. 112, Nos. 1-2, pp.241-258.
6. Bernaola-Galván, P., Grosse, I., Carpena, P., Oliver, J.L., Román-Roldán, R. and Stanley, H.E. (2000) 'Finding borders between coding and noncoding DNA regions by an entropic segmentation method', Physical Review Letters, Vol. 85, No. 6, pp.1342-1345.
7. Bernardi, G. (1989) 'The isochore organization of the human genome', Annual Reviews of Genetics, Vol. 23, pp.637-661.
8. Bernardo, J.M. and Smith, A.F.M. (1994) Bayesian Theory, Wiley, Chichester, England.
9. Braun, J.V. and Müller, H-G. (1998) 'Statistical methods for DNA sequence segmentation', Statistical Science, Vol. 13, No. 2, pp.142-162.
10. Burman, P. (1989) 'A comparative study of ordinary cross-validation, v-fold cross validation and the repeated learning-testing methods', Biometrika, Vol. 76, No. 3, pp.503-514.
11. Burnham, K. and Anderson, D.R. (1998) Model Selection and Inference: A Practical Information-Theoretic Approach, Springer-Verlag, New York.
12. Celeux, G. and Durand, J-B. (2006) Selecting Hidden Markov Chain States Number with Cross-Validated Likelihood, Technical Report 5877, Unité de recherche INRIA Futurs, Parc Club Orsay Université, France.
13. Churchill, G.A. (1989) 'Stochastic models for heterogeneous DNA sequences', Bulletin of Mathematical Biology, Vol. 51, No. 1, pp.79-94.
14. Costantini, M., Clay, O., Auletta, F. and Bernardi, G. (2006) 'An isochore map of human chromosomes', Genome Research, Vol. 16, No. 4, pp.536-541.
15. Fukagawa, T., Sugaya, K., Matsumoto, K-I., Okumura, K., Ando, A., Inoko, H. and Ikemura, T. (1995) 'A boundary of long-range G + C% mosaic domains in the human MHC locus: pseudoautosomal boundary-like sequence exists near the boundary', Genomics, Vol. 25, No. 1, pp.184-191.
16. Gutiérrez, G., Oliver, J.L. and Marín, A. (1993) 'Dinucleotides and G + C content in human genes: opposite behavior of GpG, GpC, and TpC at II-III codon positions and in introns', Journal of Molecular Evolution, Vol. 37, No. 2, pp.131-136.
17. Haiminen, N. and Mannila, H. (2007) 'Discovering isochores by least-squares optimal segmentation', Gene, Vol. 394, Nos. 1-2, pp.53-60.
18. Haiminen, N., Mannila, H. and Terzi, E. (2007) 'Comparing segmentations by applying randomization techniques', BMC Bioinformatics, Vol. 8, No. 171.
19. Hutter, M. (2007) 'Exact Bayesian regression of piecewise constant functions', Bayesian Analysis, Vol. 2, No. 1, pp.1-30.
20. International Human Genome Sequencing Consortium (2001) 'Initial sequencing and analysis of the human genome', Nature, Vol. 409, No. 6822, pp.860-921.
21. Jabbari, K. and Bernardi, G. (2004) 'Cytosine methylation and CpG, TpG (CpA) and TpA frequencies', Gene, Vol. 333, pp.143-149.
22. Lai, W.R., Johnson, M.D., Kucherlapati, R. and Park, P.J. (2005) 'Comparative analysis of algorithms for identifying amplifications and deletions in array CGH data', Bioinformatics, Vol. 21, No. 19, pp.3763-3770.
23. Li, W. (2001) 'DNA segmentation as a model selection process', RECOMB '01: Proc. 5th Intl. Conf. on Computational Biology, ACM Press, Montreal, Canada, pp.204-210.
24. Liu, J.S. and Lawrence, C.E. (1999) 'Bayesian inference on biopolymer models', Bioinformatics, Vol. 15, No. 1, pp.38-52.
25. Lyng, H., Lando, M., Brøvig, R.S., Svendsrud, D.H., Johansen, M., Galteland, E., Brustugun, O.T., Meza-Zepeda, L.A. Myklebost, O., Kristensen, G.B., Hovig, E. and Stokke, T. (2008) 'GeneCount: genome-wide calculation of absolute tumor DNA copy numbers from array CGH data', Genome Biology, Vol. 9, No. R86.
26. Macaya, G., Thiery, J.P. and Bernardi, G. (1976) 'An approach to the organization of eukaryotic genomes at a macromolecular level', Journal of Molecular Biology, Vol. 108, No. 1, pp.237-254.
27. McLachlan, G.J. (1987) 'On bootstrapping the likelihood ratio test stastistic for the number of components in a normal mixture', Applied Statistics, Vol. 36, No. 3, pp.318-324.
28. Mielikäinen, T., Terzi, E. and Tsaparas, P. (2006) 'Aggregating time partitions', Proc. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), Philadelphia, PA, USA, pp.347-356.
29. Mohammad-Djafari, A. and Féron, O. (2007) 'Bayesian approach to change points detection in time series', Intl. Journal of Imaging Systems and Technology, Vol. 16, No. 5, pp.215-221.
30. Mungall, A.J., Palmer, S.A., Sims, S.K., Edwards, C.A., Ashurst, J.L., Wilming, L., Jones, M.C., Horton, R., Hunt, S.E., Scott, C.E., Gilbert, J.G.R., Clamp, M.E., Bethel, G., Milne, S., Ainscough, R., Almeida, J.P., Ambrose, K.D., Andrews, T.D., Ashwell, R.I.S., Babbage, A.K., Bagguley, C.L., Bailey, J., Banerjee, R., Barker, D.J., Barlow, K.F., Bates, K., Beare, D.M., Beasley, H., Beasley, O., Bird, C.P., Blakey, S., Bray-Allen, S., Brook, J., Brown, A.J., Brown, J.Y., Burford, D.C., Burrill, W., Burton, J., Carder, C., Carter, N.P., Chapman, J.C., Clark, S.Y., Clark, G., Clee, C.M., Clegg, S., Cobley, V., Collier, R.E., Collins, J.E., Colman, L.K., Corby, N.R., Coville, G.J., Culley, K.M., Dhami, P., Davies, J., Dunn, M., Earthrowl, M.E., Ellington, A.E., Evans, K.A., Faulkner, L., Francis, M.D., Frankish, A., Frankland, J., French, L., Garner, P., Garnett, J., Ghori, M.J.R., Gilby, L.M., Gillson, C.J., Glithero, R.J., Grafham, D.V., Grant, M., Gribble, S., Griffiths, C., Griffiths, M., Hall, R., Halls, K.S., Hammond, S., Harley, J.L., Hart, E.A., Heath, P.D., Heathcott, R., Holmes, S.J., Howden, P.J., Howe, K.L., Howell, G.R., Huckle, E., Humphray, S.J., Humphries, M.D., Hunt, A.R., Johnson, C.M., Joy, A.A., Kay, M., Keenan, S.J., Kimberley, A.M., King, A., Laird, G.K., Langford, C., Lawlor, S., Leongamornlert, D.A., Leversha, M., Lloyd, C.R., Lloyd, D.M., Loveland, J.E., Lovell, J., Martin, S., Mashreghi-Mohammadi, M., Maslen, G.L., Matthews, L., McCann, O.T., McLaren, S.J., McLay, K., McMurray, A., Moore, M.J.F., Mullikin, J.C., Niblett, D., Nickerson, T., Novik, K.L., Oliver, K., Overton-Larty, E.K., Parker, A., Patel, R., Pearce, A.V., Peck, A.I., Phillimore, B., Phillips, S., Plumb, R.W., Porter, K.M., Ramsey, T., Ranby, S.A., Rice, C.M., Ross, M.T., Searle, S.M., Sehra, H.K., Sheridan, E., Skuce, C.D., Smith, S., Smith, M., Spraggon, L., Squares, S.L., Steward, C.A., Sycamore, N., Tamlyn-Hall, G., Tester, J., Theaker, A.J., Thomas, D.W., Thorpe, A., Tracey, A., Tromans, A., Tubby, B., Wall, M., Wallis, J.M., West, A.P., White, S.S., Whitehead, S.L., Whittaker, H., Wild, A., Willey, D.J., Wilmer, T.E., Wood, J.M., Wray, P.W., Wyatt, J.C., Young, L., Younger, R.M., Bentley, D.R., Coulson, A., Durbin, R., Hubbard, T., Sulston, J.E., Dunham, I., Rogers, J. and Beck, S. (2003) 'The DNA sequence and analysis of human chromosome 6', Nature, Vol. 425, No. 6960, pp.805-811. (Pubitemid 37351459)
31. Oliver, J.J., Baxter, R.A. and Wallace, C.S. (1998) 'Minimum message length segmentation', Proc. 2nd Pacific-Asia Conf. on Research and Development in Knowledge Discovery and Data Mining, Springer-Verlag, Melbourne, Australia, pp.222-233.
32. Oliver, J.L., Carpena, P., Hackenberg, M. and Bernaola-Galván, P. (2004) 'IsoFinder: computational prediction of isochores in genome sequences', Nucleic Acids Research, Vol. 32 (Web Server issue), pp.W287-W292.
33. Pauler, D.K. (1998) 'The Schwarz criterion and related methods for normal linear models', Biometrika, Vol. 85, No. 1, pp.13-27.
34. Picard, F., Robin, S., Lavielle, M., Vaisse, C. and Daudin, J-J. (2005) 'A statistical approach for array CGH analysis', BMC Bioinformatics, Vol. 6, No. 27.
35. Raftery, A.E., Newton, M.A., Satagopan, J.M. and Krivitsky, P.N. (2007) 'Estimating the integrated likelihood via posterior simulation using the harmonic mean identity', Bayesian Statistics 8: Proc. 8th Valencia International Meeting, Oxford University Press, Benidorm, Spain, pp.371-416.
36. Rissanen, J. (1978) 'Modelling by the shortest data description', Automatica, Vol. 14, pp.465-471.
37. Schwarz, G. (1978) 'Estimating the dimension of a model', The Annals of Statistics, Vol. 6, No. 2, pp.461-464.
38. Smyth, P. (1996) 'Clustering using Monte Carlo cross-validation', Proc. International Conference on Knowledge Discovery and Data Mining (KDD), AAAI, Portland, OR, USA, pp.126-133.
39. Smyth, P. (2000) 'Model selection for probabilistic clustering using cross-validated likelihood', Statistics and Computing, Vol. 9, pp.63-72.
40. Snijders, A.M., Nowak, N., Segraves, R., Blackwood, S., Brown, N., Conroy, J., Hamilton, G., Hindle, A.K., Huey, B., Kimura, K., Law, S., Myambo, K., Palmer, J., Ylstra, B., Yue, J.P., Gray, J.W., Jain, A.N., Pinkel, D. and Albertson, D.G. (2001) 'Assembly of microarrays for genome-wide measurement of DNA copy number', Nature Genetics, Vol. 29, No. 3, pp.263-264.
41. Spiegelhalter, D.J., Best, N.G., Carlin, B.P. and van der Linde, A. (2002) 'Bayesian measures of model complexity and fit', Journal of the Royal Statistical Society: Series B, Vol. 64, No. 4, pp.583-639.
42. Stone, M. (1974) 'Cross-validatory choice and assessment of statistical predictions', Journal of the Royal Statistical Society Series B, Vol. 36, No. 2, pp.111-147.
43. Terzi, E. and Tsaparas, P. (2006) 'Efficient algorithms for sequence segmentation', Proc. 6th SIAM Intl. Conference on Data Mining, Society for Industrial Mathematics, Bethesda, MD, pp.316-327.
44. Vasko, K. and Toivonen, H. (2002) 'Estimating the number of segments in time series data using permutation tests', Proc. 2002 IEEE Intl. Conf. on Data Mining, IEEE Computer Society, Maebashi, Japan, pp.466-473.
45. Wallace, C.S. and Boulton, D.M. (1968) 'An information measure for classification', Computer Journal, Vol. 11, No. 2, pp.185-194.
46. Weakliem, D.L. (1999) 'A critique of the Bayesian information criterion for model selection', Sociological Methods and Research, Vol. 27, No. 3, pp.359-397.