Bayesian classification and feature reduction using uniform Dirichlet priors

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 33, Issue 3, 2003, Pages 448-464

  Lynch Jr , Robert S ^a   Willett, Peter K ^b  

^a NAVAL UNDERSEA WARFARE CENTER   (United States)

^b UNIVERSITY OF CONNECTICUT   (United States)

Author keywords

Class specific features; Discrete features; Feature selection; Neural networks; Noninformative prior; UCI repository

Indexed keywords

ALGORITHMS; CLASSIFICATION (OF INFORMATION); NEURAL NETWORKS; PROBABILITY;

FEATURE REDUCTION;

DATA REDUCTION;

EID: 0038321342     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2003.811121     Document Type: Article

References (60)

1. F. M. Alkoot and J. Kittler, "Multiple expert system design by combined feature selection and probability level fusion," in Proc. Int. Conf. Information Fusion, July 2000.
2. P. M. Baggenstoss, "Class-specific feature sets in classification," IEEE Trans. Signal Processing, vol. 47; pp. 3428-3432, Dec. 1999.
3. S. Basu and N. Ebrahimi, "Estimating the number of undetected errors: Bayesian model selection," in Proc. Ninth Int. Symp. Software Engineering, 1998, pp. 22-31.
4. S. D. Bay, "Nearest neighbor classification from multiple feature subsets," Intell. Data Anal., vol. 34, no. 3, pp. 191-209, Aug. 1999.
5. Y. Bar-Shalom and X. Li, Multitarget-Multisensor Tracking: Principles and Techniques. Storrs: Univ. Connecticut, 1995, Course Notes.
6. E. Bauer and R. Kohavi, "An empirical comparison of voting classifi-cation algorithms: Bagging, boosting, and variants," Machine Learning, vol. 36, pp. 105-142, 1999.
7. R. Bellman, Adaptive Control Processes: A Guided Tour. Princeton, NJ: Princeton Univ. Press, 1961.
8. J. M. Bernardo and A. F. M. Smith, Bayesian Theory. New York: Wiley, 1994.
9. C. M. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Clarendon, 1995.
10. A. L. Blum and P. Langley, "Selection of relevant features and examples in machine learning," in Proc. Artificial Intelligence, 1997, pp. 245-271.
11. C. G. E. Boender and A. H. G. Rinnooy Kan, "A multinomial Bayesian approach to the estimation of population and vocabulary size," Biometrika, vol. 74, no. 4, pp. 849-856, 1987.
12. L. J. Buturović, "Toward Bayes-optimal linear dimension reduction," IEEE Trans. Pattern Anal. Machine Intell., vol. 16, pp. 420-423, Apr. 1994.
13. W. L. Buntine, "Learning classification trees," Statist. Comput., vol. 2, pp. 63-73, AUTHOR: What year was this published?.
14. L. L. Campbell, "Averaging entropy," IEEE Trans. Inform. Theory, vol. 41, pp. 338-339, Jan. 1995.
15. G. Casella and R. L. Berger, Statistical Inference. Belmont, CA: Duxbury, 1990.
16. B. P. Carlin and T. A. Louis, Bayes and Empirical Bayes Methods for Data Analysis. London, U.K.: Chapman & Hall, 1996.
17. M. Delampady and J. O. Berger, "Lower bounds on Bayes factors for multinomial distributions, with applications to chi-square tests of fit," Ann. Statist, vol. 18, no. 3, pp. 1295-1316, 1990.
18. H. Demuth and M. Beale, Neural Network Toolbox. Natick, MA: The Math Works, Inc., 1994.
19. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
20. P. Domingos, "Two way induction," in Proc. Int. Conf. Tools With Artificial Intelligence, Nov. 1995.
21. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
22. Y. Ephraim, "Statistical-model-based speech enhancement systems," Proc. IEEE, vol. 80, no. 10, pp. 1526-1555, Oct. 1992.
23. K. Fukunaga, Statistical Pattern Recognition. Boston, MA: Academic, 1990.
24. K. Fukunaga and R. R. Hayes, "Effects of sample size in classifier design," IEEE Trons. Pattern Anal. Machine Intell., vol. 11, pp. 873-885, Aug. 1989.
25. J. Gama, L. Toro, and C. Scares, "Dynamic discretization of continuous attributes," in Proc. 6th Ibero-Amer. Conf. Artificial Intell., 1998.
26. R. Hanson, J. Stutz, and P. Cheeseman, "Bayesian classification theory," NASA Ames Res. Center Tech. Rep., no. FIA-90-12-7-01, Dec. 1990.
27. S. Haykin, Neural Networks: A Comprehensive Foundation, Second Edition. Upper Saddle River, NJ: Prentice-Hall, 1999.
28. J. P. Hoffbeck and D. A. Landgrebe, "Covariance matrix estimation and classification with limited training data," IEEE Trans. Pattern Anal. Machine Intell., vol. 18, pp. 763-767, July 1996.
29. G. F. Hughes, "On the mean accuracy of statistical pattern recognizers," IEEE Trans. Inform. Theory, vol. 14, pp. 55-63, Jan. 1968.
30. Q. Huo, H. Jiang, and C. Lee, "A Bayesian predictive classification approach to robust speech recognition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Apr. 1997, pp. 1547-1550.
31. Q. Huo and C. Lee, "A Bayesian predictive classification approach to robust speech recognition," IEEE Trans. Speech Audio Processing, vol. 8, pp. 200-204, Mar. 2000.
32. A. Jain and D. Zongker, "Feature selection: Evaluation, application, and small sample size performance," IEEE Trans. Pattern Anal. Machine Intell., vol. 19, pp. 153-158, Feb. 1997.
33. H. Jiang, K. Hirose, and Q. Huo, "Improving Viterbi Bayesian predictive classification via sequential Bayesian learning in robust recognition," Speech Commun., vol. 28, no. 4, pp. 313-326, Aug. 1999.
34. J. Kittler, "Feature set search algorithms," in Pattern Recognition and Signal Processing, C. H. Chen, Ed. Alphen aan den Rijn, The Netherlands: Sijthoff and Noordhoff, 1978, pp. 41-60.
35. Q. Li and D. W. Tufts, "Principal feature classification," IEEE Trans. Neural Networks, vol. 8, pp. 155-160, Jan. 1997.
36. T. Lim, W. Loh, and Y. Shih, "A comparison of prediction accuracy, complexity, and naming time of thirty-three old and new classification algorithms," Machine Learning J., 1999.
37. H. Liu and R. Setiono, "Feature selection via discretization," IEEE Trans. Knowledge Data Eng., vol. 9, pp. 642-645, July/Aug. 1997.
38. R. S. Lynch, Jr. and P. K. Willett, "Utilizing a class labeling feature in an adaptive Bayesian classifier," in Proc. SPIE Int. Conf. AeroSense, Apr. 2001.
39. -."Adaptive Bayesian classification using noninformative Dirichlet priors," in Proc. IEEE Int. Conf. Systems, Man, Cybern., Oct. 2000.
40. R. S. Lynch, Jr., "Bayesian classification using noninformative Dirichlet priors," Ph.D. dissertation, Univ. Connecticut, Storrs, May 1999.
41. R. S. Lynch, Jr. and P. K. Willett, "Bayesian classification using misla beled training data and a noninformative prior," J. Franklin Inst., July 1999.
42. -, "Performance considerations for a combined information classification test using Dirichlet priors," IEEE Trans. Signal Processing, vol. 47, pp. 1711-1715, June 1999.
43. N. Merhav and Y. Ephraim, "A Bayesian classification approach with application to speech recognition," IEEE Trans. Acoust., Speech, Signal Processing, vol. 39, pp. 2157-2166, Oct. 1991.
44. C. J. Merz and P. M. Murphy, "UCI repository of machine learning databases." Dept. Inform. Comput Sci., Univ. California, Irvine, CA, 1996.
45. J. E. Mosimann, "On the compound multinomial distribution, the multivariate beta-distribution, and correlation's among proportions," Biometrika, vol. 49, no. 1, pp. 65-82, 1962.
46. K. L. Oehler and R. M. Gray, "Combining image compression and classification using vector quantization," IEEE Trans. Pattern Anal. Machine Intell., vol. 17, pp. 461-473, May 1995.
47. M. Padmanabhan and L. R. Bahl, "Model complexity adaptation using a discriminative measure," IEEE Trans. Speech Audio Processing, vol. 8, pp. 205-208, Mar. 2000.
48. K. Peleg and U. Ben Hanan, "Adaptive classification by neural net based prototype populations," Int. J. Pattern Recognition Artificial Intell., vol. 7, no. 4, pp. 917-933, Aug. 1993.
49. N. S. Philip and K. B. Joseph, "Boosting the differences: A fast Bayesian classifier neural network," Intell. Data Anal, vol. 4, no. 6, pp. 463-473, 2000.
50. W. Pieczynski, J. Bouvrais, and C. Michel, "Estimation of generalized mixture in the case of correlated sensors," IEEE Trans. Image Processing, vol. 9, pp. 308-312, Feb. 2000.
51. S. Puuronen, V. Terziyan, A. Katasonov, and A. Tsymbal, "Dynamic integration of multiple data mining techniques in a knowledge discovery management system," in Proc. SPIE Conf. Data Mining Knowledge Discovery, Apr. 1999.
52. H. Qiang, "Adaptive learning and compensation of hidden Markov model for robust speech recognition," Commun. COLIPS, vol. 8, no. 2, pp. 161-189, Dec. 1998.
53. B. D. Ripley, Pattern Recognition and Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1996.
54. G. Schwartz, "Estimating the dimensions of a model," Ann. Statist., vol., 6, pp. 461-464, 1978.
55. B. M. Shahshahani and D. A. Landgrebe, "The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon," IEEE Trans. Geosci. Remote Sensing, vol. 32, pp. 1087-1095, Sept. 1994.
56. H. Wang, D. Bell, and F. Murtagh, "Axiomatic approach to feature subset selection based on relevance," IEEE Trans. Pattern Anal. Machine Intell., vol. 21, pp. 271-277, Mar. 1999.
57. F. Wen, P. Willett, and S. Deb, "Condition monitoring of helicopter data," in Proc. IEEE Int. Conf. Syst., Man, Cybern., Oct. 2000.
58. Q. Xie, C. A. Laszlo, and R. K. Ward, "Vector quantization technique for nonparametric classifier design," IEEE Trans. Pattern Anal. Machine Intell., vol. 15. pp. 1326-1329, Dec. 1993.
59. X. Yao and Y. Liu, "A new evolutionary system for evolving artificial neural networks," IEEE Trans. Neural Networks, vol. 8, pp. 694-713, May 1997.
60. G. P. Zhang, "Neural networks for classification: A survey," IEEE Trans. Syst., Man, Cybern., vol. 30, no. 4, pp. 451-462, Nov. 2000.