An SVM classifier incorporating simultaneous noise reduction and feature selection: Illustrative case examples

Pattern Recognition

Volumn 38, Issue 1, 2005, Pages 41-49

  Kumar, R ^a   Jayaraman, V K ^a   Kulkarni, B D ^a  

^a NATIONAL CHEMICAL LABORATORY   (India)

Author keywords

Classification; Conditional entropy; SVM; Symbolization

Indexed keywords

ALGORITHMS; ENTROPY; FEATURE EXTRACTION; PRINCIPAL COMPONENT ANALYSIS; SET THEORY; STATISTICAL METHODS;

CONDITIONAL ENTROPY; LINEAR DISCRIMINATION ANALYSIS; SUPPORT VECTOR MACHINES (SVM); SYMBOLIZATION;

PATTERN RECOGNITION;

EID: 4644267885     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2004.06.002     Document Type: Article

References (61)

1. S. Markovitch, Information filtering: selection mechanisms in learning systems, Ph.D. Thesis, EECS Department, University of Michigan, 1989.
2. G.H. John, Enhancements to the data mining process, Ph.D. Thesis, Computer Science Department, School of Engineering, Stanford University, 1997.
3. R. Kohavi, G. John, Wrappers for feature subset selection, Artificial Intel, (special issue on relevance) 97 (1-2) (1996) 273-324.
4. R. Kohavi, D. Sommerfield, Feature subset selection using the wrapper method: overfitting and dynamic search space topology, in: Proceedings of the First International Conference on Knowledge Discovery and Data Mining, AAAI Press1995.
5. D. Koller, M. Sahami, Toward optimal feature selection, in: Proceedings of the 13th International Conference on Machine Learning (ML), Bari, Italy, 1996, pp. 284-292.
6. P. Langley, Elements of Machine Learning, in: B.M. Morgan (Ed.), Morgan Kaufmann, Los Altos, CA, 1996.
7. P. Langley, S. Sage, Induction of selective Bayesian Classifiers, in: Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence, Seattle, WA, Morgan Kaufmann, Los Altos, CA, 1994, pp. 399-406.
8. H. Liu, H. Motoda, Feature Extraction Construction and Selection, A Data Mining Perspective, Kluwer Academic Publisher, Norwell, MA, 1998.
9. A.L. Rendell, R. Sheshu, Learning hard concepts through constructive induction: framework and rationale, Comput. Intel. 6 (1990) 247-270.
10. R.O. Duda, P.E. Hart, Classification and Scene Analysis, Wiley, New York, 1973.
11. A. Jain, B. Chandrasekaran, Dimensionality and sample size considerations, in: Krishnaiah, I.N. Kanal (Eds.), Pattern Recognition Practice, Vol. 2, North-Holland, Amsterdam, 1982, pp. 835-855.
12. K. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press, New York, 1972.
13. A. Mucciardi, E.E. Gose, A comparison of seven techniques for choosing subsets of pattern recognition properties, IEEE Trans. Comput. 20 (9) (1971) 1023-1031.
14. J.M. Steppe, K.W. Bauer, Improved feature screening in feedforward neural networks, Neurocomputing 13 (1996) 47-58.
15. C.J. Matheus, L.A. Rendell, Constructive induction on decision trees, in: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI),1989, pp. 645-650.
16. G.H. John, R. Kohavi, K. Pfleger, Irrelevant features and the subset selection problem, in: International Conference on Machine Learning, 1994, pp. 121-129.
17. P. Langely, Selection of relevant features in machine learning, in: AAAI Fall Symposium on Relevance, 1994, pp. 140-144.
18. R. Kohavi, Wrappers for performance enhancement and oblivious decision graphs, Ph.D. Thesis, Stanford University, 1995.
19. P.A. Devijver, J. Kittler, Pattern Recognition: A Statistical Approach, Prentice-Hall, Englewood Cliffs, NJ, 1982.
20. R. Caruana, D. Freitag, Greedy attribute selection, in: Proceedings of the 11th International Conference on Machine Learning, New Brunswick, NJ, USA, 1994, pp. 28-36.
21. D.W. Aha, R.L. Bankert, A comparative evaluation of sequential feature selection algorithms, in: D. Fisher, H. Lenz (Eds.), Proceedings of the 5th International Workshop on Artificial Intelligence and Statistics, Ft. Lauderdale, FL, 1995, pp. 1-7.
22. S. Kirkpatrick, C. Gelatt, M. Vecci, Optimization by simulated annealing, Science 220 (1983) 671-680.
23. J. Doak, An evaluation of feature selection methods and their application to computer security, CSE Technical Report 92-18, University of California at Davis, 1992.
24. D.E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley Pub. Co., 1989.
25. H. Almauallium, T.G. Dietterich, Learning with many irrelevant features, Proceedings of Ninth National Conference on Artificial Intelligence, Vol. 2, AAAI Press, Anaheim, CA, 1991, pp. 547-552.
26. D. Koller, M. Sahami, Toward optimal feature selection, in: Proceedings of the 13th International Conference on Machine Learning (ML), Bari, Italy, 1996, pp. 284-292.
27. K. Kira, L.A. Rendell, A practical approach to feature selection, in: D. Sleeman, J. Edwards (Eds.), Proceedings of Ninth International Conference on Machine Learning, Alberdeen, Italy, Morgan Kaufmann, Los Altos, CA, 1992, pp. 249-256.
28. I. Kononenko, Estimating attributes: analysis and extensions of RELIEF, in: European Conference on Machine Learning, 1994, pp. 171-182.
29. C. Cardie, Using decision trees to improve case-based learning, in: Proceedings of the Tenth International Conference on Machine Learning, Amherst, MA, Morgan Kaufmann, Los Altos, CA, 1993, pp. 25-32.
30. H. Liu, H. Setiono, A probabilistic approach to feature selection: a filter solution, machine learning, in: Proceedings of the 13th International Conference on Machine Learning, Morgan Kaufmann, Los Altos, CA, 1996.
31. A.L. Blum, R.L. Rivest, Training a 3-node neural network is NP-complete, Neural Networks 5 (1992) 117-127.
32. L. Hyafil, R.L. Rivest, Constructing optimal binary decision trees is NP-complete, Informat. Process. Lett. 5 (1) (1976) 15-17.
33. R.K. Azad, J.S. Rao, R. Ramaswamy, Information-entropic analysis of chaotic time series: determination of time-delays and dynamical coupling, Chaos Solitons Fractals 14 (2002) 633-641.
34. M. Lehrman, A.B. Rechester, R.B. White, Symbolic analysis of chaotic signals and turbulent fluctuations, Phys. Rev. Lett. 78 (1) (1997) 54-57.
35. C. Cortes, V. Vapnik, Support vector networks, Machine Learn. 20 (1995) 273-297.
36. V. Vapnik, Statistical Learning Theory, Wiley, New York, 1998.
37. Abhijit Kulkarni, V.K. Jayaraman, B.D. Kulkami, Support vector classification with parameter tuning assisted by agent based technique, Comput. Chem. Eng. 28 (2004) 311-318.
38. G. Brida, L.F. Punzo, Symbolic time series analysis and economic regimes, IDEE Working papers series, 2001, pp. 2001-03.
39. C.E.A. Finney, J.B. Green, C.S. Daw, Symbolic time series analysis of engine combustion measurements, SAE paper no. 980624, 1998.
40. E.M. Bollt, T. Stanford, Y.C. Lai, K. Zyczkowski, Validity of threshold-crossing analysis of symbolic dynamics from chaotic time series, Phys. Rev. Lett. 85 (16) (2000) 3524-3527.
41. E.M. Bollt, T. Stanford, Y.C. Lai, K. Zyczkowski, What symbolic dynamics do we get with a misplaced partition?: on the validity of threshold crossings analysis of chaotic time-series, Physica D 154 (2001) 259-286.
42. J. Kurths, A. Voss, P. Saparin, A. Witt, H.J. Kleiner, N. Wessel, Quantitative analysis of heart-rate variability, Chaos 5 (1) (1995) 88-94.
43. U. Schwarz, O. Benz, J. Kurths, A. Witt, Analysis of the solar spike events by means of symbolic dynamics methods, Astronomy Astrophys. 277 (1993) 215-224.
44. X.Z. Tang, E.R. Tracy, Data compression and information retrieval via symbolization, Chaos 8 (3) (1998) 688-696.
45. X.Z. Tang, E.R. Tracy, A.D. Boozer, A. DeBrauw, R. Brown, Symbol sequence statistics in noisy chaotic signal reconstruction, Phys. Rev. E 51 (5) (1995) 3871-3889.
46. M.P.S. Brown, W.N. Grundy, D. Lin, N. Cristianini, C.W. Sugnet, T.S. Furey, M. Ares, D. Haussler, Knowledge-based analysis of microarray gene expression data by using support vector machines, Proc. Natl. Acad. Sci. USA 97 (2000) 2-267.
47. R. Burbidge, M. Trotter, B. Buxton, S. Holden, Drug design by machine learning: support vector machines for pharmaceutical data analysis, Comput. Chem. 26 (2001) 5-14.
48. Y.D. Caia, S.L. Linb, Support vector machines for predicting rRNA-, RNA-, and DNA-binding proteins from amino acid sequence, Biochim. Biophys. Acta (BBA) - Proteins Proteomics 1648 (1-2) (2003) 127-133.
49. S. Hua, Z. Sun, A novel method of protein secondary structure prediction with high segment overlap measure: support vector machine approach, J. Mol. Biol. 308 (2001) 397-407.
50. Y.F. Sun, X.D. Fan, Y.D. Li, Identifying splicing sites in eukaryotic RNA: support vector machine approach, Comput. Biol. Med. 33 (1) (2003) 17-29.
51. C.J.C. Burges, A tutorial on support vector machines for pattern recognition, Data Min. Knowledge Discovery 2 (1998) 121-167.
52. N. Cristianini, J. Shawe-Taylor, An Introduction to Support Vector Machines, Cambridge University Press, Cambridge, UK, 2000.
53. S. Gunn, Support vector machines for classification and regression, ISIS Group Technical Report, University of Southampton, 1998.
54. J. Weston, C. Watkins, Support vector machines for multiclass pattern recognition, in: M. Verleysen (Ed.), Proceedings of the Seventh ESANN, D. Facto Press, Brussels, 1999, pp. 219-224.
55. J. Zhu, T. Hastie, Kernel logistic regression and the import vector machine, in: Advances in Neural Information Processing Systems, Vancouver, British Columbia, Canada, 2001, pp. 1081-1088.
56. O. Chapelle, V. Vapnik, O. Bousquet, S. Mukherjee, Choosing multiple parameters for support vector machines, Machine Learn. 46 (2001) 131-160 Online version is at: http://www-connex.lip6.fr/~chapelle/.
57. T. Joachims, Estimating the generalization performance of a SVM efficiently, in: Proceedings of the International Conference on Machine Learning, Morgan Kaufman, Los Altos, CA, 2000.
58. K. Duan, S.S. Keerthi, A.N. Poo, Evaluation of simple performance measures for tuning SVM hyperparameters, Neurocomputing 51 (2003) 41-59.
59. S.S. Keerthi, Efficient tuning of SVM hyperparameters using radius/ margin bound and iterative algorithms, IEEE Trans. Neural Networks 13 (2002) 1225-1229.
60. T.R Edgar, D.M. Himmelblau, Optimization of Chemical Processes, McGraw-Hill, New York, 1988.
61. G.V. Reklaitis, A. Ravindran, K.M. Ragsdell, Engineering Optimization Methods and Applications, Wiley, New York, 1983.