Discriminant Analysis for Fast Multiclass Data Classification Through Regularized Kernel Function Approximation

IEEE Transactions on Neural Networks

Volumn 21, Issue 6, 2010, Pages 1020-1029

  Ghorai, Santanu ^a,b   Mukherjee, Anirban ^a   Dutta, Pranab K ^a  

^a INDIAN INSTITUTE OF TECHNOLOGY   (India)

^b MCKV INSTITUTE OF ENGINEERING   (India)

Author keywords

Discriminant analysis; Function approximation; Multiclass data classification; Support vector machine (SVM)

Indexed keywords

BENCHMARK DATA; BIAS VECTORS; CANCER CLASSIFICATION; COMPARISON OF PERFORMANCE; DATA CLASSIFICATION; FEATURE SPACE; FUNCTION APPROXIMATION; GENE MICROARRAYS; KERNEL FUNCTION; LABEL SPACE; LARGE DATASETS; LINEAR OPERATORS; LOW DIMENSIONAL; MULTI-CLASS; MULTICLASS SUPPORT VECTOR MACHINES; MULTICLASS SVM; SINGLE-STEP; STRATIFIED RANDOM SAMPLING; SUB-SAMPLING; TEST PATTERN; TESTING ACCURACY; TRAINING AND TESTING; VECTOR VALUED;

DISCRIMINANT ANALYSIS; LABELS; MATHEMATICAL OPERATORS; SUPPORT VECTOR MACHINES; VECTORS;

VECTOR SPACES;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; DISCRIMINANT ANALYSIS; HUMAN; INFORMATION PROCESSING; INFORMATION RETRIEVAL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; AUTOMATIC DATA PROCESSING; DISCRIMINANT ANALYSIS; HUMANS; INFORMATION STORAGE AND RETRIEVAL;

EID: 77953092633     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2046646     Document Type: Article

References (43)

1. E. L. Allwein, R. E. Schapire, and Y. Singer, "Reducing multiclass to binary: A unifying approach for margin classifiers," J. Mach. Learn. Res., vol.1, pp. 113-141, Dec. 2000.
2. C. Ambroise and G. McLachlan, "Selection bias in gene extraction on the basis of microarray gene-expression data," Proc. Nat. Acad. Sci. USA, vol.99, no.10, pp. 6562-6566, May 2002.
3. S. A. Armstrong, J. E. Staunton, L. B. Silverman, R. Pieters, M. L. den Boer, M. D. Minden, S. E. Sallan, E. S. Lander, T. R. Golub, and S. J. Korsmeyer, "MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia," Nature Gen., vol.30, no.1, pp. 41-47, 2002.
4. A. Bhattacharjee, W. G. Richards, J. Staunton, C. Li, S. Monti, P. Vasa, C. Ladd, J. Beheshti, R. Bueno, M. Gillette, M. Loda, G. Weber, E. J. Mark, E. S. Lander, W. Wong, B. E. Johnson, T. R. Golub, D. J. Sugarbaker, and M. Meyerson, "Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclasses," Proc. Nat. Acad. Sci. USA, vol.98, no.24, pp. 13790-13795, Nov. 2001.
5. C. L. Blake and C. J. Merz, UCI Repository for Machine Learning Databases, Dept. Inf. Comput. Sci., Univ. California, Irvine, CA, 1998 [Online]. Available: http://www.ics.uci.edu/~mlearn/MLRepository. html
6. L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard, and V. Vapnik, "Comparison of classifier methods: A case study in handwriting digit recognition," in Proc. IEEE Int. Conf. Pattern Recognit., Jerusalem, Israel, Oct. 1994, pp. 77-87.
7. L. Breiman, J. Friedman, C. J. Stone, and R. A. Olshen, Classification and Regression Trees. New York: Chapman & Hall, 1993.
8. C.-C. Chang and C.-J. Lin, LIBSVM: A Library for Support Vector Machines, 2001 [Online]. Available: http://www.csie.ntu.edu.tw/ ~cjlin/libsvm
9. P.-C. Chen, T.-J. Lee, Y.-J. Lee, and S.-Y. Huang, "Multiclass support vector classification via regression," Tech. Rep. [Online]. Available: http://www.stat.sinica.edu.tw/syhuang/papersdownload/ MC-reg-121006.pdf
10. K. Crammer and Y. Singer, "On the learnability and design of output codes for multiclass problems," Mach. Learn., vol.47, no.2-3, pp. 201-233, 2002.
11. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge, MA: Cambridge Univ. Press, 2000.
12. R. O. Duda, P. E. Hart, and D. J. Stork, Pattern Classification, 2nd ed. Singapore: Wiley, 2006.
13. G. M. Fung and O. L. Mangasarian, "Multicategory proximal support vector machine classifiers," Mach. Learn., vol.59, no.1-2, pp. 77-97, 2005.
14. S. Ghorai, S. J. Hossain, A. Mukherjee, and P. K. Dutta, "Newton's method for nonparallel plane proximal classifier," Signal Process., vol.90, no.1, pp. 93-104, 2010.
15. S. Ghorai, A. Mukherjee, and P. K. Dutta, "Fast regularized kernel function approximation," in Proc. IEEE TENCON, Hyderabad, India, Nov. 2008, DOI: 10.1109/TENCON.2008.4766523.
16. S. Ghorai, A. Mukherjee, and P. K. Dutta, "Nonparallel plane proximal classifier," Signal Process., vol.89, no.4, pp. 510-522, 2009.
17. S. Ghorai, A. Mukherjee, S. Sengupta, and P. K. Dutta, "Cancer classification from gene expression data by NPPC ensemble," IEEE/ACM Trans. Comput. Biol. Bioinf., accepted for publication.
18. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, "Gene selection for cancer classification using support vector machines," Mach. Learn., vol.46, no.1-3, pp. 389-422, 2002.
19. T. Hastie, A. Buja, and R. Tibshirani, "Penalized discriminant analysis," Ann. Stat., vol.23, no.1, pp. 73-102, 1995.
20. T. Hastie, R. Tibshirani, and A. Buja, "Flexible discriminant analysis by optimal scoring," J. Amer. Stat. Assoc., vol.89, no.428, pp. 1255-1270, 1994.
21. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, Prediction, 2nd ed. New York: Springer-Verlag, 2009.
22. C.-W. Hsu and C.-J. Lin, "A comparison of methods for multi-class support vector machines," IEEE Trans. Neural Netw., vol.13, no.2, pp. 415-425, Mar. 2002.
23. T. Joachims, "A statistical learning model of text classification with support vector machines," in Proc. Conf. Res. Develop. Inf. Retrieval, New Orleans, LA, Sep. 2001, pp. 128-136 [Online]. Available: http:// doi.acm.org/10.1145/383952.383974
24. J. Khan, J. S.Wei, and M. Ringner, "Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks," Nature Med., vol.7, no.6, pp. 673-679, 2001.
25. U. Kreßel, "Pair wise classification and support vector machines," in Advances in Kernel Methods-Support Vector Learning, B. Schölkopf, C. J. C. Burges, and A. J. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 255-268.
26. Y.-J. Lee, W.-F. Hsieh, and C.-M. Huang, " SSVR:Asmooth support vector machine for insensitive regression," IEEE Trans. Knowl. Data Eng., vol.17, pp. 678-685, May 2005.
27. Y. J. Lee and S. Y. Huang, "Reduced support vector machines: A statistical theory," IEEE Trans. Neural Netw., vol.18, no.1, pp. 1-13, Jan. 2007.
28. Y.-J. Lee and O. L. Mangasarian, "RSVM: Reduced Support Vector Machines," in Proc. 1st SIAM Int. Conf. Data Mining, Chicago, IL, Apr. 2001 [Online]. Available: ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/ 00-07.ps
29. L. Li, C. R.Weinberg, T. A. Darden, and L. G. Pedersen, "Gene selection for sample classification based on gene expression data: Study of sensitivity to choice of parameters of the GA/KNN method," Bioinformatics, vol.17, no.12, pp. 1131-1142, 2001.
30. K.-M. Lin and C.-J. Lin, "A study on reduced support vector machines," IEEE Trans. Neural Netw., vol.14, no.6, pp. 1449-1459, Nov. 2003.
31. D. Michie, D. J. Spiegelhalter, and C. C. Taylor, Eds., Machine Learning, Neural and Statistical Classification. Upper Saddle River, NJ: Ellis Horwood, 1994 [Online]. Available: ftp://ftp.ncc.up.pt/pub/statlog/, ISBN 0-13-106360-X
32. T. M. Mitchell, Machine Learning. Singapore: McGraw-Hill, 1997.
33. C. L. Nutt, D. R. Mani, R. A. Betensky, P. Tamayo, J. G. Cairncross, C. Ladd, U. Pohl, C. Hartmann, M. E. McLaughlin, T. T. Batchelor, P.M. Black, A. von Deimling, S. L. Pomeroy, T. R. Golub, and D. N. Louis, "Gene expression-based classification of malignant gliomas correlates better with survival than histological classification," Cancer Res., vol.63, no.7, pp. 1602-1607, 2003.
34. H. Peng, F. Long, and C. Ding, "Feature selection on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy, " IEEE Trans. Pattern Anal. Mach. Intell., vol.27, no.8, pp. 1226-1238, Aug. 2005.
35. J. C. Platt, N. Cristianini, and J. Shawe-Taylor, "Large margin DAGs for multiclass classification," in Advances in Neural Information Processing Systems, S.A. Solla, T. K. Leen, and K.-R. Müller, Eds. Cambridge, MA: MIT Press, 2000, pp. 547-553.
36. J. R. Quinlan, C4.5: Programs for Machine Learning. San Francisco, CA: Morgan Kaufmann, 1993.
37. A. J. Smola and B. Schölkopf, "Sparse greedy matrix approximation for machine learning," in Proc. 17th Int. Conf. Mach. Learn., San Francisco, CA, Jun. 2000, pp. 911-918.
38. J. A. K. Suykens and J. Vandewalle, "Least squares support vector machine classifiers," Neural Process. Lett., vol.9, no.3, pp. 293-300, 1999.
39. S. Szedmak and J. Shawe-Taylor, "Multiclass learning at one-class complexity," Schl. Electron. Comput. Sci., Univ. Southampton, Southampton, U. K., Tech. Rep. 1508, Aug. 2005.
40. A. N. Tikhonov and V. Y. Arsenin, Solution of Ill-Posed Problems. Washington, DC: V. H. Winston, 1977.
41. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
42. V. Vapnik, S. Golowich, and A. J. Smola, "Support vector method for function approximation, regression estimation, and signal processing," in Advances in Neural Information Processing Systems 9, M. Mozer, M. Jordan, and T. Petsche, Eds. Cambridge, MA: MIT Press, 1997, pp. 281-287.
43. J. Weston and C. Watkins, "Multi-class support vector machines," in Proc. 7th Eur. Symp. Artif. Neural Netw., M. Verleysen, Ed., Brussels, Belgium, 1999, pp. 219-224.