Principal composite kernel feature analysis: Data-dependent kernel approach

IEEE Transactions on Knowledge and Data Engineering

Volumn 25, Issue 8, 2013, Pages 1863-1875

  Motai, Yuichi ^a   Yoshida, Hiroyuki ^b  

^a Institute for Space and Nuclear Power Studies   (United States)

^b MASSACHUSETTS GENERAL HOSPITAL   (United States)

Author keywords

Data dependent kernel; Manifold structures; Nonlinear subspace; Principal component analysis

Indexed keywords

CLASSIFICATION PERFORMANCE; DATA-DEPENDENT KERNEL; FEATURE REPRESENTATION; FEATURE SELECTION ALGORITHM; HIGH-DIMENSIONAL FEATURE SPACE; KERNEL PRINCIPAL COMPONENT ANALYSES (KPCA); MANIFOLD STRUCTURES; NONLINEAR SUBSPACE;

ALGORITHMS; COMPUTER AIDED DIAGNOSIS; ITERATIVE METHODS;

PRINCIPAL COMPONENT ANALYSIS;

EID: 84897584292     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2012.110     Document Type: Article

References (77)

1. S. Winawer, R. Fletcher, D. Rex, J. Bond, R. Burt, J. Ferrucci, T. Ganiats, T. Levin, S. Woolf, D. Johnson, L. Kirk, S. Litin, and C. Simmang, "Colorectal Cancer Screening and Surveillance: Clinical Guidelines and Rationale-Update Based on New Evidence," Gastroenterology, vol. 124, pp. 544-560, Feb. 2003. (Pubitemid 36169269)
2. K.D. Bodily, J.G. Fletcher, T. Engelby, M. Percival, J.A. Christensen, B. Young, A.J. Krych, D.C. Vander Kooi, D. Rodysill, J.L. Fidler, and C.D. Johnson, "Nonradiologists as Second Readers for Intraluminal Findings at CT Colonography," Academic Radiology, vol. 12, pp. 67-73, Jan. 2005. (Pubitemid 40187289)
3. J.G. Fletcher, F. Booya, C.D. Johnson, and D. Ahlquist, "CT Colonography: Unraveling the Twists and Turns," Current Opinion in Gastroenterology, vol. 21, pp. 90-8, Jan. 2005. (Pubitemid 40189230)
4. H. Yoshida and A.H. Dachman, "CAD Techniques, Challenges, and Controversies in Computed Tomographic Colonography," Abdominal Imaging, vol. 30, pp. 26-41, Jan./Feb. 2005. (Pubitemid 40169505)
5. H. Yoshida and J. Nappi, "Three-Dimensional Computer-Aided Diagnosis Scheme for Detection of Colonic Polyps," IEEE Trans. Medical Imaging, vol. 20, no. 12, pp. 1261-1274, Dec. 2001. (Pubitemid 34125135)
6. R.M. Summers, C.F. Beaulieu, L.M. Pusanik, J.D. Malley, R.B. Jeffrey Jr., D.I. Glazer, and S. Napel, "Automated Polyp Detector for CT Colonography: Feasibility Study," Radiology, vol. 216, pp. 284-290, 2000. (Pubitemid 30434757)
7. R.M. Summers, M. Franaszek, M.T. Miller, P.J. Pickhardt, J.R. Choi, and W.R. Schindler, "Computer-Aided Detection of Polyps on Oral Contrast-Enhanced CT Colonography," Am. J. Roentgenology, vol. 184, pp. 105-108, Jan. 2005. (Pubitemid 40040912)
8. G. Kiss, J. Van Cleynenbreugel, M. Thomeer, P. Suetens, and G. Marchal, "Computer-Aided Diagnosis in Virtual Colonography via Combination of Surface Normal and Sphere Fitting Methods," European Radiology, vol. 12, pp. 77-81, Jan. 2002. (Pubitemid 36940028)
9. D.S. Paik, C.F. Beaulieu, G.D. Rubin, B. Acar, R.B. Jeffrey Jr., J. Yee, J. Dey, and S. Napel, "Surface Normal Overlap: A Computer-Aided Detection Algorithm with Application to Colonic Polyps and Lung Nodules in Helical CT," IEEE Trans. Medical Imaging, vol. 23, no. 6, pp. 661-675, June 2004.
10. A.K. Jerebko, R.M. Summers, J.D. Malley, M. Franaszek, and C.D. Johnson, "Computer-Assisted Detection of Colonic Polyps with CT Colonography Using Neural Networks and Binary Classification Trees," Medical Physics, vol. 30, pp. 52-60, Jan. 2003. (Pubitemid 36106008)
11. J. Nappi and H. Yoshida, "Fully Automated Three-Dimensional Detection of Polyps in Fecal-Tagging CT Colonography," Academic Radiology, vol. 14, pp. 287-300, Mar. 2007. (Pubitemid 46331875)
12. A.K. Jerebko, J.D. Malley, M. Franaszek, and R.M. Summers, "Multiple Neural Network Classification Scheme for Detection of Colonic Polyps in CT Colonography Data Sets," Academic Radiology, vol. 10, pp. 154-160, Feb. 2003. (Pubitemid 36150968)
13. A.K. Jerebko, J.D. Malley, M. Franaszek, and R.M. Summers, "Support Vector Machines Committee Classification Method for Computer-Aided Polyp Detection in CT Colonography," Academic Radiology, vol. 12, pp. 479-486, Apr. 2005.
14. V.N. Vapnik, The Nature of Statistical Learning Theory, second ed. Springer, 2000.
15. R. Courant and D. Hilbert, Methods of Mathematical Physics, Vol. 1 edition, Wiley-VCH, 1989.
16. O.L. Mangasarian, A.J. Smola, and B. Scholkopf, "Sparse Kernel Feature Analysis," Technical Report 99-04, Univ. of Wisconsin, 1999.
17. J. Franc and V. Hlavac, "Statistical Pattern Recognition Toolbox for Matlab," http://cmp.felk.cvut.cz/cmp/software/stprtool/, 2004.
18. "Partners Research Computing," http://www.partners.org/ rescomputing/, 2006.
19. A.J. Smola and B. Scholkopf, "Sparse Greedy Matrix Approximation for Machine Learning," Proc. 17th Int'l Conf. Machine Learning, 2000.
20. X. Jiang, R.R. Snapp, Y. Motai, and X. Zhu, "Accelerated Kernel Feature Analysis," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 109-116, 2006. (Pubitemid 44931319)
21. V. France and V. Hlavac, "Greedy Algorithm for a Training set Reduction in the Kernel Methods," Computer Analysis of Image and Patterns, vol. 2756, pp. 426-433, 2003.
22. K. Fukunaga and L. Hostetler, "Optimization of K-Nearest Neighbor Density Estimates," IEEE Trans. Information Theory, vol. IT-19, no. 3, pp. 320-326, May 1973.
23. J.H. Friedman, "Flexible Metric Nearest Neighbor Classification," technical report, Dept. of Statistics, Stanford Univ., Nov. 1994.
24. T. Hastie and R. Tibshirani, "Discriminant Adaptive Nearest Neighbor Classification," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 6, pp. 607-616, June 1996. (Pubitemid 126773896)
25. D.G. Lowe, "Similarity Metric Learning for a Variable-Kernel Classifier," Neural Computation, vol. 7, no. 1, pp. 72-85, 1995.
26. J. Peng, D.R. Heisterkamp, and H.K. Dai, "Adaptive Kernel Metric Nearest Neighbor Classification," Proc. 16th Int'l Conf. Pattern Recognition, vol. 3, pp. 33-36, 11-15 Aug. 2002. (Pubitemid 44844423)
27. Q.B. Gao and Z.Z. Wang, "Center-Based Nearest Neighbor Classifier," Pattern Recognition, vol. 40, pp. 346-349, 2007.
28. S. Li and J. Lu, "Face Recognition Using the Nearest Feature Line Method," IEEE Trans. Neural Networks, vol. 10, no. 2, pp. 439-443, Mar. 1999.
29. P. Vincent and Y. Bengio, "K-local Hyperplane and Convex Distance Nearest Neighbor Algorithms," Proc. Advances in Neural Information Processing Systems Conf., vol. 14, pp. 985-992, 2002.
30. W. Zheng, L. Zhao, and C. Zou, "Locally Nearest Neighbor Classifiers for Pattern Classification," Pattern Recognition, vol. 37, pp. 1307-1309, 2004. (Pubitemid 38549829)
31. T. Damoulas and M.A. Girolami, "Probabilistic Multi-Class Multi-Kernel Learning: On Protein Fold Recognition and Remote Homology Detection," vol. 24, no. 10, pp. 1264-1270, 2008.
32. J. Ye, S. Ji, and J. Chen, "Multi-Class Discriminant Kernel Learning Via Convex Programming," J. Machine Learning Research, vol. 9, pp. 719-758, 1999.
33. B. Souza and A. de Carvalho, "Gene Selection Based on Multi-Class Support Vector Machines and Genetic Algorithms," Molecular Research, vol. 4, no. 3, pp. 599-607, 2005.
34. B. Scholkopf and A.J. Smola, Learning with Kernels, pp. 211-214, MIT Press, 2002.
35. H. Xiong, Y. Zhang, and X.-W. Chen, "Data-Dependent Kernel Machines for Microarray Data Classification," IEEE/ACM Trans. Computational Biology and Bioinformatics, vol. 4, no. 4, pp. 583-595, Oct.-Dec. 2007. (Pubitemid 350125882)
36. H. Xiong, M.N.S. Swamy, and M.O. Ahmad, "Optimizing the Data-Dependent Kernel in the Empirical Feature Space," IEEE Trans. Neural Networks, vol. 16, no. 2, pp. 460-474, Mar. 2005. (Pubitemid 40390828)
37. G.C. Cawley, "MATLAB Support Vector Machine Toolbox, School of Information Systems, Univ. of East Anglia," http://theoval. sys.uea.ac.uk/~gcc/svm/toolbox, 2000.
38. Y. Raviv and N. Intrator, "Bootstrapping with Noise: An Efficient Regularization Technique," Connection Science, vol. 8, pp. 355-372, 1996.
39. H.A. Buchholdt, Structural Dynamics for Engineers. Thomas Telford, 1997.
40. R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. John Wiley & Sons Inc., 2001.
41. Pothin and Richard, "A Greedy Algorithm for Optimizing the Kernel Alignment and the Performance of Kernel Machines," Proc. 14th European Signal Processing Conf. (EUSIPCO '06), Sept. 2006.
42. J. Kandola, J. Shawe-Taylor, and N. Cristianini, "Optimizing Kernel Alignment over Combinations of Kernels," technical report NeuroCOLT, 2002.
43. M.E. Tipping, "Spare Kernel Principal Component Analysis," Proc. Neural Information Processing Systems Conf. (NIPS '00), pp. 633-639, 2000.
44. H. Frohlich, O. Chappelle, and B. Scholkopf, "Feature Selection for Support Vector Machines by Means of Genetic Algorithm," Proc. IEEE 15th Int'l Conf. Tools with Artificial Intelligence, pp. 142-148, 2003.
45. S. Mika, G. Ratsch, and J. Weston, "Fisher Discriminant Analysis with Kernels," Proc. Neural Networks for Signal Processing Workshop, pp. 41-48, 1999. (Pubitemid 30510358)
46. X.W. Chen, "Gene Selection for Cancer Classification Using Bootstrapped Genetic Algorithms and Support Vector Machines," Proc. IEEE Int'l Conf. Computational Systems, Bioinformatics, pp. 504-505, 2003.
47. C. Park and S.B. Cho, "Genetic Search for Optimal Ensemble of Feature-Classifier Pairs in DNA Gene Expression Profiles," Proc. Int'l Joint Conf. Neural Networks, vol. 3, pp. 1702-1707, 2003.
48. F.A. Sadjadi, "Polarimetric Radar Target Classification Using Support Vector Machines," Optical Eng., vol. 47, no. 4, p. 046201, Apr. 2008.
49. T. Briggs and T. Oates, "Discovering Domain Specific Composite Kernels" Proc. 20th Nat'l Conf. Artificial Intelligence and 17th Ann. Conf. Innovative Applications Artificial Intelligence, pp. 732-739, 2005.
50. N. Cristianini, J. Kandola, A. Elisseeff, and J. Shawe-Taylor, "On Kernel Target Alignment," Proc. Neural Information Processing Systems Conf. (NIPS '01), pp. 367-373.
51. M.A. Shipp, K.N. Ross, P. Tamayo, A.P. Weng, J.L. Kutok, R.C.T. Aguiar, M. Gaasenbeek, M. Angelo, M. Reich, G.S. Pinkus, T.S. Ray, M.A. Koval, K.W. Last, A. Norton, T.A. Lister, J. Mesirov, D.S. Neuberg, E.S. Lander, J.C. Aster, and T.R. Golub, "Diffuse Large B-Cell Lymphoma Outcome Prediction by Gene Expression Profiling and Supervised Machine Learning," Nature Medicine, vol. 8, pp. 68-74, 2002. (Pubitemid 34101731)
52. S. Dudoit, J. Fridlyand, and T.P. Speed, "Comparison of Discrimination Method for the Classification of Tumor Using Gene Expression Data," J. Am. Statistical Assoc., vol. 97, pp. 77-87, 2002.
53. C. Sotiriou et al., "Gene Expression Profiling in Breast Cancer: Understanding the Molecular Basis of Histologic Grade to Improve Prognosis," J. Nat'l Cancer Inst., vol. 98, no. 4, pp. 262-272, Feb. 2006. (Pubitemid 43264710)
54. U.R. Chandran et al., "Gene Expression Profiles of Prostate Cancer Reveal Involvement of Multiple Molecular Pathways in the Metastatic Process," BMC Cancer, pp. 7-64, Apr. 2007.
55. Y.P. Yu et al., "Gene Expression Alterations in Prostate Cancer Predicting Tumor Aggression and Preceding Development of Malignancy," J. Clinical Oncology, vol. 22, no. 14, pp. 2790-2799, July 2004. PMID: 15254046. (Pubitemid 41079896)
56. W. Zheng, C. Zou, and L. Zao, "An Improved Algorithm for Kernel Principal Component Analysis," Neural Processing Letters, vol. 22, no. 1, pp. 49-56, 2005. (Pubitemid 41156266)
57. C. Ding and I. Dubchak, "Multi-Class Protein Fold Recognition Using Support Vector Machines and Neural Networks," Bioinformatics, vol. 17, no. 4, pp. 349-358, Apr. 2001. (Pubitemid 32421929)
58. P. Pavlidis et al., "Learning Gene Functional Classifications from Multiple Data Types," J. Computational Biology, vol. 9, no. 2, pp. 401-411, 2002. (Pubitemid 34548272)
59. O. Chapelle et al., "Choosing Multiple Parameters for Support Vector Machines," Machine Learning, vol. 46, nos. 1-3, pp. 131-159, 2002. (Pubitemid 34129966)
60. C.S. Ong, A.J. Smola, and R.C. Williamson, "Learning the Kernel with Hyperkernels," J. Machine Learning Research, vol. 6, pp. 1043-1071, July 2005.
61. G.R.G. Lanckriet, N. Cristianini, P. Bartlett, L.E. Ghaoui, and M.I. Jordan, "Learning the Kernel Matrix with Semidefinite Programming," J. Machine Learning Research, vol. 5, pp. 27-72, 2004.
62. V. Atluri, S. Jajodia, and E. Bertino, "Transaction Processing in Multilevel Secure Databases with Kernelized Architecture: Challenges and Solutions," IEEE Trans. Knowledge and Data Eng., vol. 9, no. 5, pp. 697-708, Sept./Oct. 1997. (Pubitemid 127773153)
63. H. Cevikalp, M. Neamtu, and A. Barkana, "The Kernel Common Vector Method: A Novel Nonlinear Subspace Classifier for Pattern Recognition," IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 37, no. 4, pp. 937-951, Aug. 2007. (Pubitemid 47103944)
64. G. Horvath and T. Szabo, "Kernel CMAC With Improved Capability," IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 37, no. 1, pp. 124-138, Feb. 2007. (Pubitemid 46358487)
65. C. Heinz and B. Seeger, "Cluster Kernels: Resource-Aware Kernel Density Estimators over Streaming Data," IEEE Trans. Knowledge and Data Eng., vol. 20, no. 7, pp. 880-893, July 2008.
66. S. Amari and S. Wu, "Improving Support Vector Machine Classifiers by Modifying Kernel Functions," Neural Network, vol. 12, no. 6, pp. 783-789, 1999. (Pubitemid 29359218)
67. Y. Tan and J. Wang, "A Support Vector Machine with a Hybrid Kernel and Minimal Vapnik-Chervonenkis Dimension," IEEE Trans. Knowledge and Data Eng., vol. 16, no. 4, pp. 385-395, Apr. 2004.
68. A. Rakotomamonjy, F.R. Bach, S. Canu, and Y. Grandvalet, "SimpleMKL," J. Machine Learning Research, vol. 9, pp. 2491-2521, Nov. 2008.
69. M. Varma and B.R. Babu, "More Generality in Efficient Multiple Kernel Learning," Proc. Int'l Conf. Machine Learning, June 2009.
70. K. Duan, S.S. Keerthi, and A.N. Poo, "Evaluation of Simple Performance Measures for Tuning SVM Hyperparameters," Neurocomputing, vol. 51, pp. 41-59, Apr. 2003. (Pubitemid 36367224)
71. C. Cortes, M. Mohri, and A. Rostamizadeh, "L2 Regularization for Learning Kernels," Proc. 25th Conf. in Uncertainty in Artificial Intelligence, 2009.
72. D.J. Newman, S. Hettich, C.L. Blake, and C.J. Merz, UCI Repository of Machine Learning Databases, URL http://www.ics.uci.edu/mlearn/MLRepository.html. 1998.
73. K. Chaudhuri, C. Monteleoni, and A.D. Sarwate, "Differentially Private Empirical Risk Minimization," J. Machine Learning Research, vol. 12, pp. 1069-1109, Mar. 2011.
74. V. Vapnik and O. Chapelle, "Bounds on Error Expectation for Support Vector Machines," Neural Computation, vol. 12, no. 9, pp. 2013-2036, Sept. 2000.
75. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee, "Choosing Multiple Parameters for Support Vector Machines," Machine Learning, vol. 46, pp. 131-159, 2002. (Pubitemid 34129966)
76. K.M. Chung, W.C. Kao, C.-L. Sun, L.-L. Wang, and C.J. Lin, "Radius Margin Bounds for Support Vector Machines with the RBF Kernel," Neural Computation, vol. 15, no. 11, pp. 2643-2681, Nov. 2003. (Pubitemid 37206930)
77. J. Yang, Z. Jin, J.Y. Yang, D. Zhang, and A.F. Frangi, "Essence of Kernel Fisher Discriminant: KPCA Plus LDA," Pattern Recognition, vol. 37, pp. 2097-2100, 2004.