Beyond traditional kernels: Classification in two dissimilarity-based representation spaces

IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews

Volumn 38, Issue 6, 2008, Pages 729-744

  Pekalska, Elzbieta ^a   Duin, Robert P W ^b  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

^b Delft University of Technology ^*  (Netherlands)

Author keywords

Classifier design and evaluation; Indefinite kernels; Similarity measures; Statistical learning

Indexed keywords

CLASSIFIERS; COMPUTATIONAL COMPLEXITY; KETONES; LEARNING SYSTEMS;

CLASSIFIER DESIGN AND EVALUATION; DEGREE OF SIMILARITIES; DISSIMILARITY MEASURES; EUCLIDEAN; INDEFINITE KERNELS; KERNEL METHODS; NEAREST NEIGHBORS; NN RULES; REPRESENTATION SPACES; SIMILARITY MEASURES; STATISTICAL LEARNING; TRAINING OBJECTS;

EDUCATION;

EID: 56049108351     PISSN: 10946977     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCC.2008.2001687     Document Type: Article

References (80)

1. M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Comput., vol. 15, no. 6, pp. 1373-1396, 2002.
2. C. Berg, J. Christensen, and P. Ressel, Harmonic Analysis on Semigroups. New York: Springer-Verlag, 1984.
3. J. Bognár, Indefinite Inner Product Spaces. New York: Springer-Verlag, 1974.
4. I. Borg and P. Groenen, Modern Multidimensional Scaling. New York: Springer-Verlag, 1997.
5. H. Bunke, S. Günter, and X. Jiang, "Towards bridging the gap between statistical and structural pattern recognition: Two new concepts in graph matching," in Proc. Adv. Pattern Recognit., 2001, pp. 1-11.
6. H. Bunke and A. Sanfeliu, Eds., Syntactic and Structural Pattern Recognition Theory and Applications. Singapore: World Scientific, 1990.
7. S. Canu, C. Ong, and X. Mary, "Splines with non positive kernels," in Proc. Int. ISAAC Congr, 2005, pp. 1-10.
8. A. Cayley, "On the theorem in the geometry of position," Cambridge Math. J., vol. 2, pp. 267-271, 1841.
9. Y. Cheng, "Mean shift, mode seeking, and clustering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, no. 8, pp. 790-799, Aug. 1995.
10. F. Corpet, F. Servant, J. Gouzy, and D. Kahn, "ProDom and ProDom-CG: Tools for protein domain analysis and whole genome comparisons," Nucleid Acids Res., vol. 28, pp. 267-269, 2000.
11. P. Courrieu, "Straight monotonic embedding of data sets in Euclidean spaces," Neural Netw., vol. 15, pp. 1185-1196, 2002.
12. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge, UK: Cambridge Univ. Press, 2000.
13. B. Dasarthy, Ed., Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. Los Alamitos, CA: IEEE Computer Society Press, 1991.
14. D. de Ridder and R. Duin, "Sammon's mapping using neural networks: A comparison," Pattern Recognit. Lett., vol. 18, no. 11-13, pp. 1307-1316, 1997.
15. P. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach. Englewood Cliffs, NJ: Prentice-Hall, 1982.
16. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
17. M. M. Deza and M. Laurent, Geometry of Cuts and Metrics. New York: Springer-Verlag, 1997.
18. C. Domeniconi, J. Peng, and D. Gunopulos, "Locally adaptive metric nearest-neighbor classification," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 9, pp. 1281-1285, Sep. 2002.
19. D. Donoho and C. Grimes, "Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data," in Proc. PNAS, 2003, vol. 100, pp. 5591-5596.
20. M. Dubuisson and A. Jain, "Modified Hausdorff distance for object matching," in Proc. Int. Conf. Pattern Recognit., 1994, vol. 1, pp. 566-568.
21. R. Duda, P. Hart, and D. Stork, Pattern Classification, 2nd ed. New York: Wiley, 2001.
22. R. Duin, D. de Ridder, and D. Tax, "Experiments with object based discriminant functions; a featureless approach to pattern recognition," Pattern Recognit. Lett., vol. 18, no. 11-13, pp. 1159-1166, 1997.
23. R. Duin and Pȩkalska, "The science of pattern recognition. Achievements and perspectives," in Challenges for Computational Intelligence, W. Duch and J. Mańdziuk, Eds. New York: Springer-Verlag, 2007, pp. 221-259.
24. S. Edelman, S. Cutzu, and S. Duvdevani-Bar, "Representation is representation of similarities," Behavioral Brain Sci., vol. 21, pp. 449-498, 1998.
25. F. Esposito, D. Malerba, V. Tamma, H. Bock, and F. Lisi, "Similarity and dissimilarity," in Analysis of Symbolic Data. New York: Springer-Verlag, 2000.
26. C. Faloutsos and K.-I. Lin, "FastMap: A fast algorithm for indexing, datamining and visualization of tradditional and multimedia datasets," in Proc. ACM SIGMOD, 1995, pp. 163-174.
27. L. Goldfarb, "A new approach to pattern recognition," in Progress in Pattern Recognition, vol. 2, L. Kanal and A. Rosenfeld, Eds. Amsterdam, The Netherlands: Elsevier, 1985, pp. 241-402.
28. L. Goldfarb, D. Gay, O. Golubitsky, and D. Korkin, "What is a structural representation? A proposal for a representational formalism," Univ. New Brunswick, Fredericton, Canada, Tech. Rep., 2006.
29. R. Goldstone and J. Son, "Similarity," in Cambridge Handbook of Thinking and Reasoning, K. Holyoak and R. Morrison, Eds. Cambridge, UK: Cambridge Univ. Press, 2005, pp. 13-36.
30. J. Goodman and J. O'Rourke, in Handbook of Discrete and Computational Geometry, J. Goodman and J. O'Rourke, Eds. Boca Raton, FL: CRC Press, 2004.
31. J. Gower, "Euclidean distance geometry," Math. Sci., vol. 7, pp. 1-14, 1982.
32. J. Gower, "Metric and Euclidean properties of dissimilarity coefficients," J. Classification, vol. 3, pp. 5-48, 1986.
33. T. Graepel, R. Herbrich, P. Bollmann-Sdorra, and K. Obermayer, "Classification on pairwise proximity data," in Proc. Adv. Neural Inf. Syst. Process, 1999, pp. 438-444.
34. T. Graepel, R. Herbrich, B. Schölkopf, A. Smola, P. Bartlett, K.-R. Müller, K. Obermayer, and R. Williamson, "Classification on proximity data with LP-machines," in Proc. Int. Conf. Artif. Neural Netw., 1999, pp. 304-309.
35. B. Haasdonk, "Feature space interpretation of SVMs with indefinite kernels," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 5, pp. 482-492, Apr. 2005.
36. H. Haasdonk and B. Burkhardt, "Invariant kernels for pattern analysis and machine learning," Mach. Learn., vol. 68, pp. 35-61, 2007.
37. P. Hart, "The condensed nearest neighbor rule," IEEE Trans. Inf. Theory, vol. IT-14, no. 3, pp. 515-516, May 1968.
38. T. Hastie and R. Tibshirani, "Discriminant adaptive nearest neighbor classification and regression," in Advances in Neural Information Processing Systems, vol. 8. Cambridge, MA: MIT Press, 1996, pp. 409-415.
39. S. Hochreiter and K. Obermayer, "Support vector machines for dyadic data," Neural Comput., vol. 18, no. 6, pp. 1472-1510, 2006.
40. Y. Huang, C. Chiang, J. Shieh, and W. Grimson, "Prototype optimization for nearest-neighbor classification," Pattern Recognit., vol. 35, no. 6, pp. 1237-1245, 2002.
41. D. Jacobs, D. Weinshall, and Y. Gdalyahu, "Classification with non-metric distances: Image retrieval and class representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 6, pp. 583-600, Jun. 2000.
42. A. Jain and D. Zongker, "Representation and recognition of handwritten digits using deformable templates," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 12, pp. 1386-1391, Dec. 1997.
43. J. Laub and K.-R. Müller, "Feature discovery in non-metric pairwise data," J. Mach. Learn. Res., vol. 5, pp. 801-818, 2004.
44. C. Leslie, E. Eskin, A. Cohen, J. Weston, and W. Noble, "Mismatch string kernels for discriminative protein classification," Bioinformatics, vol. 20, no. 4, pp. 467-476, 2004.
45. D. Lowe, "Similarity metric learning for a variable-kernel classifier," Neural Comput., vol. 7, no. 1, pp. 72-85, 1995.
46. M. Lozano, J. Sotoca, J. Sanchez, F. Pla, Pȩkalska, and R. Duin, "Experimental study on prototype optimization algorithms for prototype-based classification," Pattern Recognit., vol. 39, pp. 1827-1838, 2006.
47. F. Lu, S. Keles, Y. Lin, S. Wright, and G. Wahba, "Kernel regularization and dimension reduction," presented at the ASA Statistical Comput. Graph. Sections, Joint Statistical Meet, Seattle, WA, 2006.
48. K. Menger, "New foundation of Euclidean geometry," Amer. J. Math. vol. 53, pp. 721-745, 1931.
49. H. Minh and T. Hofmann, "Learning over compact metric spaces," in Proc. Conf. Learn. Theory, 2004, pp. 239-254.
50. R. Mollineda, F. Ferri, and E. Vidal, "An efficient prototype merging strategy for the condensed 1-NN rule through class-conditional hierarchical clustering," Pattern Recognit., vol. 35, no. 12, pp. 2771-2782, 2002.
51. V. Mottl, S. Dvoenko, O. Seredin, C. Kulikowski, and I. Muchnik, "Featureless regularized recognition of protein fold classes in a Hilbert space of pairwise alignment scores as inner products of amino acid sequences," Pattern Recognit. Image Anal., vol. 11, no. 3, pp. 597-615, 2001.
52. C. Ong, X. Mary, S. Canu, and A. J. Smola, "Learning with non-positive kernels," in Proc. Int. Conf. Mach. Learn, 2004, pp. 639-646.
53. P. Paclík and R. Duin, "Dissimilarity-based classification of spectra: Computational issues," Real Time Imag., vol. 9, no. 4, pp. 237-244, 2003.
54. P. Paclík, J. Novovičová, P. Somol, and P. Pudil, "Road sign classification using Laplace kernel classifier," Pattern Recognit. Lett., vol. 21, no. 13-14, pp. 1165-1173, 2000.
55. R. Paredes and E. Vidal, "A class-dependent weighted dissimilarity measure for nearest neighbor classification problems," Pattern Recognit. Lett., vol. 21, no. 12, pp. 1027-1036, 2000.
56. E. Pȩkalska, D. de Ridder, R. Duin, and M. Kraaijveld, "A new method of generalizing Sammon mapping with application to algorithm speed-up," in Proc. Adv. School Comput. Imag., 1999, pp. 221-228.
57. E. Pȩkalska and R. Duin, The Dissimilarity Representation for Pattern Recognition. Foundations and Applications. Singapore: World Scientific, 2005.
58. E. Pȩkalska and R. Duin, "Dissimilarity-based classification for vectorial representations," in Proc. Int. Conf. Pattern Recognit, 2006, vol. 3, pp. 137-140.
59. E. Pȩkalska, R. Duin, S. Günter, and H. Bunke, "On not making dissimilarities Euclidean," in Proc. Joint IAPR Int. Workshops SSPR SPR, 2004, pp. 1145-1154.
60. E. Pȩkalska, R. Duin, and P. Paclík, "Prototype selection for dissimilarity-based classifiers," Pattern Recognit., vol. 39, no. 2, pp. 189-208, 2006.
61. E. Pȩkalska, P. Paclík, and R. Duin, "A generalized kernel approach to dissimilarity based classification," J. Mach. Learn. Res., vol. 2, no. 2, pp. 175-211, 2002.
62. V. Roth, J. Laub, J. Buhmann, and K.-R. Müller, "Going metric: Denoising pairwise data," in Proc. Adv. Neural Inf. Process. Syst. 2003, pp. 841-856.
63. V. Roth, J. Laub, M. Kawanabe, and J. Buhmann, "Optimal cluster preserving embedding of nonmetric proximity data," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 12, pp. 1540-1551, Dec. 2003.
64. J. Rovnyak, "Methods of Krein space operator theory," Oper. Theory: Adv. Appl., vol. 134, pp. 31-66, 2002.
65. L. Saul and S. Roweis, "Think globally, fit locally: Unsupervised learning of low-dimensional manifolds," J. Mach. Learn. Res., vol. 4, pp. 119-155, 2003.
66. I. Schoenberg, "On certain metric spaces arising from Euclidean spaces by a change of metric and their imbedding in Hilbert space," Ann. Math., vol. 38, pp. 787-797, 1937.
67. B. Schölkopf, "The kernel trick for distances," in Proc. Adv. Neural Inf. Process. Syst, 2000, pp. 301-307.
68. J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge, UK: Cambridge Univ. Press, 2004.
69. P. Simard, Y. A. Le, Cun, J. S. Denker, and B. Victorri, "Transformation invariance in pattern recognition - Tangent distance and tangent propagation," Int. J. Imag. Syst. Tech., vol. 11, no. 3, pp. 181-194, 2001.
70. R. Sokal and P. Sneath, Principles of Numerical Taxonomy. San Francisco, CA: Freeman, 1963.
71. A. Strehl and J. Ghosh, "Relationship-based clustering and visualization for high-dimensional data mining," INFORMS J. Comput. vol. 15, no. 2, pp. 208-230, 2003.
72. D. Tax and C. Veenman, "Tuning the hyperparameter of an AUC-optimized classifier," in Proc. Belgian-Dutch Conf. Artif. Intell, 2005, pp. 224-231.
73. M. Tipping, "The relevance vector machine," presented at the Adv. Neural Inf. Process. Syst., San Mateo, CA, 2000.
74. A. Tversky, "Features of similarity," Psychol. Rev., vol. 84, no. 4, pp. 327-352, 1977.
75. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
76. R. Veltkamp and M. Hagedoorn, "State-of-the-art in shape matching," in Principles of Visual Information Retrieval, M. Lew, Ed. New York: Springer-Verlag, 2001, pp. 87-119.
77. U. von Luxburg and O. Bousquet, "Distance-based classification with Lipschitz functions," J. Mach. Learn. Res., vol. 5, pp. 669-695, 2004.
78. G. Wahba, "Support vector machines, reproducing kernel Hilbert spaces and the randomized GACV," in Advances in Kernel Methods, Support Vector Learning, B. Schölkopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 69-88.
79. C. Watkins, "Dynamic alignment kernels," in Advances in Large Margin Classiers, A. Smola, P. Bartlett, D. Scholköpf, and B. Schuurmans, Eds. Cambridge, MA: MIT Press, 2000, pp. 39-50.
80. H. Zha and Z. Zhang, "Isometric embedding and continuum ISOMAP," in Proc. Int. Conf. Mach. Learn., Washington, DC, 2003, pp. 864-871.