Multiclass from binary: Expanding One-versus-all, one-versus-one and ECOC-based approaches

IEEE Transactions on Neural Networks and Learning Systems

Volumn 25, Issue 2, 2014, Pages 289-302

  Rocha, Anderson ^a   Goldenstein, Siome Klein ^a  

^a UNIVERSITY OF CAMPINAS   (Brazil)

Author keywords

Error correcting output codes (ECOC); multiclass from binary; one versus all (OVA); one versus one (OVO)

Indexed keywords

BINARY CLASSIFICATION PROBLEMS; ERROR CORRECTING OUTPUT CODE; MULTI-CLASS CLASSIFICATION; MULTI-CLASS PROBLEMS; MULTICLASS FROM BINARY; ONE-VERSUS-ALL (OVA); ONE-VERSUS-ONE (OVO); STATISTICAL CLASSIFICATION TECHNIQUES;

BAYESIAN NETWORKS; ITERATIVE METHODS; PROBLEM SOLVING;

CLASSIFICATION (OF INFORMATION);

EID: 84893668465     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2274735     Document Type: Article

References (61)

1. C. Bishop, Pattern Recognition and Machine Learning. New York, NY, USA: Springer-Verlag, 2006.
2. A. Passerini, M. Pontil, and P. Frasconi, "New results on error correcting output codes of kernel machines," IEEE Trans. Neural Netw. Learn. Syst., vol. 15, no. 1, pp. 45-54, Jan. 2004.
3. N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar, "Describable visual attributes for face verification and image search," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 10, pp. 1962-1977, Oct. 2011.
4. P. J. Phillips, J. R. Beveridge, B. A. Draper, G. Givens, A. J. O'Toole, D. S. Bolme, J. Dunlop, Y. M. Lui, H. Sahibzada, and S. Weimer, "An introduction to the good, the bad, & the ugly face recognition challenge problem," Nat. Inst. Standards Technol., Gaithersburg, MD, USA, Tech. Rep. NISTIR 7758, Mar. 2011.
5. G. Chiachia, A. Falcão, and A. Rocha, "Person-specific face representation for recognition," in Proc. IJCB, Oct. 2011, pp. 1-8.
6. R. Cappelli, M. Ferrara, and D. Maltoni, "Minutia cylinder-code: A new representation and matching technique for fingerprint recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 12, pp. 2128-2141, Dec. 2010.
7. A. K. Jain, A. A. Ross, and K. Nandakumar, Introduction to Biometrics. New York, NY, USA: Springer-Verlag, 2011, pp. 51-96.
8. J. Deng, A. Berg, K. Li, and L. Fei-Fei, "What does classifying more than 10,000 image categories tell us?" in Proc. 11th ECCV, 2010, pp. 71-84.
9. L.-J. Li, H. Su, Y. Lim, and L. Fei-Fei, "Objects as attributes for scene classification," in Proc. ECCV, 2010, pp. 57-69.
10. L. Afonso, J. P. Papa, A. N. Marana, and A. Rocha, "Automatic visual dictionary generation through optimun-path forest clustering," in Proc. 19th ICIP, Sep. 2012, pp. 1897-1900.
11. R. G. Cinbis, J. Verbeek, and C. Schmid, "Image categorization using Fisher kernels of non-iid image models," in Proc. IEEE Int. CVPR, Jun. 2012, pp. 1-8.
12. F. Perronnin, J. Sánchez, and Y. Liu, "Large-scale image categorization with explicit data embedding," in Proc. IEEE Int. Conf. CVPR, Jun. 2010, pp. 2297-2304.
13. H. Stranneheim, M. Käller, T. Allander, B. Andersson, L. Arvestad, and J. Lundeberg, "Classification of DNA sequences using bloom filters," Bioinformatics, vol. 26, no. 13, pp. 1595-1600, Jul. 2010.
14. C. Cortes and V. Vapnik, "Support-vector networks," Mach. Learn., vol. 20, no. 3, pp. 273-297, Mar. 1995.
15. R. P. Duin and E. Pekalska, "Open issues in pattern recognition," in Computer Recognition Systems, M. Kurzynski, E. Puchala, M. Wozniak, and A. Zolnierek, Eds. New York, NY, USA: Springer-Verlag, 2005, pp. 27-42.
16. N. Pedrajas and D. Boyer, "Improving multi-class pattern recognition by the combination of two strategies," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 6, pp. 1001-1006, Jun. 2006.
17. O. Pujol, P. Radeva, and J. Vitria, "Discriminant ECOC: A heuristic method for application dependent design of ECOC," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 6, pp. 1007-1012, Jun. 2006.
18. A. Rocha and S. Goldenstein, "From binary to multi-class: Divide to conquer," in Proc. Int. Conf. Comput. Vis. Theory Appl. VISAPP, 2009, pp. 323-330.
19. P. Clark and R. Boswell, "Rule induction with CN2: Some improvements," in Proc. EWSL, Mar. 1991, pp. 151-163.
20. R. Anand, K. Mehrotra, C. Mohan, and S. Ranka, "Efficient classification for multi-class problems using modular neural networks," IEEE Trans. Neural Netw. Learn. Syst., vol. 6, no. 1, pp. 117-124, Jan. 1995.
21. J. Platt, N. Christiani, and J. Taylor, "Large margin dags for multi-class classification," in Proc. Int. Conf. NIPS, 1999, pp. 547-553.
22. S. Abe, "Analysis of multiclass support vector machines," in Proc. Int. Conf. CIMCA, 2003, pp. 385-396.
23. T. Dietterich and G. Bakiri, "Solving multi-class problems via ECOC," J. Artif. Intell. Res., vol. 2, no. 1, pp. 263-286, Jan. 1996.
24. E. Allwein, R. Shapire, and Y. Singer, "Reducing multi-class to binary: A unifying approach for margin classifiers," J. Mach. Learn. Res., vol. 1, no. 1, pp. 113-141, Jan. 2000. (Pubitemid 33738778)
25. T. Windeatt and R. Ghaderi, "Coding and decoding strategies for multiclass learning problems," Inf. Fusion, vol. 4, no. 1, pp. 11-21, 2003. (Pubitemid 36210678)
26. A. Klautau, N. Jevtic, and A. Orlitsky, "On nearest-neighbor ECOC with application to all-pairs multiclass support vector machines," J. Mach. Learn. Res., vol. 4, no. 1, pp. 1-15, Jan. 2004.
27. M. Moreira and E. Mayoraz, "Improved pairwise coupling classification with correcting classifiers," in Proc. Eur. ICML, Apr. 1998, pp. 160-171. (Pubitemid 128067181)
28. V. Athisos, A. Stefan, Q. Yuan, and S. Sclaroff, "ClassMap: Efficient multiclass recognition via embeddings," in Proc. IEEE 11th ICCV, Oct. 2007, pp. 1-8.
29. K. Crammer and Y. Singer, "On the learnability and design of output codes for multi-class problems," J. Mach. Learn. Res., vol. 47, nos. 2-3, pp. 201-233, Mar. 2002. (Pubitemid 34126110)
30. T. Takenouchi and S. Ishii, "Multi-class classification as a decoding problem," in Proc. IEEE Symp. FOCI, Apr. 2007, pp. 470-475.
31. C. Young, C. Yen, Y. Pao, and M. Nagurka, "One-class-at-time removal sequence planning method for multi-class problems," IEEE Trans. Neural Netw. Learn. Syst., vol. 17, no. 6, pp. 1544-1549, Nov. 2006. (Pubitemid 44824266)
32. J. Manikandan and B. Venkataramani, "Design of a modified oneagainst-all SVM classifier," in Proc. IEEE Int. Conf. SMC, Oct. 2009, pp. 1869-1874.
33. T. Mota and A. Thome, "One-against-all-based multiclass SVM strategies applied to vehicle plate character recognition," in Proc. IEEE IJCNN, Jun. 2009, pp. 2153-2159.
34. Y. Liu and Y. Zheng, "One-against-all multi-class SVM classification using reliability measures," in Proc. IEEE IJCNN, Aug. 2005, pp. 849-854. (Pubitemid 44055562)
35. Y. Liu, R. Wang, and Y.-S. Zeng, "An improvement of one-against-one method for multi-class support vector machine," in Proc. IEEE Int. Conf. Mach. Learn. Cybern., Aug. 2007, pp. 2915-2920.
36. T. Yeh, J. Lee, and T. Darrell, "Scalable classifiers for internet vision tasks," in Proc. IEEE Int. Conf. CVPRW, 2008, pp. 1-8.
37. A. Vedaldi and A. Zisserman, "Efficient additive kernels via explicit feature maps," in Proc. IEEE Int. Conf. CVPR, Jun. 2010, pp. 3539-3546.
38. T. Tommasi, F. Orabona, and B. Caputo, "Safety in numbers: Learning categories from few examples with multi model knowledge transfer," in Proc. IEEE Int. Conf. CVPR, Jun. 2010, pp. 3081-3088.
39. Y. Lin, F. Lv, S. Zhu, M. Yang, T. Cour, K. Yu, L. Cao, and T. Huang, "Large-scale image classification: Fast feature extraction and SVM training," in Proc. IEEE Int. Conf. CVPR, Jun. 2011, pp. 1689-1696.
40. M. Bautista, X. Baro, O. Pujol, P. Radeva, J. Vitria, and S. Escalera, "Compact evolutive design of error-correcting output codes," in Proc. Supervised Unsupervised Ensemble Methods Appl. Eur. Conf. Mach. Learn., Dec. 2010, pp. 119-128.
41. G. Madjarov, D. Kocev, D. Gjorgjevikj, and S. Dzeroski, "An extensive experimental comparison of methods for multi-label learning," Pattern Recognit., vol. 45, no. 1, pp. 3084-3104, 2012.
42. C.-P. Lee and C.-J. Lin, "A study on L2-loss (squared hinge-loss) multiclass SVM," Dept. Comput. Sci., Nat. Taiwan Univ., Taipei, Taiwan, Tech. Rep. 1, Nov. 2012.
43. A. Narasimhamrthy, "Theoretical bounds of majority voting performance for a binary classification problem," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 12, pp. 1988-1995, Dec. 2005.
44. New methods for spectral clustering,School Comput. Sci. Eng
45. I. Fisher and J. Poland, "New methods for spectral clustering," School Comput. Sci. Eng., Hebrew Univ., Jerusalem, Israel, Tech. Rep. IDSIA-12-04, Jun. 2004.
46. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
47. L. Kaufman and P. Rousseeuw, Finding Groups in Data: An Introduction to Cluster Analysis, 1st ed. New York, NY, USA: Wiley, 1990.
48. T. Dietterich and G. Bakiri, "Error-correcting output codes: A general method for improving multiclass inductive learning programs," in Proc. Int. Conf. AAAI, 1991, pp. 572-577.
49. R. Schapire, "Using output codes to boost multiclass learning problems," in Proc. ICML, 1997, pp. 313-321.
50. G. M. James and T. Hastie, "The error coding method and PICT's," Comput. Graph. Stat., vol. 7, no. 1, pp. 377-387, 1998.
51. A. Blum, "Random projection, margins, kernels, and feature-selection," in Proc. Int. Conf. SLSFS, 2006, pp. 52-68. (Pubitemid 44029879)
52. I. Tosic and P. Frossard, "Dictionary learning," IEEE Signal Process. Mag., vol. 28, no. 2, pp. 27-38, Mar. 2011.
53. S. Dasgupta, "Experiments with random projection," in Proc. Conf. UAI, 2000, pp. 143-151.
54. C. R. Rao, "The utilization of multiple measurements in problems of biological classification," J. R. Stat. Soc. Ser. B, Stat. Methodol., vol. 10, no. 2, pp. 159-203, Feb. 1948.
55. B. Ripley, Pattern Recognition and Neural Networks, 1st ed. Cambridge, U.K.: Cambridge Univ. Press, 1996.
56. C.-W. Hsu and C.-J. Lin. (2012, Jun.). The BSVM Support Vector Machine Library, Vs. 2.08 [Online]. Available: http://www. csie.ntu.edu.tw/?cjlin/bsvm/
57. K. Crammer and Y. Singer, "On the algorithmic implementation of multiclass kernel-based vector machines," School Comput. Sci. Eng., Hebrew Univ., Jerusalem, Israel, Tech. Rep. 1, Dec. 2001.
58. F. Wilcoxon, "Individual comparisons by ranking methods," Biometrics Bulletin, vol. 1, no. 6, pp. 80-83, 1945.
59. S. Siegel, Non-Parametric Statistics for the Behavioral Sciences. New York, NY, USA: McGraw-Hil, 1956.
60. R. Stehling, M. Nascimento, and A. Falcão, "A compact and efficient image retrieval approach based on border/interior pixel classification," in Proc. ACM CIKM, 2002, pp. 102-109.
61. D. H. Wolpert and W. G. Macready, "No free lunch theorems for optimization," IEEE Trans. Evol. Comput., vol. 1, no. 1, pp. 67-82, Jan. 1997. (Pubitemid 127771019)