Improving random forests by neighborhood projection for effective text classification

Information Systems

Volumn 77, Issue , 2018, Pages 1-21

  Salles, Thiago ^a   Gonçalves, Marcos ^a   Rodrigues, Victor ^a   Rocha, Leonardo ^b  

^a FEDERAL UNIVERSITY OF MINAS GERAIS   (Brazil)

^b FEDERAL UNIVERSITY OF SÃO JOÃO DEL REI   (Brazil)

Author keywords

Classification; Lazy learning; Nearest neighbors; Random forests

Indexed keywords

DECISION TREES; TEXT PROCESSING;

AUTOMATIC TEXT CLASSIFICATION; EXPERIMENTAL ANALYSIS; EXPERIMENTAL EVALUATION; LAZY LEARNING; NEAREST NEIGHBORHOOD; NEAREST NEIGHBORS; RANDOM FOREST CLASSIFIER; RANDOM FORESTS;

CLASSIFICATION (OF INFORMATION);

EID: 85048563895     PISSN: 03064379     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.is.2018.05.006     Document Type: Article

References (49)

1. Breiman, L., Random forests. Mach. Learn. 45:1 (2001), 5–32.
2. Díaz-Uriarte, R., Alvarez de Andrés, S., Gene selection and classification of microarray data using random forest. BMC Bioinf., 7(1), 2006, 3+.
3. Hastie, T., Tibshirani, R., Friedman, J.H., The Elements of Statistical Learning. 2009, Springer.
4. Segal, M.R., Machine Learning Benchmarks and Random Forest Regression. Technical Report, 2004, eScholarship Repository, University of California.
5. Liu, F.T., he Utility of Randomness in Decision Tree Ensembles, 2005, Faculty of Information Technology Monash University Master's thesis.
6. Statnikov, A., Wang, L., Aliferis, C.F., A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification. BMC Bioinf., 9(1), 2008, 319+.
7. Ye, Y., Wu, Q., Zhexue Huang, J., Ng, M.K., Li, X., Stratified sampling for feature subspace selection in random forests for high dimensional data. Pattern Recognit. 46:3 (2013), 769–787.
8. Wolpert, D.H., The lack of a priori distinctions between learning algorithms. Neural Comput. 8:7 (1996), 1341–1390.
9. Canuto, S.D., Belém, F.M., Almeida, J.M., Gonçalves, M.A., A comparative study of learning-to-rank techniques for tag recommendation. JIDM 4:3 (2013), 453–468.
10. Schulz, H., Waldvogel, B., Sheikh, R., Behnke, S., CURFIL: random forests for image labeling on GPU. Proc. of the Int. Conf. on Computer Vision Theory and Applications, vol. 2, 2015, 156–164.
11. Liu, Y., Chen, J., Su, Z., Luo, Z., Luo, N., Liu, L., Zhang, K., Robust head pose estimation using Dirichlet-tree distribution enhanced random forests. Neurocomputing 173 (2016), 42–53.
12. Robnik-Sikonja, M., Improving random forests. Proc. ECML, 2004, 359–370.
13. Rodriguez, J.J., Kuncheva, L.I., Alonso, C.J., Rotation forest: a new classifier ensemble method. IEEE Trans. Pattern Anal. Mach. Intell. 28:10 (2006), 1619–1630.
14. Geurts, P., Ernst, D., Wehenkel, L., Extremely randomized trees. Mach. Learn. 63:1 (2006), 3–42.
15. Ishwaran, H., The effect of splitting on random forests. Mach. Learn. 99:1 (2015), 75–118.
16. Lin, Y., Jeon, Y., Random forests and adaptive nearest neighbors. J. Am. Stat. Assoc. 101:474 (2006), 578–590.
17. Tsymbal, A., Pechenizkiy, M., Cunningham, P., Dynamic integration with random forests. Proc.ICML, 2006, 801–808.
18. Manning, C.D., Raghavan, P., Schtze, H., Introduction to Information Retrieval. 2008, Cambridge University Press, New York, NY.
19. Wang, J., Neskovic, P., Cooper, L.N., Improving nearest neighbor rule with a simple adaptive distance measure. Pattern Recogn. Lett. 28:2 (2007), 207–213.
20. Wang, J.-G., Neskovic, P., Cooper, L., An adaptive nearest neighbor algorithm for classification. Proceedings of the Int. Conf. on Machine Learning and Cybernetics, vol. 5, 2005, 3069–3074.
21. Pang, G., Jiang, S., A generalized cluster centroid based classifier for text categorization. Inf. Process. Manage. 49:2 (2013), 576–586.
22. Pang, G., Jin, H., Jiang, S., Cenknn: a scalable and effective text classifier. Data Min. Knowl. Discov. 29:3 (2015), 593–625.
23. Veloso, A., Meira Jr., W., Zaki, M.J., Lazy associative classification. Proc. ICDM, 2006, 645–654.
24. Baralis, E., Chiusano, S., Garza, P., A lazy approach to associative classification. TKDE 20:2 (2008), 156–171.
25. Thabtah, F., Pruning techniques in associative classification: survey and comparison. JDIM 4:3 (2006), 197–202.
26. Friedman, J.H., Lazy decision trees. Proceedings of the Thirteenth National Conference on Artificial intelligence - Volume 1, 1996, 717–724.
27. Fern, X.Z., Brodley, C.E., Boosting lazy decision trees. Proc. ICML, 2003, 178–185.
28. Zhang, H., Berg, A.C., Maire, M., Malik, J., SVM-KNN: discriminative nearest neighbor classification for visual category recognition. CVPR, 2006, II:2126–2136.
29. Lin, Y., Lin, Y., Jeon, Y., Jeon, Y., Random Forests and Adaptive Nearest Neighbors. Technical Report, 2002, Journal of the American Statistical Association.
30. Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J., Liblinear: a library for large linear classification. JMLR 9 (2008), 1871–1874.
31. Chang, C.-C., Lin, C.-J., Libsvm: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2:3 (2011), 27:1–27:27.
32. Rocha, L., Mourão, F., Pereira, A., Gonçalves, M.A., Meira Jr., W., Exploiting temporal contexts in text classification. Proc. CIKM, 2008, 243–252.
33. Lewis, D.D., Yang, Y., Rose, T.G., Li, F., Rcv1: a new benchmark collection for text categorization research. JMLR 5 (2004), 361–397.
34. M. Thelwall, Heart and Soul: Sentiment Strength Detection in the Social Web with SentiStrength 1, 2013.
35. Hutto, C.J., Gilbert, E., VADER: a parsimonious rule based-model for sentiment analysis of social media text. Adar, E., Resnick, P., Choudhury, M.D., Hogan, B., Oh, A., (eds.) Proc. of the Int. Conf. on Weblogs and Social Media, 2014.
36. Ramasubramanian, V., Paliwal, K., Fast k-dimensional tree algorithms for nearest neighbor search with application to vector quantization encoding. IEEE Trans. Signal Process. 40:3 (1992), 518–531.
37. Liu, T., Moore, A.W., Gray, A., Yang, K., An investigation of practical approximate nearest neighbor algorithms. Advances in Neural Information Processing Systems, 2004, 825–832.
38. Gao, J., Jagadish, H.V., Lu, W., Ooi, B.C., Dsh: data sensitive hashing for high-dimensional k-nn search. Proceedings of the Int. Conf. on Management of Data SIGMOD, 2014, 1127–1138.
39. Zhang, B., Srihari, S.N., Fast k-nearest neighbor classification using cluster-based trees. IEEE Trans. Pattern Anal. Mach. Intell. 26:4 (2004), 525–528.
40. Gutiérrez, P.D., Lastra, M., Bacardit, J., Benítez, J.M., Herrera, F., Gpu-sme-knn: scalable and memory efficient knn and lazy learning using gpus. Inf. Sci. 373 (2016), 165–182.
41. Silpa-Anan, C., Hartley, R., Optimised kd-trees for fast image descriptor matching. 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008, 1–8 Anchorage, AK, USA.
42. Muja, M., Lowe, D.G., Fast approximate nearest neighbors with automatic algorithm configuration. In VISAPP International Conference on Computer Vision Theory and Applications, 2009, 331–340.
43. Dong, W., Moses, C., Li, K., Efficient k-nearest neighbor graph construction for generic similarity measures. Proceedings of the 20th International Conference on World Wide Web, 2011, ACM, New York, NY, USA, 577–586.
44. Ribeiro, M.T., Singh, S., Guestrin, C., “Why should I trust you?”: explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13–17, 2016, 2016, 1135–1144.
45. Wang, F., Sun, J., Survey on distance metric learning and dimensionality reduction in data mining. Data Min. Knowl. Discov. 29:2 (2014), 534–564.
46. Weinberger, K.Q., Saul, L.K., Distance metric learning for large margin nearest neighbor classification. JMLR 10 (2009), 207–244.
47. Jin, R., Wang, S., Zhou, Y., Regularized distance metric learning: theory and algorithm. NIPS, 2009, 862–870.
48. Qi, G.-J., Tang, J., Zha, Z.-J., Chua, T.-S., Zhang, H.-J., An efficient sparse metric learning in high-dimensional space via l1-penalized log-determinant regularization. ICML, 2009, 841–848.
49. Canuto, S.D., Gonçalves, M.A., Santos, W., Rosa, T., Martins, W., An efficient and scalable metafeature-based document classification approach based on massively parallel computing. SIGIR, 2015, 333–342.