Visual methods for analyzing probabilistic classification data

IEEE Transactions on Visualization and Computer Graphics

Volumn 20, Issue 12, 2014, Pages 1703-1712

  Alsallakh, Bilal ^a   Hanbury, Allan ^a   Hauser, Helwig ^b   Miksch, Silvia ^a   Rauber, Andreas ^a  

^a VIENNA UNIVERSITY OF TECHNOLOGY   (Austria)

^b UNIVERSITY OF BERGEN   (Norway)

Author keywords

Confusion analysis; Feature evaluation and selection; Probabilistic classification; Visual inspection

Indexed keywords

CONFUSION ANALYSIS; FEATURE EVALUATION AND SELECTION; PROBABILISTIC CLASSIFICATION; VISUAL INSPECTION;

EID: 84909592940     PISSN: 10772626     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVCG.2014.2346660     Document Type: Article

References (43)

1. B. Alsallakh, W. Aigner, S. Miksch, and M. E. Gröller. Reinventing the contingency wheel: Scalable visual analytics of large categorical data. Visualization and Computer Graphics, IEEE Transactions on, 18(12):2849-2858, 2012.
2. B. Alsallakh, W. Aigner, S. Miksch, and H. Hauser. Radial sets: Interactive visual analysis of large overlapping sets. Visualization and Computer Graphics, IEEE Transactions on, 19(12):2496-2505, 2013.
3. A. Anand, L. Wilkinson, and D. N. Tuan. An L-infinity norm visual classifier. In IEEE International Conference on Data Mining (ICDM), pages 687-692, 2009.
4. M. Ankerst, C. Elsen, M. Ester, and H.-P. Kriegel. Visual classification: an interactive approach to decision tree construction. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 392-396. ACM, 1999.
5. K. Bache and M. Lichman. UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml (accessed: March 2014).
6. S. Bremm, T. von Landesberger, J. Bernard, and T. Schreck. Assisted descriptor selection based on visual comparative data analysis. Computer Graphics Forum, 30(3):891-900, 2011.
7. E. Brown, J. Liu, C. Brodley, and R. Chang. Dis-function: Learning distance functions interactively. In IEEE Conference on Visual Analytics Science and Technology (VAST), pages 83-92, 2012.
8. D. Caragea, D. Cook, H. Wickham, and V. Honavar. Visual methods for examining SVM classifiers. In Visual Data Mining, pages 136-153. Springer, 2008.
9. J. Choo, H. Lee, J. Kihm, and H. Park. iVisClassifier: An interactive visual analytics system for classification based on supervised dimension reduction. In IEEE Symposium on Visual Analytics Science and Technology (VAST), pages 27-34, 2010.
10. A. E. Daniels. Incorporating domain knowledge and spatial relationships into land cover classifications: a rule-based approach. International Journal of Remote Sensing, 27(14):2949-2975, 2006.
11. J. Devore. Probability and Statistics for Engineering and the Sciences. Cengage Learning, 2011.
12. C. Drummond and R. Holte. Cost curves: An improved method for visualizing classifier performance. Machine Learning, 65(1):95-130, 2006.
13. T. Fawcett. An introduction to ROC analysis. Pattern recognition letters, 27(8):861-874, 2006.
14. M. R. Hassan, K. Ramamohanarao, C. Karmakar, M. M. Hossain, and J. Bailey. A novel scalable multi-class ROC for effective visualization and computation. In Advances in Knowledge Discovery and Data Mining, pages 107-120. Springer, 2010.
15. F. Heimerl, S. Koch, H. Bosch, and T. Ertl. Visual classifier training for text document retrieval. Visualization and Computer Graphics, IEEE Trans. on, 18(12):2839-2848, 2012.
16. B. Hoferlin, R. Netzel, M. Hoferlin, D. Weiskopf, and G. Heidemann. Inter-active learning of ad-hoc classifiers for video visual analytics. In IEEE Conference on Visual Analytics Science and Technology (VAST), pages 23-32, 2012.
17. T. Iwata, K. Saito, N. Ueda, S. Stromsten, T. L. Griffiths, and J. B. Tenenbaum. Parametric embedding for class visualization. Neural Computation, 19(9):2536-2556, 2007.
18. A. Kapoor, B. Lee, D. Tan, and E. Horvitz. Interactive optimization for steering machine classification. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI), pages 1343-1352. ACM, 2010.
19. W. Kienreich and C. Seifert. Visual exploration of feature-class matrices for classification problems. In International Workshop on Visual Analytics (EuroVA), pages 37-41. The Eurographics Association, 2012.
20. N. Lachiche and P. Flach. Improving accuracy and cost of two-class and multi-class probabilistic classifiers using ROC curves. In International Conference on Machine Learning (ICML), volume 20, pages 416-423, 2003.
21. L. Lam and C. Y. Suen. Optimal combinations of pattern classifiers. Pattern Recognition Letters, 16(9):945-954, 1995.
22. F. J. Massey Jr. The Kolmogorov-Smirnov test for goodness of fit. Journal of the American statistical Association, 46(253):68-78, 1951.
23. T. May, A. Bannach, J. Davey, T. Ruppert, and J. Kohlhammer. Guiding feature subset selection with an interactive visualization. In IEEE Conference on Visual Analytics Science and Technology (VAST), pages 111-120, 2011.
24. I. Mierswa, M. Wurst, R. Klinkenberg, M. Scholz, and T. Euler. YALE: Rapid prototyping for complex data mining tasks. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 935-940. ACM, 2006.
25. M. Migut and M. Worring. Visual exploration of classification models for risk assessment. In IEEE Symposium on Visual Analytics Science and Technology (VAST), pages 11-18. IEEE, 2010.
26. M. Migut, M. Worring, and C. Veenman. Visualizing multi-dimensional decision boundaries in 2d. Data Mining and Knowledge Discovery, pages 1-23, 2013.
27. T. Pham, R. Metoyer, K. Bezrukova, and C. Spell. Visualization of cluster structure and separation in multivariate mixed data: A case study of diversity faultlines in work teams. Computers & Graphics, 38:117-130, 2014.
28. D. M. Powers. Evaluation: from precision, recall and f-measure to roc, informedness, markedness & correlation. Journal of Machine Learning Technologies, 2(1):37-63, 2011.
29. P. Rheingans and M. desJardins. Visualizing high-dimensional predictive model quality. In Proceedings of IEEE Visualization, pages 493-496, 2000.
30. M. Sedlmair, A. Tatu, T. Munzner, and M. Tory. A taxonomy of visual cluster separation factors. In Computer Graphics Forum, volume 31, pages 1335-1344. Wiley Online Library, 2012.
31. C. Seifert and M. Granitzer. User-based active learning. In IEEE International Conference on Data Mining Workshops (ICDMW), pages 418-425, 2010.
32. C. Seifert and E. Lex. A novel visualization approach for data-mining-related classification. In Information Visualisation (IV), 13th International Conference, pages 490-495, 2009.
33. B. Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison, 2009.
34. F. Shafait, M. Reif, C. Kofler, and T. M. Breuel. Pattern recognition engineering. In RapidMiner Community Meeting and Conference, volume 9, 2010.
35. J. Talbot, B. Lee, A. Kapoor, and D. S. Tan. EnsembleMatrix: Interactive visualization to support machine learning with multiple classifiers. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI), pages 1283-1292. ACM, 2009.
36. A. Tatu, G. Albuquerque, M. Eisemann, J. Schneidewind, H. Theisel, M. Magnor, and D. Keim. Combining automated analysis and visualization techniques for effective exploration of high-dimensional data. In IEEE Symposium on Visual Analytics Science and Technology (VAST)., pages 59-66. IEEE, 2009.
37. D. M. Tax, M. Van Breukelen, R. P. Duin, and J. Kittler. Combining multiple classifiers by averaging or by multiplying? Pattern recognition, 33(9):1475-1485, 2000.
38. S. T. Teoh and K.-L. Ma. PaintingClass: interactive construction, visualization and exploration of decision trees. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 667-672, New York, NY, USA, 2003. ACM.
39. T. Van de Voorde, W. De Genst, and F. Canters. Improving pixel-based vhr land-cover classifications of urban areas with post-classification techniques. Photogrammetric Engineering and Remote Sensing, 73(9):1017, 2007.
40. S. van den Elzen and J. van Wijk. Baobabview: Interactive construction and analysis of decision trees. In IEEE Conference on Visual Analytics Science and Technology (VAST), pages 151-160, 2011.
41. M. Ware, E. Frank, G. Holmes, M. Hall, and I. Witten. Interactive machine learning: letting users build classifiers. Intl. Journal of Human-Computer Studies, 55(3):281-292, 2001.
42. L. Xu, A. Krzyzak, and C. Y. Suen. Methods of combining multiple classifiers and their applications to handwriting recognition. Systems, Man and Cybernetics, IEEE Transactions on, 22(3):418-435, 1992.
43. J. Zhang and L. Gruenwald. Opening the black box of feature extraction: Incorporating visualization into high-dimensional data mining processes. In IEEE International Conference on Data Mining (ICDM), pages 1188-1192, 2006.