Squares: Supporting Interactive Performance Analysis for Multiclass Classifiers

IEEE Transactions on Visualization and Computer Graphics

Volumn 23, Issue 1, 2017, Pages 61-70

  Ren, Donghao ^a   Amershi, Saleema ^b   Lee, Bongshin ^b   Suh, Jina ^b   Williams, Jason D ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

classification; Performance analysis; usable machine learning

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS;

APPLIED MACHINE LEARNING; CURRENT PERFORMANCE; INTERACTIVE PERFORMANCE; MULTI-CLASS CLASSIFIER; MULTICLASS CLASSIFICATION PROBLEMS; PERFORMANCE ANALYSIS; PERFORMANCE METRICS; PERFORMANCE VISUALIZATION;

CLASSIFICATION (OF INFORMATION);

ARTICLE; CLASSIFIER; HUMAN; PHYSICIAN;

EID: 84998817113     PISSN: 10772626     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVCG.2016.2598828     Document Type: Article

References (38)

1. Microsoft Azure Machine Learning. https://studio.azureml. net. Accessed: 2016-03-31.
2. Microsoft Power BI. https://powerbi.microsoft.com/. Ac-cessed: 2016-07-31.
3. Python. https://www.python.org/. Accessed: 2016-03-31.
4. B. Alsallakh, A. Hanbury, H. Hauser, S. Miksch, and A. Rauber. Visual methods for analyzing probabilistic classification data. IEEE Transac-tions on Visualization and Computer Graphics, 20(12):1703-1712, 2014.
5. M. Aly. Survey on multiclass classification methods. Neural Networks, (11):1-9, 2005.
6. S. Amershi, M. Chickering, S. M. Drucker, B. Lee, P. Simard, and J. Suh. ModelTracker: Redesigning performance analysis tools for ma-chine learning. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems-CHI '15, pages 337-346, 2015.
7. M. Ankerst, C. Elsen, M. Ester, and H.-P. Kriegel. Visual classification: an interactive approach to decision tree construction. In Proceedings of the fifth ACM SIGKDD International Conference on Knowledge Discov-ery and Data Mining, pages 392-396. ACM, 1999.
8. B. Becker, R. Kohavi, and D. Sommerfield. Visualizing the simple bayesian classifier. Information Visualization in Data Mining and Knowl-edge Discovery, 18:237-249, 2001.
9. M. Bostock, V. Ogievetsky, and J. Heer. D3: Data-driven docu-ments. IEEE Transactions on Visualization and Computer Graphics, 17(12):2301-2309, 2011.
10. D. Caragea, D. Cook, and V. G. Honavar. Visual methods for examining support vector machine results, with applications to gene expression data analysis. Technical report, 2005.
11. J. Choo, H. Lee, J. Kihm, and H. Park. iVisClassifier: An interactive visual analytics system for classification based on supervised dimension reduction. In Proceedings of the IEEE Symposium on Visual Analytics Science and Technology-VAST '10, pages 27-34, 2010.
12. T. E. de Campos, B. R. Babu, and M. Varma. Character recognition in natural images. In Proceedings of the International Conference on Com-puter Vision Theory and Applications (VISAPP), pages 273-280, 2009.
13. C. Ferri, J. Hernández-Orallo, and R. Modroiu. An experimental com-parison of performance measures for classification. Pattern Recognition Letters, 30(1):27-38, 2009.
14. E. Frank and M. Hall. Visualizing class probability estimators. In Pro-ceedings of European Conference on Principles of Data Mining and Knowledge Discovery, pages 168-179. Springer, 2003.
15. A. N. Gorban, B. Kégl, D. C. Wunsch, and A. Y. Zinovyev. Principal Manifolds for Data Visualization and Dimension Reduction, volume 58. Springer, 2008.
16. M. Harrower and C. A. Brewer. ColorBrewer.org: an online tool for selecting colour schemes for maps. The Cartographic Journal, 40(1):27-37, 2003.
17. F. Heimerl, S. Koch, H. Bosch, and T. Ertl. Visual classifier training for text document retrieval. IEEE Transactions on Visualization and Com-puter Graphics, 18(12):2839-2848, 2012.
18. G. Holmes, A. Donkin, and I. H. Witten. Weka: A machine learning workbench. In Proceedings of Australian New Zealnd Intelligent Infor-mation Systems Conference-ANZIIS '94, pages 357-361. IEEE, 1994.
19. R. Ihaka and R. Gentleman. R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics, 5(3):299-314, 1996.
20. D. H. Jeong, C. Ziemkiewicz, B. Fisher, W. Ribarsky, and R. Chang. iPCA: An interactive system for PCA-based visual analytics. Computer Graphics Forum, 28(3):767-774, 2009.
21. B. Klimt and Y. Yang. The enron corpus: A new dataset for email clas-sification research. In Proceedings of the 15th European Conference on Machine Learning-ECML '04, pages 217-226, 2004.
22. T. Kohonen. The self-organizing map. Proceedings of the IEEE, 78(9):1464-1480, 1990.
23. J. Krause, A. Perer, and K. Ng. Interacting with predictions: Visual in-spection of black-box machine learning models. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems-CHI '16, pages 5686-5697, 2016.
24. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
25. S. Lin, J. Fortuna, C. Kulkarni, M. Stone, and J. Heer. Selecting semantically-resonant colors for data visualization. Computer Graphics Forum, 32(3):401-410, 2013.
26. Mathworks. MATLAB and Statistics Toolbox Release 2012b, 2012.
27. J. G. S. Paiva,W. R. Schwartz, H. Pedrini, and R. Minghim. An approach to supporting incremental visual data classification. IEEE Transactions on Visualization and Computer Graphics, 21(1):4-17, 2015.
28. K. Patel, N. Bancroft, S. M. Drucker, J. Fogarty, A. J. Ko, and J. Landay. Gestalt: Integrated support for implementation and analysis in machine learning. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology-UIST '10, pages 37-46, 2010.
29. K. Patel, J. Fogarty, J. A. Landay, and B. Harrison. Investigating statisti-cal machine learning as a tool for software development. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems-CHI '08, pages 667-676, 2008.
30. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Van-derplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and É. Duch-esnay. Scikit-learn: Machine learning in Python. The Journal of Machine Learning Research, 12:2825-2830, 2011.
31. S. Ramaswamy, P. Tamayo, R. Rifkin, S. Mukherjee, C.-H. Yeang, M. Angelo, C. Ladd, M. Reich, E. Latulippe, J. P. Mesirov, T. Poggio, W. Gerald, M. Loda, E. S. Lander, and T. R. Golub. Multiclass can-cer diagnosis using tumor gene expression signatures. volume 98, pages 15149-15154, 2001.
32. P. Rheingans and M. DesJardins. Visualizing high-dimensional predictive model quality. In Proceedings of Visualization 2000, pages 493-496, 2000.
33. C. Seifert and E. Lex. A novel visualization approach for data-mining-related classification. In Proceedings of the 13th International Confer-ence Information Visualisation, pages 490-495, 2009.
34. M. Sokolova and G. Lapalme. A systematic analysis of performance mea-sures for classification tasks. Information Processing & Management, 45(4):427-437, 2009.
35. J. T. Townsend. Theoretical analysis of an alphabetic confusion matrix. Perception & Psychophysics, 9(1):40-50, 1971.
36. S. van den Elzen and J. J. vanWijk. BaobabView: Interactive construction and analysis of decision trees. In Proceedings of the IEEE Conference on Visual Analytics Science and Technology-VAST '11, pages 151-160, 2011.
37. B. Zadrozny and C. Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Proceedings of the 18th International Conference on Machine Learning-ICML '01, vol-ume 1, pages 609-616, 2001.
38. X. Zhu and I. Davidson. Knowledge Discovery and Data Mining: Chal-lenges and Realities. IGI Global, Hershey, PA, USA, 2007.