A survey of intelligent assistants for data analysis

ACM Computing Surveys

Volumn 45, Issue 3, 2013, Pages

  Serban, Floarea ^a   Vanschoren, Joaquin ^a   Kietz, Jorg Uwe ^b   Bernstein, Abraham ^a  

^a UNIVERSITY OF ZURICH   (Switzerland)

^b LEIDEN UNIVERSITY   (Netherlands)

Author keywords

Automatic workflow generation; Data mining; Intelligent assistants; User support

Indexed keywords

DATA MINING; DECISION MAKING; INDUSTRIAL RESEARCH; KNOWLEDGE MANAGEMENT;

ANALYTIC APPROACH; AUTOMATIC WORKFLOW GENERATION; DECISIONS MAKINGS; INTELLIGENT ASSISTANTS; LARGE AMOUNTS OF DATA; PRECISE ANALYSIS; REFINEMENT PROCESS; TECHNICAL EXPERTISE; USER SUPPORT; WORK-FLOWS;

SURVEYS;

EID: 84880124349     PISSN: 03600300     EISSN: 15577341     Source Type: Journal    
DOI: 10.1145/2480741.2480748     Document Type: Review

References (99)

1. Aha, D. W. 1992. Generalizing From Case Studies: A Case Study. In Proceedings Of The 9Th International Workshop On Machine Learning. 1-10.
2. Amant, R. And Cohen, P. 1998A. Interaction With A Mixed-Initiative System For Exploratory Data Analysis. Knowl. Based Syst. 10, 5, 265-273.
3. Amant, R. S. And Cohen, P. 1998B. Intelligent Support For Exploratory Data Analysis. J. Comput. Graph. Stat. 7, 4, 545-558.
4. Ashburner, M., Ball, C., Blake, J., Botstein, D., Butler, H., Cherry, J., Davis, A., Dolinski, K., Dwight, S., Eppig, J., Harris, M., Hill, D., Issel-Tarver, L., Kasarskis, A., Lewis, S., Matese, J., Richardson, J., Ringwald, M., Rubin, G., And Sherlock, G. 2000. Gene Ontology: Tool For The Unification Of Biology. Nature Genetics 25, 25-29.
5. Bensusan, H. And Kalousis, A. 2001. Estimating The Predictive Accuracy Of A Classifier. In Machine Learning, Lecture Notes In Computer Science, Vol. 2167, Springer, 25-36.
6. Bernstein, A. And Daenzer, M. 2007. The Next System: Towards True Dynamic Adaptations Of Semantic Web Service Compositions. In The Semantic Web: Research And Applications, Lecture Notes In Computer Science, Vol. 4519, Springer, 739-748.
7. Bernstein, A., Provost, F., And Hill, S. 2005. Toward Intelligent Assistance For A Data Mining Process: An Ontology-Based Approach For Cost-Sensitive Classification. Ieee Trans. Knowl. Data Eng. 17, 4, 503-518.
8. Berthold, M. R., Cebron, N., Dill, F., Gabriel, T. R., Kotter, T.,Meinl, T.,Ohl, P., Thiel, K., Andwiswedel, B. 2009. Knime -The Konstanz Information Miner: Version 2.0 And Beyond. Sigkdd Explor. Newsl. 11, 26-31.
9. Blockeel, H. And Vanschoren, J. 2007. Experiment Databases: Towards An Improved Experimental Methodology In Machine Learning. In Knowledge Discovery In Databases, Lecture Notes In Computer Science, Vol. 4702, Springer, 6-17.
10. Blum, A. And Furst, M. 1997. Fast Planning Through Planning Graph Analysis 1. Artificial Intelligence 90, 1-2, 281-300.
11. Botia, J., Gomez-Skarmeta, A., Valdes, M., And Padilla, A. 2001. Metala: A Meta-Learning Architecture. In Computational Intelligence. Theory And Apllications, Lecture Notes In Computer Science, Vol. 2206, Springer, 688-698.
12. Boulos, M. N. K. 2009. Semantic Wikis: A Comprehensible Introduction With Examples From The Health Sciences. J. Emerging Technol. Web Intel.
13. Castiello, C., Castellano, G., And Fanelli, A. 2005. Meta-Data: Characterization Of Input Features For Metalearning. Model. Decisions Artif. Intel. 3558, 457-468.
14. Castiello, C. And Fanelli, A. 2005. Meta-Learning Experiences With The Mindful System. In Computational Intelligence And Security, Lecture Notes In Computer Science, Vol. 3801, Springer, 321-328.
15. Cerrito, P. 2007. Introduction To Data Mining Using Sas Enterprise Miner. Sas Publishing, Cary, Nc.
16. Chandrasekaran, B., Johnson, T., And Smith, J. 1992. Task-Structure Analysis For Knowledge Modeling. Commun. Acm 35, 9, 124-137.
17. Chandrasekaran, B. And Josephson, J. 1999. What Are Ontologies, And Why Do We Need Them? Ieee Intell. Sys. 14, 1, 20-26.
18. Chapman, P., Clinton, J., Khabaza, T., Reinartz, T., And Wirth, R. 1999. The Crisp-Dm Process Model. The Crip-Dm Consortium 310.
19. Charest, M.,Delisle, S.,Cervantes, O., And Shen, Y. 2008. Bridging The Gap Between Data Mining And Decision Support: A Case-Based Reasoning And Ontology Approach. Intell. Data Anal. 12, 1-26.
20. Choinski, M. And Chudziak, J. 2009. Ontological Learning Assistant For Knowledge Discovery And Data Mining. In Proceedings Of The Ieee International Conference On Computer Science And Information Technology. 147-155.
21. Craw, S., Sleeman, D., Graner, N., And Rissakis, M. 1992. Consultant: Providing Advice For The Machine Learning Toolbox. In Proceedings Of The Annual Technical Conference On Expert Systems (Es). 5-23.
22. Derriere, S., Preite-Martinez, A., And Richard, A. 2006. Ucds And Ontologies. Asp Conf. Series 351, 449.
23. Diamantini, C., Potena, D., And Storti, E. 2009A. Kddonto: An Ontology For Discovery And Composition Of Kdd Algorithms. In Proceedings Of The Ecml-Pkdd Workshop On Service-Oriented Knowledge Discovery. 13-24.
24. Diamantini, C., Potena, D., And Storti, E. 2009B. Ontology-Driven Kdd Process Composition. In Advances In Intelligent Data Analysis Viii, Lecture Notes In Computer Science, Vol. 5772, Springer, 285-296.
25. Engels, R. 1996. Planning Tasks For Knowledge Discovery In Databases: Performing Task-Oriented Userguidance. In Proceedings Of The Acm Sigkdd International Conference On Knowledge Discovery And Data Mining (Kdd). 170-175.
26. Engels, R., Lindner, G., And Studer, R. 1997. A Guided Tour Through The Data Mining Jungle. In Proceedings Of The 3Rd International Conference On Knowledge Discovery In Databases. 163-166.
27. Erol, K. 1996. Hierarchical Task Network Planning: Formalization, Analysis, And Implementation. Ph.D. Dissertation, University Of Maryland At College Park, College Park, Md. Umi Order No. Gax96-22054.
28. Fayyad, U., Piatetsky-Shapiro, G., And Smyth, P. 1996. From Data Mining To Knowledge Discovery In Databases. Ai Mag. 17, 3, 37-54.
29. Fox, M. And Long, D. 2003. Pddl2. 1: An Extension To Pddl For Expressing Temporal Planning Domains. J. Artif. Intell. Res. 20, 1, 61-124.
30. Gale,W. 1986. Rex Review. In Artificial Intelligence And Statistics. Addison-Wesley Longman Publishing Co., Inc., Boston, Ma. 173-227.
31. Giraud-Carrier, C. 2005. The Data Mining Advisor: Meta-Learning At The Service Of Practitioners. In Proceedings Of The International Conference On Machine Learning And Applications (Icmla). 113-119.
32. Goble, C., Bhagat, J., Aleksejevs, S.,Cruickshank, D.,Michaelides, D.,Newman, D.,Borkum,M., Bechhofer, S., Roos, M., Li, P., And De Roure, D. 2010. Myexperiment: A Repository And Social Network For The Sharing Of Bioinformatics Workflows. Nucl. Acids Res..
33. Goebel, M. And Gruenwald, L. 1999. A Survey Of Data Mining And Knowledge Discovery Software Tools. Sigkdd Explor. Newsl. 1, 20-33.
34. Grabczewski, K. And Jankowski, N. 2007. Versatile And Efficient Meta-Learning Architecture: Knowledge Representation And Management In Computational Intelligence. In Proceedings Of The Ieee Symposium On Computational Intelligence And Data Mining. 51-58.
35. Graner, N., Sharma, S., Sleeman, D., Rissakis, M., Craw, S., And Moore, C. 1993. The Machine Learning Toolbox Consultant. Int. J. Ai Tools 2, 3, 307-328.
36. Grimmer, U. 1996. Clementine: Data Mining Software. In Classification And Multivariate Graphics: Models, Software And Applications. 25-31.
37. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., And Witten, I. 2009. The Weka Data Mining Software: An Update. Acm Sigkdd Explor. News. 11, 1, 10-18.
38. Hand, D. 1985. Statistical Expert Systems: Necessary Attributes. J. Appl. Stat. 12, 1, 19-27.
39. Hand, D. 1987. A Statistical Knowledge Enhancement System. J. Royal Stat. Soc. Series A (General) 150, 4, 334-345.
40. Hand, D. 1990. Practical Experience In Developing Statistical Knowledge Enhancement Systems. Ann. Math. Artif. Intell. 2, 1, 197-208.
41. Hand, D. 1997. Intelligent Data Analysis: Issues And Opportunities. In Proceedings Of The 2Nd International Symposium On Advances In Intelligent Data Analysis. Reasoning About Data (Ida'97). 1-14.
42. Hernansaez, J., Bota, J., And Skarmeta, A. 2004. Metala: A J2Ee Technology Based Framework For Web Mining. Revista Colombiana De Computacíon 5, 1.
43. Hilario, M. And Kalousis, A. 2001. Fusion Of Meta-Knowledge And Meta-Data For Case-Based Model Selection. In Proceedings Of The 5Th European Conference On Principles Of Data Mining And Knowledge Discovery (Pkdd '01). 180-191.
44. Hilario,M.,Kalousis, A.,Nguyen, P., Andwoznica, A. 2009. A Data Mining Ontology For Algorithm Selection And Meta-Mining. In Proceedings Of The Ecml-Pkdd Workshop On Service-Oriented Knowledge Discovery. 76-87.
45. Hoffmann, J. And Nebel, B. 2001. The Ff Planning System: Fast Plan Generation Through Heuristic Search. J. Artif. Intell. Res. 14, 253-302.
46. Horrocks, I., Patel-Schneider, P., And Boley, H. 2004. Swrl: A Semantic Web Rule Language Combining Owl And Ruleml. http://www.W3.Org/Submission/Swrl/.
47. Ihaka, R. And Gentleman, R. 1996. R: A Language For Data Analysis And Graphics. J. Computation. Graph. Stat. 5, 3, 299-314.
48. Kalousis, A. 2002. Algorithm Selection Via Meta-Learning. Ph.D. Dissertation, University Of Geveve.
49. Kalousis, A., Bernstein, A., And Hilario, M. 2008. Meta-Learning With Kernels And Similarity Functions For Planning Of Data Mining Workflows. In Proceedings Of The Icml/Uai/Colt Workshop On Planning To Learn. 23-28.
50. Kalousis, A. And Hilario, M. 2001. Model Selection Via Meta-Learning: A Comparative Study. Int. J. Artif. Intell. Tools 10, 4, 525-554.
51. Kalousis, A. And Theoharis, T. 1999. Noemon: Design, Implementation And Performance Results Of An Intelligent Assistant For Classifier Selection. Intell. Data Anal. 3, 4, 319-337.
52. Kietz, J., Serban, F., And Bernstein, A. 2010. Eproplan: A Tool To Model Automatic Generation Of Data Mining Workflows. In Proceedings Of The 3Rd Planning To Learn Workshop (Ws9) At The European Conference On Artificial Intelligence (Ecai'10). 15.
53. Kietz, J., Serban, F., Bernstein, A., And Fischer, S. 2009. Towards Cooperative Planning Of Data Mining Workflows. In Proceedings Of The Ecml-Pkdd Workshop On Service-Oriented Knowledge Discovery. 1-12.
54. Kietz, J., Vaduva, A., And Zucker, R. 2000. Mining Mart: Combining Case-Based-Reasoning And Multi-Strategy Learning Into A Framework To Reuse Kdd-Application. In Proceedings Of The 5Th International Workshop On Multistrategy Learning (Msl'00). Vol. 311.
55. Klusch,M.,Gerber, A., And Schmidt, M. 2005. Semanticweb Service Composition Planning With Owls-Xplan. In Proceedings Of The Aaai Fall Symposium On Agents And The Semantic Web. 55-62.
56. Kodratoff, Y., Sleeman, D., Uszynski, M., Causse, K., And Craw, S. 1992. Building A Machine Learning Toolbox. In Enhancing The Knowledge Engineering Process: Contributions From Esprit, L. Steels And B. Lepape, Eds., Elsevier, 81-108.
57. Kohavi, R.,Brodley, C. E.,Frasca, B.,Mason, L., And Zheng, Z. 2000. Kdd-Cup 2000 Organizers' Report: Peeling The Onion. Sigkdd Explor. Newsl. 2, 86-93.
58. Leite, R. And Brazdil, P. 2007. An Iterative Process For Building Learning Curves And Predicting Relative Performance Of Classifiers. In Progress In Artificial Intelligence, Lecture Notes In Computer Science, Vol. 4874, Springer, 87-98.
59. Levesque, R. 2005. Spss Programming And Data Management: A Guide For Spss And Sas Users. Spss, Chicago, Il.
60. Lindner, G. And Studer, R. 1999. Ast: Support For Algorithm Selection With A Cbr Approach. In Principles Of Data Mining And Knowledge Discovery, Lecture Notes In Computer Science, Vol. 1704, Springer, 418-423.
61. Liu, Z.,Ranganathan, A., And Riabov, A. 2007. A Planning Approach For Message-Oriented Semantic Web Service Composition. In Proceedings Of The Aaai National Conference On Artificial Intelligence 5, 2, 1389-1394.
62. Mathworks. 2004. Matlab. The Mathworks, Natick, Ma.
63. Mcdermott, D., Ghallab, M., Howe, A., Knoblock, C., Ram, A., Veloso, M., Weld, D., And Wilkins, D. 1998. Pddl-The Planning Domain Definition Language. http://Academic.Research.Microsoft.com/Paper/2024980.
64. Michie, D., Spiegelhalter, D., And Taylor, C. 1994. Machine Learning, Neural And Statistical Classification. Ellis Horwood, Upper Saddle River, Nj.
65. Mierswa, I.,Wurst, M.,Klinkenberg, R., Scholz, M., And Euler, T. 2006. Yale: Rapid Prototyping For Complex Data Mining Tasks. In Proceedings Of The 12Th Acm Sigkdd International Conference On Knowledge Discovery And Data Mining (Kdd'06). 935-940.
66. Mikut, R. And Reischl, M. 2011. Data Mining Tools. Wiley Interdisciplinary Rev. Data Mining Knowl. Discov..
67. Morik, K. And Scholz, M. 2004. The Miningmart Approach To Knowledge Discovery In Databases. In Intelligent Technologies For Information Analysis, N. Zhong, And J. Liu, Eds., Springer, 47-65.
68. Nonaka, I. And Takeuchi, H. 1995. The Knowledge-Creating Company: How Japanese Companies Create The Dynamics Of Innovation. Oxford University Press, New York, Ny.
69. Oinn, T., Addis, M., Ferris, J.,Marvin, D., Greenwood, M., Carver, T., Pocock, M., Wipat, A., And Li, P. 2004. Taverna: A Tool For The Composition And Enactment Of Bioinformatics Workflows. Bioinformatics 20, 17, 3045-3054.
70. Panov, P., Soldatova, L., And Dzeroski, S. 2009. Towards An Ontology Of Data Mining Investigations. In Discovery Science, Lecture Notes In Computer Science, Vol. 5808, Springer, 257-271.
71. Patel-Schneider, P., Hayes, P., And Horrocks, I. 2004. Owl Web Ontology Language Semantics And Abstract Syntax. http://www.W3.Org/Tr/Owl-Semantics/.
72. Peng, Y., Flach, P., Brazdil, P., And Soares, C. 2002A. Decision Tree-Based Data Characterization For Metalearning. In Proceedings Of The Ecml-Pkddworkshop On Integration And Collaboration Aspects Of Data Mining, Decision Support And Meta-Learning. 111-122.
73. Peng, Y., Flach, P., Soares, C., And Brazdil, P. 2002B. Improved Dataset Characterisation For Meta-Learning. In Discovery Science, Lecture Notes In Computer Science, Vol. 2534, Springer, 141-152.
74. Pfahringer, B., Bensusan, H., And Giraud-Carrier, C. 2000. Meta-Learning By Landmarking Various Learning Algorithms. In Proceedings Of The International Conference On Machine Learning (Icml) 951, 743-750.
75. Podpecan, V., Zemenova, M., And Lavrac, N. 2011. Orange4Ws Environment For Service-Oriented Data Mining. Comput. J.
76. Raes, J. 1992. Inside Two Commercially Available Statistical Expert Systems. Stat. Comput. 2, 2, 55-62.
77. Rendell, L., Seshu, R., And Tcheng, D. 1987. Layered Concept Learning And Dynamically-Variable Bias Management. In Proceedings Of The International Joint Conference On Artificial Intelligence (Ijcai). 308-314.
78. Rice, J. 1976. The Algorithm Selection Problem. Adv. Comput. 15, 65-118.
79. Roure, D. D., Goble, C., And Stevens, R. 2009. The Design And Realisation Of The Myexperiment Virtual Research Environment For Social Sharing Of Workflows. Future Gen. Comput. Syst. 25, 561-567.
80. Russell, D. M., Stefik, M. J., Pirolli, P., And Card, S. K. 1993. The Cost Structure Of Sensemaking. In Proceedings Of The Interact And Chi Conference On Human Factors In Computing Systems (Chi'93). 269-276.
81. Sacerdoti, E. 1974. Planning In A Hierarchy Of Abstraction Spaces. Artif. Intell. 5, 2, 115-135.
82. Schaffer, C. 1994. A Conservation Law For Generalization Performance. In Proceedings Of The International Conference On Machine Learning. 259-265.
83. Sirin, E. And Parsia, B. 2007. Sparql-Dl: Sparql Query For Owl-Dl. In Proceedings Of The International Workshop On Owl Experiences And Directions (Owled).
84. Sirin, E., Parsia, B., Grau, B., Kalyanpur, A., And Katz, Y. 2007. Pellet: A Practical Owl-Dl Reasoner. Web Semantics: Sci. Services Agents World Wide Web 5, 2, 51-53.
85. Sirin, E., Parsia, B.,Wu, D., Hendler, J., And Nau, D. 2004. Htn Planning For Web Service Composition Using Shop2. J. Web Semantics 1, 4, 377-396.
86. Sleeman, D., Rissakis, M., Craw, S., Graner, N., And Sharma, S. 1995. Consultant-2: Pre-And Post-Processing Of Machine Learning Applications. Int. J. Human Comput. Studies 43, 1, 43-63.
87. Smith-Miles, K. 2008. Cross-Disciplinary Perspectives On Meta-Learning For Algorithm Selection. Acm Comput. Surv. 41, 1, Article 6.
88. Soares, C., Brazdil, P., And Kuba, P. 2004. A Meta-Learning Method To Select The Kernel Width In Support Vector Regression. Machine Learn. 54, 195-209.
89. Stoeckert, C., Causton, H., And Ball, C. 2002. Microarray Databases: Standards And Ontologies. Nature Genetics 32, 469-473.
90. Szalay, A. And Gray, J. 2001. The World-Wide Telescope. Science 293, 2037-2040.
91. Taylor, I., Shields, M., Wang, I., And Harrison, A. 2007. The Triana Workflow Environment: Architecture And Applications. In Workflows For E-Science I. Taylor, E. Deelman, D. Gannon, And M. Shields, Eds., Springer, London, U.K. 320-339.
92. Todorovski, L.,Blockeel, H., And Dzeroski, S. 2002. Ranking With Predictive Clustering Trees. In Proceedings Of The 13Th European Conference On Machine Learning, Lecture Notes In Computer Science, Vol. 2430, Springer, 444-455.
93. Vanschoren, J. 2010. Understanding Machine Learning Performance With Experiment Databases. Ph.D. Dissertation, Katholieke Universiteit Leuven, Fianders, Belgium.
94. Vanschoren, J.,Blockeel,H., Pfahringer, B., Andholmes,G. 2012. Experiment Databases: Anewway To Share, Organize And Learn From Experiments. Machine Learn. Doi 10.1007/S10994-011-5277-0. To Appear.
95. Vilalta, R. And Drissi, Y. 2002A. A Characterization Of Difficult Problems In Classification. In Proceedings Of The International Conference On Machine Learning And Applications (Icmla). 133-138.
96. Vilalta, R. And Drissi, Y. 2002B. A Perspective View And Survey Of Meta-Learning. Artif. Intell. Rev. 18, 77-95.
97. Wirth, R., Shearer, C., Grimmer, U., Reinartz, T., Schlosser, J., Breitner, C., Engels, R., And Lindner, G. 1997. Towards Process-Oriented Tool Support For Knowledge Discovery In Databases. In Proceedings Of The 1St European Symposium On Principles Of Data Mining And Knowledge Discovery, Lecture Notes In Computer Science, Vol. 1263, Springer, 243-253.
98. Wolpert, D. 2001. The Supervised Learning No-Free-Lunch Theorems. In Proceedings Of The Online World Conference On Soft Computing In Industrial Applications. 25-42.
99. Yang, G., Kifer, M., Zhao, C., And Chowdhary, V. 2002. Flora-2: Users Manual. Department Of Computer Science, Stony Brook University, Stony Brook, Ny. Z ́akov́a, M., Kremen, P., ZelezńY, F., And Lavrac, N. 2010. Automating Knowledge Discovery Workflow Composition Through Ontology-Based Planning. Ieee Tran. Autom. Sci. Eng. 8, 2, 253-264.