A multi-dimensional quality assessment of state-of-the-art process discovery algorithms using real-life event logs

Information Systems

Volumn 37, Issue 7, 2012, Pages 654-676

  De Weerdt, Jochen ^a   De Backer, Manu ^a,b   Vanthienen, Jan ^a   Baesens, Bart ^a,c  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b GHENT UNIVERSITY   (Belgium)

^c UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

Accuracy; Benchmarking; Comprehensibility; Process mining; Real life event logs

Indexed keywords

ACCURACY; BUSINESS PROCESS EXECUTION; COMPLEX EVENTS; COMPREHENSIBILITY; CONTROL-FLOW; DATA SETS; ESSENTIAL ELEMENTS; POSTERIORI ANALYSIS; PROCESS DISCOVERY; PROCESS MINING; PROCESS MODEL; QUALITY ASSESSMENT; REAL-LIFE EVENT LOGS; RESEARCH DOMAINS; SUBDOMAIN;

ALGORITHMS; BENCHMARKING; RESEARCH;

DATA MINING;

EID: 84861093725     PISSN: 03064379     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.is.2012.02.004     Document Type: Article

References (67)

1. W.M.P. van der Aalst, H.A. Reijers, A.J.M.M. Weijters, B.F. van Dongen, A.K. Alves de Medeiros, M. Song, and H.M.W. Verbeek Business process mining: an industrial application Information Systems 32 2007 713 732 (Pubitemid 46560274)
2. W.M.P. van der Aalst, A.J.M.M. Weijters, and L. Maruster Workflow mining: discovering process models from event logs IEEE Transactions on Knowledge and Data Engineering 16 2004 1128 1142
3. P.-N. Tan, M. Steinbach, and V. Kumar Introduction to Data Mining 2005 Addison-Wesley
4. M. Weske Business Process Management: Concepts, Languages, Architectures 2007 Springer
5. M. Song, and W.M.P. van der Aalst Towards comprehensive support for organizational mining Decision Support Systems 46 2008 300 317
6. W.M.P. van der Aalst, M.H. Schonenberg, and M. Song Time prediction based on process mining Information Systems 36 2011 450 475
7. W.M.P. van der Aalst, B.F. van Dongen, C.W. Günther, A. Rozinat, E. Verbeek, and T. Weijters ProM: the process mining toolkit A.K.A. de Medeiros, B. Weber, BPM (Demos), CEUR Workshop Proceedings, CEUR-WS.org vol. 489 2009
8. R. Agrawal, D. Gunopulos, and F. Leymann Mining process models from workflow logs H. Schek, F. Saltor, I. Ramos, G. Alonso, Proceedings of the 6th International Conference on Extending Database Technology (EDBT'98), Lecture Notes in Computer Science vol. 1377 1998 Springer 469 483 (Pubitemid 128044768)
9. J. Cook, and A. Wolf Discovering models of software processes from event-based data ACM Transactions on Software Engineering and Methodology 7 1998 215 249 (Pubitemid 128658170)
10. A. Datta Automating the discovery of AS-IS business process models: probabilistic and algorithmic approaches Information Systems Research 9 1998 275 301
11. H. Mannila, C. Meek, Global partial orders from sequential data, in: Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '00), ACM, New York, NY, USA, 2000, pp. 161-168.
12. G. Schimm Process miner - a tool for mining process schemes from event-based data S. Flesca, S. Greco, N. Leone, G. Ianni, JELIA, Lecture Notes in Computer Science vol. 2424 2002 Springer 525 528
13. G. Schimm Mining exact models of concurrent workflows Computers in Industry 53 2004 265 281
14. A.K. Alves de Medeiros, B.F. van Dongen, W.M.P. van der Aalst, A.J.M.M. Weijters, Process Mining: Extending the Alpha-Algorithm to Mine Short Loops, BETA Working Paper Series 113, TU Eindhoven, 2004.
15. L. Wen, W.M.P. van der Aalst, J. Wang, and J. Sun Mining process models with non-free-choice constructs Data Mining and Knowledge Discovery 15 2007 145 180 (Pubitemid 47293485)
16. L. Wen, J. Wang, W.M.P. van der Aalst, B. Huang, and J. Sun A novel approach for process mining based on event types Journal of Intelligent Information Systems 32 2009 163 190
17. A.J.M.M. Weijters, W.M.P. van der Aalst, A.K. Alves de Medeiros, Process Mining with the HeuristicsMiner algorithm, BETA Working Paper Series 166, TU Eindhoven, 2006.
18. C.W. Günther, W.M.P. van der Aalst, Fuzzy mining - adaptive process simplification based on multi-perspective metrics, in: G. Alonso, P. Dadam, M. Rosemann (Eds.), Proceedings of the 5th International Conference on Business Process Management, BPM 2007, Lecture Notes in Computer Science, vol. 4714, Brisbane, Australia, Springer, September 24-28, 2007, pp. 328-343.
19. L. Maruster, A.J.M.M. Weijters, W.M.P. van der Aalst, and A. van den Bosch A rule-based approach for process discovery: dealing with noise and imbalance in process logs Data Mining and Knowledge Discovery 13 2006 67 87 (Pubitemid 43882557)
20. H. Ferreira, and D. Ferreira An integrated life cycle for workflow management based on learning and planning International Journal of Cooperative Information Systems 15 2006 485 505 (Pubitemid 44600117)
21. E. Lamma, P. Mello, M. Montali, F. Riguzzi, S. Storari, Inducing declarative logic-based models from labeled traces, in: G. Alonso, P. Dadam, M. Rosemann (Eds.), Proceedings of the 5th International Conference on Business Process Management, BPM 2007, Lecture Notes in Computer Science, vol. 4714, Brisbane, Australia, Springer, September 24-28, 2007, pp. 344-359.
22. A.K. Alves de Medeiros, A.J.M.M. Weijters, and W.M.P. van der Aalst Genetic process mining: an experimental evaluation Data Mining and Knowledge Discovery 14 2007 245 304 (Pubitemid 46418257)
23. S. Goedertier, D. Martens, J. Vanthienen, and B. Baesens Robust process discovery with artificial negative events Journal of Machine Learning Research 10 2009 1305 1340
24. H. Blockeel, and L. De Raedt Top-down induction of first-order logical decision trees Artificial Intelligence 101 1998 285 297 (Pubitemid 128387397)
25. G. Greco, A. Guzzo, L. Pontieri, and D. Saccà Discovering expressive process models by clustering log traces IEEE Transactions on Knowledge and Data Engineering 18 2006 1010 1027 (Pubitemid 44109532)
26. G. Greco, A. Guzzo, and L. Pontieri Mining taxonomies of process models Data & Knowledge Engineering 67 2008 74 102
27. W.M.P. van der Aalst, V. Rubin, B.F. van Dongen, E. Kindler, C.W. Günther, Process Mining: A Two-Step Approach using Transition Systems and Regions, BPM-06-30, BPM Center Report, 2006.
28. W.M.P. van der Aalst, V. Rubin, H.M.W. Verbeek, B.F. van Dongen, E. Kindler, and C.W. Günther Process mining: a two-step approach to balance between underfitting and overfitting Software and System Modeling 9 2010 87 111
29. J. Carmona, J. Cortadella, and M. Kishinevsky New region-based algorithms for deriving bounded Petri nets IEEE Transactions on Computers 59 2010 371 384
30. J. Herbst, and D. Karagiannis Workflow mining with InWoLvE Computers in Industry 53 2004 245 264
31. B.F. van Dongen, W.M.P. van der Aalst, Multi-phase process mining: aggregating instance graphs into EPCs and Petri nets, in: Proceedings of the 2nd International Workshop on Applications of Petri Nets to Coordination, Workflow and Business Process Management (PNCWB).
32. W. Gaaloul, K. Baïna, and C. Godart Towards mining structural workflow patterns K.V. Andersen, J.K. Debenham, R. Wagner, DEXA, Lecture Notes in Computer Science vol. 3588 2005 Springer 24 33 (Pubitemid 41450438)
33. A.J.M.M. Weijters, and W.M.P. van der Aalst Rediscovering workflow models from event-based data using little thumb Integrated Computer-Aided Engineering 10 2003 151 162
34. A.K. Alves de Medeiros, Genetic Process Mining, Ph.D. Thesis, TU Eindhoven, 2006.
35. F. Folino, G. Greco, A. Guzzo, and L. Pontieri Discovering expressive process models from noised log data B.C. Desai, D. Saccà, S. Greco, IDEAS, ACM International Conference Proceeding Series 2009 ACM 162 172
36. D.R. Ferreira, D. Gillblad, Discovering process models from unlabelled event logs, in: U. Dayal, J. Eder, J. Koehler, H.A. Reijers (Eds.), Proceedings of the 7th International Conference on Business Process Management, BPM 2009, Lecture Notes in Computer Science, vol. 5701, Ulm, Germany, Springer, September 8-10, 2009, pp. 143-158.
37. J.M.E.M. van der Werf, B.F. van Dongen, C.A.J. Hurkens, and A. Serebrenik Process discovery using integer linear programming Fundamenta Informaticae 94 2009 387 412
38. C.J. van Rijsbergen Information Retrieval 1979 Butterworth
39. A. Rozinat, M. Veloso, W.M.P. van der Aalst, Using Hidden Markov Models to Evaluate the Quality of Discovered Process Models, BPM Center Report BPM-08-10, 2008.
40. A. Rozinat, and W.M.P. van der Aalst Conformance checking of processes based on monitoring real behavior Information Systems 33 2008 64 95 (Pubitemid 350051578)
41. A. Rozinat, Process Mining: Conformance and Extension, Ph.D. Thesis, TU Eindhoven, 2010.
42. J. De Weerdt, M. De Backer, J. Vanthienen, B. Baesens, A robust F-measure for evaluating discovered process models, in: IEEE Symposium Series in Computational Intelligence (SSCI 2011), Paris, France, 2011.
43. J. Munoz-Gama, and J. Carmona A fresh look at precision in process conformance R. Hull, J. Mendling, S. Tai, Business Process Management, Lecture Notes in Computer Science vol. 6336 2010 Springer 211 226
44. A.K. Alves de Medeiros, W.M.P. van der Aalst, and A.J.M.M. Weijters Quantifying process equivalence based on observed behavior Data & Knowledge Engineering 64 2008 55 74 (Pubitemid 350087285)
45. T. Calders, C.W. Günther, M. Pechenizkiy, and A. Rozinat Using minimum description length for process mining S.Y. Shin, S. Ossowski, SAC 2009 ACM 1451 1455
46. J. De Weerdt, M. De Backer, J. Vanthienen, and B. Baesens A critical evaluation study of model-log metrics in process discovery M. zur Muehlen, J. Su, Business Process Management Workshops, Lecture Notes in Business Information Processing vol. 66 2010 Springer 158 169
47. S. Goedertier, J. De Weerdt, D. Martens, J. Vanthienen, and B. Baesens Process discovery in event logs: an application in the telecom industry Applied Soft Computing 11 2011 1697 1710
48. C.W. Günther, Process Mining in Flexible Environments, Ph.D. Thesis, TU Eindhoven, 2009.
49. J. Mendling, H.A. Reijers, J. Cardoso, What makes process models understandable? in: G. Alonso, P. Dadam, M. Rosemann (Eds.), Proceedings of the 5th International Conference on Business Process Management, BPM 2007, Lecture Notes in Computer Science, vol. 4714, Brisbane, Australia, Springer, September 24-28, 2007, pp. 48-63.
50. K.B. Lassen, and W.M.P. van der Aalst Complexity metrics for workflow nets Information & Software Technology 51 2009 610 626
51. J. Cardoso, Evaluating the process control-flow complexity measure, in: ICWS, IEEE Computer Society, 2005, pp. 803-804. (Pubitemid 44460284)
52. T.J. McCabe A complexity measure IEEE Transactions on Software Engineering 2 1976 308 320
53. J. Demsar Statistical comparisons of classifiers over multiple data sets Journal of Machine Learning Research 7 2006 1 30 (Pubitemid 43022939)
54. M. Friedman A comparison of alternative tests of significance for the problem of m rankings The Annals of Mathematical Statistics 11 1940 86 92
55. O.J. Dunn Multiple comparisons among means Journal of the American Statistical Association 56 1961 52 64
56. I.T. Jolliffe Principal Component Analysis 2nd ed. 2002 Springer
57. H. Hotelling Relations between two sets of variates Biometrika 28 1936 321 377
58. C. Bratosin, N. Sidorova, and W.M.P. van der Aalst Discovering process models with genetic algorithms using sampling R. Setchi, I. Jordanov, R.J. Howlett, L.C. Jain, KES (1), Lecture Notes in Computer Science vol. 6276 2010 Springer 41 50
59. R. Cattell Scree test for number of factors Multivariate Behavioral Research 1 1966 245 276
60. D. Stewart, and W. Love A general canonical correlation index Psychological Bulletin 70 1968 160 163
61. R.P.J.C. Bose, W.M.P. van der Aalst, Abstractions in process mining: a taxonomy of patterns, in: U. Dayal, J. Eder, J. Koehler, H.A. Reijers (Eds.), Proceedings of the 7th International Conference on Business Process Management, BPM 2009, Lecture Notes in Computer Science, vol. 5701, Ulm, Germany, Springer, 2009, September 8-10, pp. 159-175.
62. M. Song, C.W. Günther, and W.M.P. van der Aalst Trace clustering in process mining D. Ardagna, M. Mecella, J. Yang, Business Process Management Workshops, Lecture Notes in Business Information Processing vol. 17 2008 Springer 109 120
63. D.R. Ferreira, M. Zacarias, M. Malheiros, P. Ferreira, Approaching process mining with sequence clustering: experiments and findings, in: G. Alonso, P. Dadam, M. Rosemann (Eds.), Proceedings of the 5th International Conference on Business Process Management, BPM 2007, Lecture Notes in Computer Science, vol. 4714, Brisbane, Australia, Springer, September 24-28, 2007, pp. 360-374.
64. W.M.P. van der Aalst, M. Pesic, and H. Schonenberg Declarative workflows: balancing between flexibility and support Computer Science - R&D 23 2009 99 113
65. W.M.P. van der Aalst Process Mining - Discovery, Conformance and Enhancement of Business Processes 2011 Springer
66. W. van der Aalst The application of petri nets to workflow management Journal of Circuits, Systems, and Computers 8 1998 21 66 (Pubitemid 128540249)
67. R.P.J.C. Bose, and W.M.P. van der Aalst Process diagnostics using trace alignment: opportunities, issues, and challenges Information Systems 37 2012 117 141