Automated Discovery of Process Models from Event Logs: Review and Benchmark

IEEE Transactions on Knowledge and Data Engineering

Volumn 31, Issue 4, 2019, Pages 686-705

  Augusto, Adriano ^a,b   Conforti, Raffaele ^b   Dumas, Marlon ^a   La Rosa, Marcello ^b   Maggi, Fabrizio Maria ^a   Marrella, Andrea ^c   Mecella, Massimo ^c   Soo, Allar ^a  

^a UNIVERSITY OF TARTU   (Estonia)

^b UNIVERSITY OF MELBOURNE   (Australia)

^c SAPIENZA UNIVERSITY OF ROME   (Italy)

Author keywords

automated process discovery; benchmark; Process mining; survey

Indexed keywords

AUTOMATION; BENCHMARKING; COMMERCE; DATA MINING; DATA STRUCTURES; JOB ANALYSIS; OPEN SYSTEMS; SCALABILITY; SURVEYING;

AUTOMATED PROCESS; BENCHMARK TESTING; PROCESS MINING; SYSTEMATICS; TASK ANALYSIS;

PROCESS CONTROL;

EID: 85047825679     PISSN: 10414347     EISSN: 15582191     Source Type: Journal    
DOI: 10.1109/TKDE.2018.2841877     Document Type: Review

References (122)

1. W. M. P. van der Aalst, A. J. M. M. Weijters, and L. Maruster, "Workflow mining: Discovering process models from event logs, " IEEE Trans. Knowl. Data Eng., vol. 16, no. 9, pp. 1128-1142, Sep. 2004.
2. W. M. P. van der Aalst, Process Mining: Data Science in Action. Berlin, Germany: Springer, 2016.
3. J. De Weerdt, M. De Backer, J. Vanthienen, and B. Baesens, "A multi-dimensional quality assessment of state-of-the-art process discovery algorithms using real-life event logs, " Inf. Syst., vol. 37, no. 7, pp. 654-676, 2012.
4. B. Kitchenham, "Procedures for performing systematic reviews, " Keele University, Keele, U.K., Tech. Rep. TR/SE-0401, 2004.
5. W. M. P. van der Aalst, A. Kalenkova, V. Rubin, and H. M. W. Verbeek, "Process discovery using localized events, " in Proc. Int. Conf. Appl. Theory Petri Nets Concurrency, 2015, pp. 287-308.
6. F. Folino, M. Guarascio, and L. Pontieri, "On the discovery of explainable and accurate behavioral models for complex lowlystructured business processes, " in Proc. 17th Int. Conf. Enterprise Inf. Syst., 2015, pp. 206-217.
7. A. Fink, Conducting Research Literature Reviews: From the Internet to Paper, 3rd ed. Newbury Park, CA, USA: Sage, 2010.
8. C. Okoli and K. Schabram, "A guide to conducting a systematic literature review of information systems research, " Sprouts: Work. Papers Inf. Syst., vol. 10, no. 26, pp. 1-49, 2010.
9. J. Randolph, "A guide towriting the dissertation literature review, " Practical Assessment Res. Eval., vol. 14, no. 13, pp. 1-13, 2009.
10. R. Torraco, "Writing integrative literature reviews: Guidelines and examples, " Human Resource Develop. Rev., vol. 4, no. 3, pp. 356-367, 2005.
11. W. M. P. van der Aalst, A. J. M. M. Weijters, and L. Maruster, "Workflow mining: Discovering process models from event logs, " IEEE Trans. Knowl. Data Eng., vol. 16, no. 9, pp. 1128-1142, Sep. 2004.
12. A. K. Alves de Medeiros, B. F. van Dongen, W. M. P. van der Aalst, and A. J. M. M. Weijters, "Process mining: Extending the a-algorithm to mine short loops, " BETA Working Paper Series, vol. 113, pp. 145-180, 2004.
13. L. Wen, W. M. P. van der Aalst, J. Wang, and J. Sun, "Mining process models with non-free-choice constructs, " Data Mining Knowl. Discovery, vol. 15, no. 2, pp. 145-180, 2007.
14. Q. Guo, L. Wen, J. Wang, Z. Yan, and S. Y. Philip, "Mining invisible tasks in non-free-choice constructs, " in Proc. Int. Conf. Bus. Process Manage., 2015, pp. 109-125.
15. S. Goedertier, D. Martens, J. Vanthienen, and B. Baesens, "Robust process discovery with artificial negative events, " J. Mach. Learn. Res., vol. 10, no. Jun, pp. 1305-1340, 2009.
16. A. K. Alves deMedeiros and A. J.M. M.Weijters, "Genetic process mining, " in Proc. Int. Conf.Appl. Theory Petri Nets, 2005, pp. 48-69.
17. A. K. Alves de Medeiros, A. J. M. M. Weijters, and W. M. P. van der Aalst, "Genetic process mining: An experimental evaluation, " Data Mining Knowl. Discovery, vol. 14, no. 2, pp. 245-304, 2007.
18. J. C. A. M. Buijs, B. F. van Dongen, and W. M. P. van der Aalst, "Quality dimensions in process discovery: The importance of fitness, precision, generalization and simplicity, " Int. J. Cooperative Inf. Syst., vol. 23, no. 1, 2014, Art. no. 1440001.
19. A. J. M. M. Weijters and W. M. P. van der Aalst, "Rediscovering workflow models from event-based data using little thumb, " Integr. Comput.-Aided Eng., vol. 10, no. 2, pp. 151-162, 2003.
20. A. J. M. M. Weijters, W. M. P. van der Aalst, and A. K. Alves de Medeiros, "Process mining with the heuristics miner-algorithm, " Technische Universiteit Eindhoven, Tech. Rep. WP, vol. 166, pp. 1-34, 2006.
21. A. Augusto, R. Conforti, M. Dumas, and M. La Rosa, "Automated discovery of structured process models from event logs: The discover-and-structure approach, " Data Knowl. Eng., 2018, doi: https://doi.org/10.1016/j.datak.2018.04.007.
22. F. Mannhardt, M. de Leoni, H. A. Reijers, and W. M. P. van der Aalst, "Data-driven process discovery-revealing conditional infrequent behavior from event logs, " in Proc. Int. Conf. Adv. Inf. Syst. Eng., 2017, pp. 545-560.
23. A. J. M. M. Weijters and J. Ribeiro, "Flexible heuristics miner (FHM), " in Proc. IEEE Symp. Comput. Intell. Data Mining, 2011, pp. 310-317.
24. S. De Cnudde, J. Claes, and G. Poels, "Improving the quality of the Heuristics Miner in ProM 6.2, " Expert Syst. Appl., vol. 41, no. 17, pp. 7678-7690, 2014.
25. J. M. E. van der Werf, B. F. van Dongen, C. A. Hurkens, and A. Serebrenik, "Process discovery using Integer Linear Programming, " Fundam. Inf., vol. 94, no. 3-4, pp. 387-412, 2009.
26. S. J. van Zelst, B. F. van Dongen, W. M. P. van der Aalst, and H. M. W. Verbeek, "Discovering Workflow nets using Integer Linear Programming, " Comput., pp. 1-28, 2017.
27. Z. Huang and A. Kumar, "A study of quality and accuracy tradeoffs in process mining, " INFORMS J. Comput., vol. 24, no. 2, pp. 311-327, 2012.
28. F. M. Maggi, R. P. J. C. Bose, and W. M. P. van der Aalst, "Efficient discovery of understandable declarative process models from event logs, " in Proc. 24th Int. Conf. Adv. Inf. Syst. Eng., 2012, pp. 270-285.
29. F. M. Maggi, A. J. Mooij, and W. M. P. van der Aalst, "Userguided discovery of declarative process models, " in Proc. IEEE Symp. Comput. Intell. Data Mining, 2011, pp. 192-199.
30. F.M. Maggi, R. P. J. C. Bose, and W.M. P. van der Aalst, "A knowledge-based integrated approach for discovering and repairing declare maps, " in Proc. Int. Conf. Adv. Inf. Syst. Eng., 2013, pp. 433-448.
31. M. L. Bernardi, M. Cimitile, and F. M. Maggi, "Discovering crossorganizational business rules from the cloud, " in Proc. IEEE Symp. Comput. Intell. Data Mining, 2014, pp. 389-396.
32. F. M. Maggi, "Discovering metric temporal business constraints from event logs, " in Proc. 13th Int. Conf. Perspectives Bus. Inf. Res., 2014, pp. 261-275.
33. T. Kala, F. M. Maggi, C. Di Ciccio, and C. Di Francescomarino, "Apriori and sequence analysis for discovering declarative process models, " in Proc. IEEE 20th Int. Enterprise Distrib. Object Comput. Conf., 2016, pp. 1-9.
34. F. M. Maggi, C. Di Ciccio, C. Di Francescomarino, and T. Kala, "Parallel algorithms for the automated discovery of declarative process models, " Inf. Syst., vol. 74, pp. 136-152, 2018.
35. C. Di Ciccio and M. Mecella, "A two-step fast algorithm for the automated discovery of declarative workflows, " in Proc. IEEE Symp. Comput. Intell. Data Mining, 2013, pp. 135-142.
36. C. Di Ciccio and M. Mecella, "Mining constraints for artful processes, " in Proc. 15th Int. Conf. Bus. Inf. Syst., 2012, pp. 11-23.
37. C. Di Ciccio and M. Mecella, "On the discovery of declarative control flows for artful processes, " ACM Trans. Manage. Inf. Syst., vol. 5, no. 4, pp. 24:1-24:37, 2015.
38. C. Di Ciccio, F. M. Maggi, and J. Mendling, "Discovering targetbranched declare constraints, " in Proc. Int. Conf. Bus. Process Manage., 2014, pp. 34-50.
39. C. Di Ciccio, F. M. Maggi, and J. Mendling, "Efficient discovery of target-branched declare constraints, " Inf. Syst., vol. 56, pp. 258-283, 2016.
40. S. J. J. Leemans, D. Fahland, and W. M. P. van der Aalst, "Discovering block-structured process models from event logs containing infrequent behaviour, " in Proc. Int. Bus. Process Manage. Workshops, 2013, pp. 66-78.
41. S. J. J. Leemans, D. Fahland, and W. M. P. van der Aalst, "Discovering block-structured process models from event logs-A constructive approach, " in Proc. 34th Int. Conf. Appl. Theory Petri Nets Concurrency, 2013, pp. 311-329.
42. S. J. J. Leemans, D. Fahland, and W. M. P. van der Aalst, "Discovering block-structured process models from incomplete event logs, " in Proc. 35th Int. Conf. Appl. Theory Petri Nets Concurrency, 2014, pp. 91-110.
43. S. J. J. Leemans, D. Fahland, and W. M. P. van der Aalst, "Exploring processes and deviations, " in Proc. Int. Workshops Bus, Process Manage., 2014, pp. 304-316.
44. S. J. J. Leemans, D. Fahland, and W. M. P. van der Aalst, "Using life cycle information in process discovery, " in Proc. 13th Int. Workshops Bus. Process Manage., 2015, pp. 204-217.
45. S. J. J. Leemans, D. Fahland, and W. M. P. van der Aalst, "Scalable process discovery with guarantees, " in Proc. Int. Conf. Enterprise, Bus.-Process Inf. Syst. Model., 2015, pp. 85-101.
46. S. J. J. Leemans, D. Fahland, and W. M. P. van der Aalst, "Scalable process discovery and conformance checking, " Softw. Syst. Model., vol. 17, no. 2, pp. 599-631, 2018.
47. S. J. J. Leemans and W. M. P. van der Aalst, "Modeling and discovering cancelation behavior, " in Proc. OTM Confederated Int. Conf. Move Meaningful Internet Syst., 2017, pp. 93-113.
48. F. M. Maggi, M. Dumas, L. García-Bañuelos, and M. Montali, "Discovering data-aware declarative process models from event logs, " in Business Process Management. Berlin, Germany: Springer, 2013, pp. 81-96.
49. M. Abe and M. Kudo, "Business monitoring framework for process discovery with real-life logs, " in Proc. Int. Conf. Bus. Process Manage., 2014, pp. 416-423.
50. M. Kudo, A. Ishida, and N. Sato, "Business process discovery by using process skeletonization, " in Proc. Int. Conf. Signal-Image Technol. Internet-Based Syst., 2013, pp. 976-982.
51. S. K. L. M. vanden Broucke, J. Vanthienen, and B. Baesens, "Declarative process discovery with evolutionary computing, " in IEEE Congress Evol. Comput., 2014, pp. 2412-2419.
52. W. M. P. van der Aalst, J. C. A. M. Buijs, and B. F. van Dongen, "Towards improving the representational bias of process mining, " in Proc. 1st Int. Symp. Data-Driven Process Discovery Anal., 2011, pp. 39-54.
53. J. C. A. M. Buijs, B. F. van Dongen, and W. M. P. van der Aalst, "A genetic algorithm for discovering process trees, " in Proc. IEEE Congress Evol. Comput., 2012, pp. 1-8.
54. J. C. A. M. Buijs, B. F. van Dongen, W. M. P. van der Aalst, "On the role of fitness, precision, generalization and simplicity in process discovery, " in Proc. OTM Conf., 2012, pp. 305-322.
55. J. C. A. M. Buijs, B. F. van Dongen, and W. M. P. van der Aalst, "Discovering and navigating a collection of process models using multiple quality dimensions, " in Proc. Int. Workshops Bus. Process Manage., 2013, pp. 3-14.
56. M. L. van Eck, J. C. A. M. Buijs, and B. F. van Dongen, "Genetic process mining: Alignment-based process model mutation, " in Proc. Int. Workshops Bus. Process Manage., 2014, pp. 291-303.
57. J. Carmona and J. Cortadella, "Process discovery algorithms using numerical abstract domains, " IEEE Trans. Knowl. Data Eng., vol. 26, no. 12, pp. 3064-3076, Dec. 2014.
58. S. Ferilli, "WoMan: Logic-based workflow learning and management, " IEEE Trans. Syst. Man Cybern.: Syst., vol. 44, no. 6, pp. 744-756, Jun. 2014.
59. S. Ferilli, B. De Carolis, and D. Redavid, "Logic-based incremental process mining in smart environments, " in Proc. Int. Conf. Ind. Eng. Other Appl. Appl. Intell. Syst., 2013, pp. 392-401.
60. B. De Carolis, S. Ferilli, and G. Mallardi, "Learning and recognizing routines and activities in SOFiA, " in Proc. Eur. Conf. Ambient Intell., 2014, pp. 191-204.
61. S. Ferilli, B. De Carolis, and F. Esposito, "Learning complex activity preconditions in process mining, " in Proc. 3rd Int. Workshop New Frontiers Mining Complex Patterns, 2014, pp. 164-178.
62. S. Ferilli, D. Redavid, and F. Esposito, "Logic-based incremental process mining, " in Proc. Joint Eur. Conf. Mach. Learn. Knowl. Discovery Databases, 2015, pp. 218-221.
63. S. Ferilli, "The WoMan formalism for expressing process models, " in Proc. 16th Ind. Conf. Advances Data Mining. Appl. Theoretical Aspects, 2016, pp. 363-378.
64. F. M. Maggi, T. Slaats, and H. A. Reijers, "The automated discovery of hybrid processes, " in Proc. Int. Conf. Bus. Process Manage., 2014, pp. 392-399.
65. D. Redlich, T. Molka, W. Gilani, G. S. Blair, and A. Rashid, "Scalable dynamic business process discovery with the constructs competition miner, " in Proc. 4th Int. Symp. Data-Driven Process Discovery Anal., 2014, pp. 91-107.
66. D. Redlich, T. Molka, W. Gilani, G. Blair, and A. Rashid, "Constructs competition miner: Process control-flow discovery of bp-domain constructs, " in Proc. Int. Conf. Bus. Process Manage., 2014, pp. 134-150.
67. D. Redlich, W. Gilani, T. Molka, M. Drobek, A. Rashid, and G. Blair, "Introducing a framework for scalable dynamic process discovery, " in Proc. Enterprise Eng. Work. Conf., 2014, pp. 151-166.
68. D. Redlich, T. Molka, W. Gilani, G. Blair, and A. Rashid, "Dynamic constructs competition miner-occurrence-vs. timebased ageing, " in Proc. Int. Symp. Data-Driven Process Discovery Anal., 2014, pp. 79-106.
69. O. Vasilecas, T. Savickas, and E. Lebedys, "Directed acyclic graph extraction from event logs, " in Proc. Int. Conf. Inf. Softw. Technol., 2014, pp. 172-181.
70. J. De Smedt, J. De Weerdt, and J. Vanthienen, "Fusion miner: Process discovery for mixed-paradigm models, " Decision Support Syst., vol. 77, pp. 123-136, 2015.
71. G. Greco, A. Guzzo, F. Lupia, and L. Pontieri, "Process discovery under precedence constraints, " ACM Trans Knowl. Discovery Data, vol. 9, no. 4, 2015, Art. no. 32.
72. G. Greco, A. Guzzo, and L. Pontieri, "Process discovery via precedence constraints, " in Proc. 20th Eur. Conf. Artif. Intell., 2012, pp. 366-371.
73. V. Liesaputra, S. Yongchareon, and S. Chaisiri, "Efficient process model discovery using maximal pattern mining, " in Proc. Int. Conf. Bus. Process Manage., 2015, pp. 441-456.
74. T. Molka, D. Redlich, M. Drobek, X.-J. Zeng, and W. Gilani, "Diversity guided evolutionary mining of hierarchical processmodels, " in Proc. Annu. Conf. Genetic Evol. Comput., 2015, pp. 1247-1254.
75. B. Vázquez-Barreiros, M. Mucientes, and M. Lama, "ProDiGen: Mining complete, precise and minimal structure process models with a genetic algorithm, " Inf. Sci., vol. 294, pp. 315-333, 2015.
76. B. Vázquez-Barreiros, M. Mucientes, and M. Lama, "A genetic algorithm for process discovery guided by completeness, precision and simplicity, " in Proc. Int. Conf. Bus. Process Manage., 2014, pp. 118-133.
77. M. L. Bernardi, M. Cimitile, C. Di Francescomarino, and F. M. Maggi, "Do activity lifecycles affect the validity of a business rule in a business process?" Inf. Syst., vol. 62. pp. 42-59, 2016.
78. M. L. Bernardi, M. Cimitile, C. Di Francescomarino, and F. M. Maggi, "Using discriminative rule mining to discover declarative process models with non-atomic activities, " in Proc. 8th Int. Symp. RuleML Rules Web. From Theory Appl. Co-Located 21st Eur. Conf. Artif. Intell., 2014, pp. 281-295.
79. D. Breuker, M. Matzner, P. Delfmann, and J. Becker, "Comprehensible predictive models for business processes, " MIS Quart., vol. 40, no. 4, pp. 1009-1034, 2016.
80. D. Breuker, P. Delfmann, M. Matzner, and J. Becker, "Designing and evaluating an interpretable predictive modeling technique for business processes, " in Proc. Int. Workshops Bus. Process Manage., 2014, pp. 541-553.
81. R. Conforti, M. Dumas, L. García-Bañuelos, and M. La Rosa, "BPMN miner: Automated discovery of BPMN process models with hierarchical structure, " Inf. Syst., vol. 56, pp. 284-303, 2016.
82. R. Conforti, M. Dumas, L. García-Bañuelos, and M. La Rosa, "Beyond tasks and gateways: Discovering BPMN models with subprocesses, boundary events and activity markers, " in Proc. Int. Conf. Bus. Process Manage., 2014, pp. 101-117.
83. M. L. van Eck, N. Sidorova, and W. M. P. van der Aalst, "Discovering and exploring state-based models for multi-perspective processes, " in Proc. Int. Conf. Bus. Process Manage., 2016, pp. 142-157.
84. M. L. van Eck, N. Sidorova, and W. M. P. van der Aalst, "Guided interaction exploration in artifact-centric process models, " in Proc. 19th IEEE Conf. Bus. Inf., 2017, pp. 109-118.
85. C. Li, J. Ge, L. Huang, H. Hu, B. Wu, H. Yang, H. Hu, and B. Luo, "Process mining with token carried data, " Inf. Sci., vol. 328, pp. 558-576, 2016.
86. A. Mokhov, J. Carmona, and J. Beaumont, "Mining conditional partial order graphs from event logs, " in Transactions on Petri Nets and Other Models of Concurrency XI. Berlin, Germany: Springer, 2016, pp. 114-136.
87. S. Schönig, A. Rogge-Solti, C. Cabanillas, S. Jablonski, and J. Mendling, "Efficient and customisable declarative process mining with SQL, " in Proc. Int. Conf. Adv. Inf. Syst. Eng., 2016, pp. 290-305.
88. S. Schönig, C. Di Ciccio, F. M. Maggi, and J. Mendling, "Discovery of multi-perspective declarative process models, " in Proc. Int. Conf. Serv.-Oriented Comput., 2016, pp. 87-103.
89. W. Song, H.-A. Jacobsen, C. Ye, and X. Ma, "Process discovery from dependence-complete event logs, " IEEE Trans. Serv. Comput., vol. 9, no. 5, pp. 714-727, Sep./Oct. 2016.
90. T. Tapia-Flores, E. Rodríguez-Pérez, and E. López-Mellado, "Discovering process models from incomplete event logs using conjoint occurrence classes, " in Proc. ATAED@ Petri Nets/ACSD, 2016, pp. 31-46.
91. B. N. Yahya, M. Song, H. Bae, S.-O. Sul, and J.-Z. Wu, "Domaindriven actionable process model discovery, " Comput. Industrial Eng., vol. 99, no. C, pp. 382-400, 2016.
92. B. N. Yahya, H. Bae, S.-O. Sul, and J.-Z. Wu, "Process discovery by synthesizing activity proximity and user's domain knowledge, " in Proc. Asia-Pacific Conf. Bus. ProcessManage., 2013, pp. 92-105.
93. A. Augusto, R. Conforti, M. Dumas, M. La Rosa, and G. Bruno, "Automated discovery of structured process models: Discover structured versus discover and structure, " in Proc. 35th Int. Conf. Conceptual Model., 2016, pp. 313-329.
94. A. Augusto, R. Conforti, M. Dumas, and M. La Rosa, "Split miner: Discovering accurate and simple business process models from event logs, " in Proc. IEEE Int. Conf. Data Mining, New Orleans, LA, USA, Nov. 18-21, 2017, pp. 1-10.
95. S. K. L. M. vanden Broucke and J. De Weerdt, "Fodina: A robust and flexible heuristic process discovery technique, " Decision Support Syst., vol. 100, pp. 109-118, 2017.
96. J. De Weerdt, S. K. L. M. vanden Broucke, and F. Caron, "Bidimensional process discovery for mining BPMN models, " in Proc. Int. Workshops Bus. Process Manage., Eindhoven, The Netherlands, Sep. 7-8, 2014, pp. 529-540.
97. H. Nguyen, M. Dumas, A. H. M. ter Hofstede, M. La Rosa, and F. M. Maggi, "Mining business process stages from event logs, " in Proc. Int. Conf. Adv. Inf. Syst. Eng., 2017, pp. 577-594.
98. H. M. W. Verbeek, W. M. P. van der Aalst, and J. Munoz-Gama, "Divide and conquer: A tool framework for supporting decomposed discovery in process mining, " Comput. J., vol. 60, pp. 1-26, 2017.
99. H. M. W. Verbeek and W. M. P. van der Aalst, "An experimental evaluation of passage-based process discovery, " in Proc. Int. Workshop Bus. Process Intell. Bus. Process Manage., vol. 132, pp. 205-210, 2012.
100. W. M. P. van der Aalst, "Decomposing Petri nets for process mining: A generic approach, " Distrib. Parallel Databases, vol. 31, no. 4, pp. 471-507, 2013.
101. B. Hompes, H. M. W. Verbeek, and W. M. P. van der Aalst, "Finding suitable activity clusters for decomposed process discovery, " in Proc. Int. Symp. Data-Driven Process Discovery Anal., 2014, pp. 32-57.
102. H. M. W. Verbeek and W. M. P. van der Aalst, "Decomposed processmining: The ILP case, " in Proc. Int. Workshops Bus. ProcessManage., Eindhoven, TheNetherlands, Sep. 7-8, 2014, pp. 264-276.
103. W. M. P. van der Aalst and H. M. W. Verbeek, "Process discovery and conformance checking using passages, " Fund. Inf., vol. 131, no. 1, pp. 103-138, 2014.
104. S. J. van Zelst, B. F. van Dongen, and W. M. P. van der Aalst, "Avoiding over-fitting in ILP-based process discovery, " in Int. Conf. Bus. Process Manage., 2015, pp. 163-171.
105. S. J. van Zelst, B. F. van Dongen, and W. M. P. van der Aalst, "ILP-based process discovery using hybrid regions, " in Proc. Int. Workshop Algorithms Theories Anal. Event Data, 2015, pp. 47-61.
106. M. Pesic, H. Schonenberg, and W.M. P. van der Aalst, "DECLARE: Full support for loosely-structured processes, " in Proc. 11th IEEE Int. Enterprise Distrib. Object Comput. Conf., 2007, pp. 287-300.
107. M. Westergaard and F. M. Maggi, "Declare: A tool suite for declarative workflow modeling and enactment, " in Proc. Demo Track 9th Conf. Bus. Process Manage., vol. 820, pp. 1-5, 2011.
108. T. Slaats, D. M. M. Schunselaar, F. M. Maggi, and H. A. Reijers, "The semantics of hybrid process models, " in Proc. OTM Confederated Int. Conf. Move Meaningful Internet Syst., 2016, pp. 531-551.
109. M.Westergaard and T. Slaats, "Mixing paradigms formore comprehensible models, " inProc. Bus. ProcessManage., 2013, pp. 283-290.
110. C. Favre, D. Fahland, and H. Völzer, "The relationship between workflow graphs and free-choice workflow nets, " Inf. Syst., vol. 47, pp. 197-219, 2015.
111. A. Adriansyah, B. F. van Dongen, and W. M. P. van der Aalst, "Conformance checking using cost-based fitness analysis, " in Proc. 15th IEEE Int. EnterpriseDistrib.Object Comput. Conf., 2011, pp. 55-64.
112. A. Adriansyah, J. Munoz-Gama, J. Carmona, B. F. van Dongen, and W. M. P. van der Aalst, "Alignment based precision checking, " in Proc. BPM Workshops, vol. 132, pp. 137-149, 2012.
113. R. Kohavi, "A study of cross-validation and bootstrap for accuracy estimation and model selection, " in Proc. Int. Joint Conf. Artif. Intell., 1995, pp. 1137-1145.
114. A. Rozinat, A. K. Alves de Medeiros, C. Günther, A. J. M. M. Weijters, and W. M. P. van der Aalst, "Towards an evaluation framework for process mining algorithms, " BPM Center Report BPM-07-06, 2007.
115. A. Bolt, M. de Leoni, and W. M. P. van der Aalst, "Scientific workflows for process mining: Building blocks, scenarios, and implementation, " Softw. Tools Technol. Transfer, vol. 18, no. 6, pp. 607-628, 2016.
116. B. F. van Dongen, J. Carmona, T. Chatain, and F. Taymouri, "Aligning modeled and observed behavior: A compromise between computation complexity and quality, " in Proc. 29th Int. Conf. Adv. Inf. Syst. Eng., 2017, pp. 94-109.
117. J. Mendling, Metrics for Process Models: Empirical Foundations of Verification, Error Prediction, and Guidelines for Correctness. Berlin, Germany: Springer, 2008.
118. W. M. P. van der Aalst, Verification of Workflow Nets. Berlin, Germany: Springer, 1997, pp. 407-426.
119. R. Conforti, M. La Rosa, and A. H. M. ter Hofstede, "Filtering out infrequent behavior from business process event logs, " IEEE Trans. Knowl. Data Eng., vol. 29, no. 2, pp. 300-314, Feb. 2017.
120. W. M. P. van der Aalst, B. F. van Dongen, J. Herbst, L. Maruster, G. Schimm, and A. J. M. M. Weijters, "Workflow mining: A survey of issues and approaches, " Data Knowl. Eng., vol. 47, no. 2, pp. 237-267, 2003.
121. J. Claes and G. Poels, "Process Mining and the ProM Framework: An exploratory survey, " in Proc. Bus. Process Manage. Workshops, 2012, pp. 187-198.
122. S. K. L. M. vanden Broucke, J. De Weerdt, J. Vanthienen, and B. Baesens, "A comprehensive benchmarking framework (Co-BeFra) for conformance analysis between procedural process models and event logs in ProM, " in Proc. IEEE Symp. Comput. Intell. Data Mining, 2013, pp. 254-261.