A generic and customizable framework for the design of ETL scenarios

Information Systems

Volumn 30, Issue 7, 2005, Pages 492-525

  Vassiliadis, Panos ^a   Simitsis, Alkis ^b   Georgantas, Panos ^b   Terrovitis, Manolis ^b   Skiadopoulos, Spiros ^b  

^a UNIVERSITY OF IOANNINA   (Greece)

^b NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

Author keywords

Data warehousing; ETL

Indexed keywords

COMPUTER PROGRAMMING LANGUAGES; DATA ACQUISITION; DATA FLOW ANALYSIS; DATA STORAGE EQUIPMENT; DATABASE SYSTEMS; MATHEMATICAL MODELS; XML;

DATABASE PROGRAMMING LANGUAGES; EXTRACTION-TRANSFORMATION LOADING (ETL); METAMODELS; WORKFLOW-LIKE APPROACH;

DATA WAREHOUSES;

EID: 21244460030     PISSN: 03064379     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.is.2004.11.002     Document Type: Conference Paper

References (51)

1. IBM, IBM Data warehouse manager, available at http://www-3.ibm.com/ software/data/db2/datawarehouse/.
2. Informatica, Power Center, available at http://www.informatica.com/ products/data+integration/powercenter/default.htm.
3. Microsoft, Data transformation services, available at http://www.microsoft.com.
4. Ora, Oracle warehouse builder product page, available at http://otn.oracle.com/products/warehouse/content.html.
5. W.M.P. van der Aalst, A.H.M. ter Hofstede, B. Kiepuszewski, A.P. Barros. Workflow Patterns, BETA Working Paper Series, WP 47, Eindhoven University of Technology, Eindhoven, 2000, available at the Workflow Patterns web site, at tmit http://www.tm.tue.nl/research/patterns/documentation.htm.
6. P. Vassiliadis, A. Simitsis, S. Skiadopoulos, Modeling ETL activities as graphs, in: Proceedings of the Fourth International. Workshop on Design and Management of Data Warehouses (DMDW), pp. 52-61, Toronto, Canada, 2002.
7. P. Vassiliadis, A. Simitsis, P. Georgantas, M. Terrovitis, A framework for the design of ETL scenarios, in: Proceedings of the 15th Conference on Advanced Information Systems Engineering (CAiSE '03), pp. 520-535, Klagenfurt/Velden, Austria, 16-20 June, 2003.
8. R. Kimbal, L. Reeves, M. Ross, and W. Thornthwaite The Data Warehouse Lifecycle Toolkit Expert Methods for Designing, Developing, and Deploying Data Warehouses 1998 Wiley New York
9. Workflow Management Coalition, Interface 1: Process Definition Interchange Process Model, Document no. WfMC TC-1016-P, 1998, available at http://www.wfmc.org.
10. S. Naqvi, and S. Tsur A Logical Language for Data and Knowledge Bases 1989 Computer Science Press Rockville, MD
11. C. Zaniolo, LDL++ Tutorial, UCLA, http://pike.cs.ucla.edu/ldl/, December 1998.
12. D. Dori Conceptual modeling and system architecting Commun. ACM 46 10 2003 62 65
13. P. Vassiliadis, A. Simitsis, P. Georgantas, M. Terrovitis, S. Skiadopoulos, A generic and customizable framework for the design of ETL scenarios (long version), Technical Report TR-2004-1, Knowledge and Database Systems Laboratory, National Technical University of Athens, available at http://www.dbnet.ece.ntua.gr/pubs/.
14. Giga Information Group. Market Overview Update: ETL. Technical Report RPA-032002-00021, March 2002.
15. Ascential Software Inc., available at: http://www.ascentialsoftware.com.
16. Ascential Software Products - Data Warehousing Technology, available at: http://www.ascentialsoftware.com/products/datastage.html.
17. Gartner Inc., ETL magic quadrant update: market pressure increases, Gartner's Strategic Data Management Research Note, M-19-1108, January 2003.
18. P.A. Bernstein, and T. Bergstraesser Meta-data support for data transformations using Microsoft repository. Special issue on data transformations Bull. Tech. Committee Data Eng. 22 1 1999 9 14
19. Microsoft Corp, OLEDB specification, available at http://www.microsoft. com/data/oledb.
20. C. Graves, M. Scott, M. Benkovich, P. Turley, R. Skoglund, R. Dewson, S. Youness, D. Lee, S. Ferguson, T. Bain, and T. Joubert Professional SQL Server 2000 data warehousing with analysis services 1st ed 2001 Wrox Press Ltd.
21. Oracle, Oracle 9i Warehouse Builder, Architectural White paper, April 2002.
22. H. Galhardas, D. Florescu, D. Shasha, E. Simon, Ajax: An extensible data cleaning tool, in: Proceedings of the ACM SIGMOD International Conference on the Management of Data, pp. 590, Dallas, TX, 2000.
23. W. Cohen. Some practical observations on integration of Web information, in: WebDB'99 Workshop in conj. with ACM SIGMOD, 1999.
24. H. Galhardas, D. Florescu, D. Shasha, E. Simon, An extensible framework for data cleaning, Technical Report INRIA 1999 (RR-3742).
25. V. Raman, J. Hellerstein, Potters Wheel: an interactive framework for data cleaning and transformation, Technical Report University of California at Berkeley, Computer Science Division, 2000, available at http://www.cs.berkeley. edu/ rshankar/papers/pwheel.pdf.
26. V. Raman, J. Hellerstein, Potter's Wheel: an interactive data cleaning system, in: Proceedings of 27th International Conference on Very Large Data Bases (VLDB), pp. 381-390, Roma, Italy, 2001.
27. M. Jarke, M. Lenzerini, Y. Vassiliou, and P. Vassiliadis 2000 Springer New York
28. E. Rundensteiner Special issue on data transformations Bull. Tech. Committee Data Eng. 22 1 1999
29. S. Sarawagi Special issue on data cleaning Bull. Tech. Committee Data Eng. 23 4 2000
30. E. Rahm, and H. Hai Do Data cleaning problems and current approaches Bull. Tech. Committee Data Eng. 23 4 2000
31. V. Borkar, K. Deshmuk, and S. Sarawagi Automatically extracting structure form free text Addresses Bull. Tech. Committee Data Eng. 23 4 2000
32. A. Monge Matching algorithms within a duplicate detection system Bull. Tech. Committee Data Eng. 23 4 2000
33. A. Calì, D. Calvanese, G. De Giacomo, M. Lenzerini, P. Naggar, F. Vernacotola, IBIS: Semantic data integration at work, in: Proceedings of the 15th International Conference on Advanced Information Systems Engineering (CAiSE 2003), vol. 2681 of Lecture Notes in Computer Science, pp. 79-94. Springer, 2003.
34. A. Calì, D. Calvanese, G. De Giacomo, M. Lenzerini, Data integration under integrity constraints, in: Proceedings of the 14th International Conference on Advanced Information Systems Engineering (CAiSE 2002), vol. 2348 of Lecture Notes in Computer Science, pp. 262-279. Springer, 2002.
35. J. Eder, W, Gruber, A meta model for structured workflows supporting workflow transformations, in Proceedings of the Sixth East European Conference on Advances in Databases and Information Systems (ADBIS 2002), pp. 326-339, Bratislava, Slovakia, September 8-11, 2002.
36. W. Sadiq, M.E. Orlowska, On business process model transformations, 19th International Conference on Conceptual Modeling (ER 2000), Salt Lake City, UT, USA, October 9-12, 2000, pp. 267-280.
37. B. Kiepuszewski, A.H.M. ter Hofstede, C. Bussler, On structured workflow modeling, in: Proceedings of the 12th International Conference on Advanced Information Systems Engineering (CAiSE 2000), pp: 431-445, Stockholm, Sweden, June 5-9, 2000.
38. P. Dadam, M. Reichert, (eds.), Enterprise-wide and cross-enterprise workflow management: concepts, systems, applications, GI Workshop Informatik'99, 1999, available at http://www.informatik.uni-ulm.de/dbis/veranstaltungen/ Workshop-Informatik99-Proceedings.pdf.
39. M. Jarke, C. Quix, G. Blees, D. Lehmann, G. Michalk, S. Stierl, Improving OLTP Data Quality Using Data Warehouse Mechanisms, Proceedings of 1999 ACM SIGMOD International Conference on Management of Data, Philadelphia, USA, June 1999, pp. 537-538.
40. E. Schafer, J.-D. Becker, M. Jarke, DB-Prism: Integrated data warehouses and knowledge networks for bank controlling, Proceedings of the 26th International Conference on Very Large Databases, Cairo, Egypt, 2000.
41. M. Jarke, T. List, J. Koller, The challenge of process warehousing. Proceedings of the 26th International Conference on Very Large Databases, Cairo, Egypt, 2000.
42. P. Vassiliadis, A. Simitsis, S. Skiadopoulos, Conceptual modeling for ETL processes, in: Proceedings of the Fifth ACM International Workshop on Data Warehousing and OLAP (DOLAP), pp. 14-21, McLean, VA, USA, 2002.
43. A. Simitsis, P. Vassiliadis, A methodology for the conceptual modeling of ETL processes, in: Proceedings of the Decision Systems Engineering (DSE '03), Velden, Austria, June 17, 2003.
44. A. Simitsis, Modeling and managing ETL processes, in: Proceedings of the VLDB 2003 Ph.D Workshop, Berlin, Germany, September 12-13, 2003.
45. F. Casati, S. Ceri, B. Pernici, G. Pozzi. Conceptual Modeling of Workflows, in: Proceedings of the OO-ER Conference, Australia, 1995.
46. A.J. Albrecht, Measuring Application Development Productivity, in IBM Applications Development Symposium, Monterey, CA, 1979, pp. 83-92.
47. R.S. Pressman Software Engineering A Practitioner's Approach 5th ed 2000 McGraw-Hill New York
48. J. Adzic, V. Fiore, Data Warehouse Population Platform, in: Proceedings of the Fifth International Workshop on the Design and Management of Data Warehouses (DMDW'03), Berlin, Germany, September 2003.
49. W. Labio, J.L. Wiener, H. Garcia-Molina, V. Gorelik, Efficient resumption of interrupted warehouse loads, in: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data (SIGMOD 2000), pp. 46-57, Dallas, TX, USA, 2000.
50. J. Chen, S. Chen, E.A. Rundensteiner, A Transactional Model for Data Warehouse Maintenance, in: Proceedings of the of ER 2002, LNCS 2503, pp. 247-262, 2002.
51. B. Liu, S. Chen, E.A. Rundensteiner, A transactional approach to parallel data warehouse maintenance, in: Proceedings of DaWaK 2002, LNCS 2454, 2002. pp. 307-316.