Provenance from log files: A BigData problem

ACM International Conference Proceeding Series

Volumn , Issue , 2013, Pages 290-297

  Ghoshal, Devarshi ^a   Plale, Beth ^a  

^a INDIANA UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEX PROCESSES; DATA PRODUCTS; HIGH QUALITY; LOG FILE;

COMPUTER APPLICATIONS;

EID: 84876810217     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2457317.2457366     Document Type: Conference Paper

References (39)

1. W. Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, and I. Foster. The globus striped gridftp framework and server. In Proceedings of the 2005 ACM/IEEE conference on Supercomputing, SC '05, pages 54-, Washington, DC, USA, 2005. IEEE Computer Society.
2. Apache Logging. http://logging.apache.org.
3. R. Bose and J. Frew. Lineage retrieval for scientific data processing: a survey. ACM Comput. Surv., 37(1):1-28, Mar. 2005. (Pubitemid 41246320)
4. P. Buneman, S. Khanna, and W. chiew Tan. Why and where: A characterization of data provenance. In In ICDT, pages 316-330. Springer, 2001.
5. B. Cao, B. Plale, G. Subramanian, E. Robertson, and Y. L. Simmhan. Provenance information model of karma version 3. In SERVICES I, pages 348-351, 2009.
6. J. J. Carroll, C. Bizer, P. Hayes, and P. Stickler. Named graphs, provenance and trust. In Proceedings of the 14th international conference on World Wide Web, WWW '05, pages 613-622, New York, NY, USA, 2005. ACM.
7. Y. Cheah and B. Plale. Provenance analysis: Towards quality provenance. 8th IEEE International Conference on eScience, Oct. 2012.
8. J. Cheney, A. Ahmed, and U. A. Acar. Provenance as dependency analysis. In Proceedings of the 11th international conference on Database programming languages, DBPL'07, pages 138-152, Berlin, Heidelberg, 2007. Springer-Verlag.
9. J. Cheney, L. Chiticariu, and W.-C. Tan. Provenance in databases: Why, how, and where. Found. Trends databases, 1:379-474, April 2009.
10. B. Chun, D. Culler, T. Roscoe, A. Bavier, L. Peterson, M. Wawrzoniak, and M. Bowman. Planetlab: an overlay testbed for broad-coverage services. SIGCOMM Comput. Commun. Rev., 33(3):3-12, July 2003.
11. S. B. Davidson and S. C. Boulakia. Provenance in scientific workflow systems. IEEE Data Eng. Bull., 30(4):44-50, 2007.
12. S. B. Davidson and J. Freire. Provenance and scientific workflows: challenges and opportunities. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, SIGMOD '08, pages 1345-1350, New York, NY, USA, 2008. ACM.
13. J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S.-H. Bae, J. Qiu, and G. Fox. Twister: a runtime for iterative mapreduce. In Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, HPDC '10, pages 810-818, New York, NY, USA, 2010. ACM.
14. P. Groth, S. Miles, and L. Moreau. Preserv: Provenance recording for services. In UK e-Science All Hands Meeting 2005. EPSRC, 2005.
15. D. Gunter and B. Tierney. Netlogger: a toolkit for distributed system performance tuning and debugging. In Integrated Network Management, 2003. IFIP/IEEE Eighth International Symposium on, pages 97-100, march 2003.
16. GUSH. http://gush.cs.williams.edu/trac/gush.
17. W. Jiang, C. Hu, S. Pasupathy, A. Kanevsky, Z. Li, and Y. Zhou. Understanding customer problem troubleshooting from storage system logs. In Proccedings of the 7th conference on File and storage technologies, FAST '09, pages 43-56, Berkeley, CA, USA, 2009. USENIX Association.
18. Karma. http://d2i.indiana.edu/provenance-karma.
19. B. Ludscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, E. Lee, J. Tao, and Y. Zhao. Scientific workflow management and the kepler system, 2005.
20. D. Margo and R. Smogor. Using provenance to extract semantic file attributes. In Proceedings of the 2nd conference on Theory and practice of provenance, TAPP'10, pages 7-7, Berkeley, CA, USA, 2010. USENIX Association.
21. S. Miles, S. C. Wong, W. Fang, P. Groth, K. peter Zauner, and L. Moreau. Provenance-based validation of e-science experiments. In ISWC, pages 801-815. Springer-Verlag, 2005.
22. L. Moreau et al. Special issue: The first provenance challenge. Concurr. Comput. : Pract. Exper., 20(5):409-418, Apr. 2008. (Pubitemid 351428355)
23. W. M.P. van der Aalst et. al. Process mining manifesto. In 7th International Workshop on Business Process Intelligence (BPI 2011), pages 169-194, Campus des Cézeaux, Clermont-Ferrand, 2012. Springer-Verlag.
24. K.-K. Muniswamy-Reddy, D. A. Holland, U. Braun, and M. Seltzer. Provenance-aware storage systems. In Proceedings of the annual conference on USENIX '06 Annual Technical Conference, pages 4-4, Berkeley, CA, USA, 2006. USENIX Association.
25. K.-K. Muniswamy-Reddy and M. Seltzer. Provenance as first class cloud data. SIGOPS Oper. Syst. Rev., 43:11-16, January 2010.
26. Netkarma. http://d2i.indiana.edu/provenance-netkarma.
27. T. Oinn et al. Taverna: Lessons in creating a workflow environment for the life sciences: Research articles. Concurr. Comput. : Pract. Exper., 18(10):1067-1100, 2006. (Pubitemid 44258498)
28. A. Oliner and J. Stearley. What supercomputers say: A study of five system logs. In Proceedings of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN '07, pages 575-584, Washington, DC, USA, 2007. IEEE Computer Society.
29. Phoebus. http://damsl.cs.indiana.edu/projects/phoebus/xsp.php.
30. J. E. Prewett. Analyzing cluster log ïňAles using logsurfer. In Proc. Annual Conf. on Linux Cluster, 2003.
31. S. R. Sarukkai and A. D. Malony. Perturbation analysis of high level instrumentation for spmd programs. SIGPLAN Not., 28(7):44-53, July 1993.
32. Y. Simmhan and B. Plale. Using provenance for personalized quality ranking of scientific datasets. International Journal of Computers and Their Applications (IJCA), 18(3):180-195, 2011.
33. Y. L. Simmhan, B. Plale, and D. Gannon. A survey of data provenance in e-science. SIGMOD Rec., 34(3):31-36, Sept. 2005. (Pubitemid 43152529)
34. Y. L. Simmhan, B. Plale, and D. Gannon. A framework for collecting provenance in data-centric scientific workflows. In In ICWS, pages 427-436, 2006.
35. W. C. Tan. Provenance in databases: Past, current, and future. IEEE Data Eng. Bull., 30(4):3-12, 2007.
36. P. Townend, P. Groth, and J. Xu. A provenance-aware weighted fault tolerance scheme for service-based applications. In Proceedings of the Eighth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing, ISORC '05, pages 258-266, Washington, DC, USA, 2005. IEEE Computer Society.
37. W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan. Detecting large-scale system problems by mining console logs. In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, SOSP '09, pages 117-132, New York, NY, USA, 2009. ACM.
38. D. Yuan, H. Mai, W. Xiong, L. Tan, Y. Zhou, and S. Pasupathy. Sherlog: error diagnosis by connecting clues from run-time logs. SIGARCH Comput. Archit. News, 38(1):143-154, Mar. 2010.
39. W. Zhou, J. Zhan, D. Meng, D. Xu, and Z. Zhang. Logmaster: Mining event correlations in logs of large scale cluster systems. CoRR, abs/1003.0951, 2010.