A generic parallel processing model for facilitating data mining and integration

Parallel Computing

Volumn 37, Issue 3, 2011, Pages 157-171

  Han, Liangxiu ^a,c   Liew, Chee Sun ^a,b   Van Hemert, Jano ^a   Atkinson, Malcolm ^a  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b UNIVERSITY OF MALAYA   (Malaysia)

^c MANCHESTER METROPOLITAN UNIVERSITY   (United Kingdom)

Author keywords

Data mining and data integration (DMI); Life sciences; OGSA DAI; Parallelism; Pipeline streaming; Workflow

Indexed keywords

DATA INTEGRATION; LIFE SCIENCES; OGSA-DAI; PARALLELISM; WORKFLOW;

COMPUTER HARDWARE DESCRIPTION LANGUAGES; DATA FLOW ANALYSIS; DATA MINING; GRAPHIC METHODS; INTEGRATION; PIPELINES;

DATA HANDLING;

EID: 79952503742     PISSN: 01678191     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.parco.2011.02.006     Document Type: Article

References (43)

1. H.P. Graf, E. Cosatto, L. Bottou, I. Durdanovic, and V. Vapnik Parallel support vector machines: the cascade SVM L. Saul, Y. Weiss, L. Bottou, Advances in Neural Information Processing Systems vol. 17 2005 MIT Press Cambridge, MA 521 528
2. B. Catanzaro, N. Sundaram, and K. Keutzer Fast support vector machine training and classification on graphics processors ACM International Conference Proceeding Series: Proceedings of the 25th international conference on Machine learning vol. 307 2008 ACM New York, NY, USA 104 111
3. D. Caragea, A. Silvescu, and V. Honavar A framework for learning from distributed data using sufficient statistics and its application to learning decision trees International Journal of Hybrid Intelligent Systems 1 2 2004 80 89
4. R. Jin, G. Yang, and G. Agrawal Shared memory parallelization of data mining algorithms: techniques, programming interface, and performance IEEE Transactions on Knowledge and Data Engineering 17 1 2005 71 89 (Pubitemid 40536472)
5. J. Dean, S. Ghemawat, MapReduce: Simplified data processing on large clusters, in: Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI), 2004, pp. 137-150.
6. M. Stonebraker, D. Abadi, D.J. DeWitt, S. Madden, E. Paulson, A. Pavlo, and A. Rasin MapReduce and parallel DBMSs: friends or foes? Communications of the ACM 53 1 2010 64 71
7. M. Isard, M. Budiu, Y. Yu, A. Birrell, D. Fetterly, Dryad: Distributed data-parallel programs from sequential building blocks, in: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems, 2007, pp. 59-72. (Pubitemid 47281575)
8. E. Deelman, D. Gannon, M. Shields, and I. Taylor Workflows and e-science: an overview of workflow system features and capabilities Future Generation Computer Systems 25 2009 528 540
9. T. Oinn, M. Greenwood, M. Addis, N. Alpdemir, J. Ferris, K. Glover, C. Goble, A. Goderis, D. Hull, D. Marvin, P. Li, P. Lord, M. Pocock, M. Senger, R. Stevens, A. Wipat, and C. Wroe Taverna: lessons in creating a workflow environment for the life sciences Concurrency and Computation: Practice and Experience 18 10 2006 1067 1100 (Pubitemid 44258498)
10. I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludascher, S. Mock, Kepler: an extensible system for design and execution of scientific workflows, in: Proceedings. of 16th International Conference on Scientific and Statistical Database Management, 2004, pp. 423-424.
11. E. Deelman, G. Singh, M.-H. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G.B. Berriman, J. Good, A.C. Laity, J.C. Jacob, and D.S. Katz Pegasus: a framework for mapping complex scientific workflows onto distributed systems Scientific Programming 13 3 2005 219 237 (Pubitemid 43022937)
12. C. Rueda, and M. Gertz Real-time integration of geospatial raster and point data streams Proceedings of the 20th international conference on Scientific and Statistical Database Management vol. 5069 2008 Springer 605 611
13. H.P. Hofstee, Power efficient processor architecture and the cell processor, in: Proceedings of the 11th International Symposium on High-Performance Computer Architecture, Washington, DC, USA, IEEE Computer Society, 2005, pp. 258-262. (Pubitemid 41731505)
14. J. Gummaraju, J. Coburn, Y. Turner, and M. Rosenblum Streamware: programming general-purpose multicore processors using streams ACM SIGARCH Computer Architecture News 36 1 2008 297 307 (Pubitemid 351585415)
15. B. Dobrzelecki, A. Krause, A. Hume, A. Grant, M. Antonioletti, T. Alemu, M. Atkinson, M. Jackson, and E. Theocharopoulos Integrating distributed data sources with OGSA-DAI DQP and views Philosophical Transactions of the Royal Society of London, Series A 368 1926 2010 4133 4145
16. M. Flynn Some computer organizations and their effectiveness IEEE Transansactions on Computers C-21 1972 948
17. R. Duncan A survey of parallel computer architectures IEEE Computer 23 2 1990 5 16
18. R. Pike, S. Dorward, R. Griesemer, and S. Quinlan Interpreting the data: parallel analysis with Sawzal Scientific Programming 13 4 2005 277 298 (Pubitemid 43063583)
19. R. Jin, G. Agrawal., A middleware for developing parallel data mining implementations, in: Proceedings of the First SIAM Conference on Data Mining, April, 2001.
20. H. Kargupta, I. Hamzaoglu, B. Stafford, PADMA: Parallel data mining agents for scalable text classification, in: The Proceedings of High Performance Computing '97, 1997.
21. P. Brezany, I. Janciak, J. Brezanyova, A.M. Tjoa, GridMiner: an advanced grid-based support for brain informatics data mining tasks, in: Proceedings of the 1st WICI International Conference on Web Intelligence Meets Brain Informatics, 2006, pp. 353-366.
22. R.H. Arpaci-Dusseau, E. Anderson, N. Treuhaft, D.E. Culler, J.M. Hellerstein, D. Patterson, and K. Yelick Cluster i/o with river: making the fast case common Proceedings of the Sixth Workshop on I/O in Parallel and Distributed Systems 1999 ACM New York, NY, USA 10 22
23. Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. von Laszewski, V. Nefedova, I. Raicu, T. Stef-Praun, M. Wilde, Swift: Fast, reliable, loosely coupled parallel computation, in: IEEE Congress on Services (Services 2007), 2007, pp. 199-206.
24. P. Couvares, T. Kosar, A. Roy, J. Weber, and K. Wenger Workflow management in condor I.J. Taylor, E. Deelman, D.B. Gannon, M. Shields, Workflows for e-Science 2007 Springer London 357 375
25. I. Taylor, M. Shields, I. Wang, and A. Harrison The triana workflow environment: architecture and applications I. Taylor, E. Deelman, D. Gannon, M. Shields, Workflows for e-Science 2007 Springer London 320 339
26. G. Laszewski, M.Hategan, Java CoG Kit Karajan-gridant workflow guide, Tech. rep., Argonne National Laboratory, Argonne, IL, USA, 2005.
27. G. Laszewski, and M. Hategan Workflow concepts of the Java cog kit Grid Computing 3 3-4 2005 239 258 (Pubitemid 43280304)
28. T. Andrews, F. Cubera, H. Dholakia, et al., Business Process Execution Language for Web Services Version 1.1. OASIS, 2003. Available from: .
29. Active BPEL Engine. Available from: .
30. S. Amarasinghe, M.I. Gordon, M. Karczmarek, J. Lin, D. Maze, R.M. Rabbah, and W. Thies Language and compiler design for streaming applications International Journal of Parallel Programming 33 2 2005 261 278 (Pubitemid 41202445)
31. D. Carney, U. etintemel, M. Cherniack, C. Convey, S. Lee, G. Seidman, M. Stonebraker, N. Tatbul, S. Zdonik, Monitoring streams - a new class of data management applications, in: Proceedings of the 28th international conference on Very Large Data Bases, 2002, pp. 215-226.
32. S. Chandrasekaran, O. Cooper, A. Deshpande, M.J. Franklin, J.M. Hellerstein, W. Hong, S. Krishnamurthy, S. Madden, V. Raman, F. Reiss, M.A. Shah, TelegraphCQ: Continuous dataflow processing for an uncertan world, in: First Biennial Conference on Innovative Data Systems Research (CIDR 2003), Asilomar, CA, USA, 2003.
33. B. Plale, and K. Schwan Dynamic querying of streaming data with the dQUOB system IEEE Transanctions on Parallel and Distributed System 14 3 2003 422 432
34. M.D. Beynon, T. Kurc, U. Catalyurek, C. Chang, A. Sussman, and J. Saltz Distributed processing of very large datasets with DataCutter Parallel Computing 27 2001 1457 1478 (Pubitemid 32723674)
35. L. Chen, K. Reddy, G. Agrawal, GATES: A grid-based middleware for processing distributed data streams, in: Proceedings of the International Symposium on High Performance Distributed Computing (HPDC), Washington, DC, USA, IEEE Computer Society, 2004, pp.192-201.
36. N. Jain, L. Amini, H. Andrade, R. King, Y. Park, P. Selo, C. Venkatramani, Design, implementation, and evaluation of the linear road benchmark on the stream processing core, in: Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data, 2006, pp. 431-442. (Pubitemid 46946529)
37. O. Sinnen, L. Sousa, A platform independent parallelising tool based on graph theoretic models, in: Vector and Parallel Processing - VECPAR 2000, vol. 1981, 2001, pp. 154-167. (Pubitemid 33269431)
38. M.P. Atkinson, J.I. van Hemert, L. Han, A. Hume, C.S. Liew, A Distributed Architecture for Data Mining and Integration, in: Proceedings of the second international workshop on Data-aware distributed computing, DADC '09, New York, NY, USA, ACM, 2009, pp. 11-20.
39. OGSA-DAI, June, 2010. Available from: .
40. EURExpress-II, June, 2010. Available from: .
41. L. Han, J.I. van Hemert, R. Baldock, M.P. Atkinson, Automating gene expression annotation for mouse embryo, in: R. Huang, Q. Jian, P. Jian, G.X. Meng, X. Li (Eds.), Lecture Notes in Computer Science (Advanced Data Mining and Applications, ADMA 2009), vol. LANI 5678, 2009, pp. 469-478.
42. OGSADAI, OGSA-DAI 4.0 Documentation. Available from: .
43. Linkpack, July, 2010. Available from: .