Parallel and Distributed Data Mining for Astronomy Applications

Advances in Machine Learning and Data Mining for Astronomy

Volumn , Issue , 2012, Pages 595-615

  Das, Kamalika ^a   Bhaduri, Kanishka ^a  

^a NASA AMES RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85081817574     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1201/b11822-21     Document Type: Chapter

References (85)

1. N. Amado, J. Gama, and F. Silva. Parallel implementation of decision tree learning algorithms. In Proceedings of the 10th Portuguese Conference on Artificial Intelligence on Progress in Artificial Intelligence, Knowledge Extraction, Multi-Agent Systems, Logic Programming and Constraint Solving, pp. 6-13, Porto, Portugal, 2001.
2. S. Bailey, R. Grossman, H. Sivakumar, and A. Turinsky. Papyrus: A system for data mining over local and wide area clusters and super-clusters. In Proceedings of CDROM’99, p. 63, New York, USA, 1999.
3. N. M. Ball and R. J. Brunner. Data mining and machine learning in astronomy. International Journal of Modern Physics D, 19(7):1049-1106, 2010.
4. S. Bandyopadhyay, C. Giannella, U. Maulik, H. Kargupta, K. Liu, and S. Datta. Clustering distributed data streams in peer-to-peer environments. Information Science, 176(14):1952-1985, 2006.
5. Y. Ben-Haim and E. Tom-Tov. A streaming parallel decision tree algorithm. Journal ofMachine Learning Research, 11:849-872, March 2010.
6. K. Bhaduri, K. Das, and C. Giannella. Distributed monitoring of the R2 statistic for linear regression. In SIAM International Conference on Data Mining, pp. 438-449, Mesa, AZ, 2011.
7. K. Bhaduri and H. Kargupta. A scalable local algorithm for distributed multivariate regression. Statistical Analysis and Data Mining J., 1(3):177-194, 2008.
8. K. Bhaduri and A. N. Srivastava. A local scalable distributed expectation maximization algorithm for large peer-to-peer networks. In Proceedings of ICDM’09, pp. 31-40, Miami, Florida, USA, 2009.
9. K. Bhaduri, R. Wolff, C. Giannella, and H. Kargupta. Distributed decision tree induction in P2P systems. Statistical Analysis and Data Mining, 1(2):85-103, 2008.
10. K. Borne. Distributed data mining in the national virtual observatory. SPIE Data Mining and Knowledge Discovery: Theory, Tools, and Technology V, 5098:211-218, 2003.
11. K. Borne. Scientific data mining in astronomy. In H. Kargupta, J. Han, P. S. Yu, R. Motwani, and V. Kumar, editors, Next Generation of Data Mining, Chapter 5, pp. 91-114. CRC Press, Boca Raton, FL, 2009.
12. K. W. Bowyer, L. O. Hall, T. Moore, N. Chawla, and W. P. Kegelmeyer. A parallel decision tree builder for mining very large visualization datasets. In Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, 2000, Vol. 3, pp. 1888-1893, 2000.
13. S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah. Gossip algorithms: Design, analysis and applications. In Proceedings of Infocom’05, pp. 1653-1664, Miami, FL, March 2005.
14. J. Branch, B. Szymanski, C. Giannella, R. Wolff, and H. Kargupta. In-network outlier detection in wireless sensor networks. In Proceedings of ICDCS’06, p. 51, Lisbon, Portugal, July 2006.
15. L. Breiman, J. Friedman, R. Olshen, and C. Stone. Classification and Regression Trees. Wadsworth and Brooks, Monterey, CA, 1984.
16. L. Breiman. Bagging predictors. Machine Learning, 24(2):123-140, 1996.
17. L. Breiman. Random forests. Machine Learning, 45(1):5-32, 2001.
18. R. Brunner, G. Djorgovski, T. Prince, and A. Szalay. Handbook of massive data sets. In J. Abello, P. M. Pardalos, and M. G. C. Resende, editors, Massive Datasets in Astronomy, pp. 931-979. Kluwer Academic Publishers, Norwell, MA, USA, 2002.
19. M. Cannataro and D. Talia. The knowledge grid. Communication of ACM, 46(1):89-93, 2003.
20. D. Caragea, A. Silvescu, and V. Honavar. A framework for learning from distributed data using sufficient statistics and its application to learning decision trees. International Journal of Hybrid Intelligent Systems, 1(1-2):80-89, 2004.
21. P. K. Chan and S. Stolfo. Experiments on multistrategy learning by meta-learning. In Proceedings of the Second International Conference on Information and Knowledge Management, Washington, DC, USA, pp. 314-323, 1993.
22. P. K. Chan and S. J. Stolfo. Toward parallel and distributed learning by meta-learning. In Working Notes of AAAI Workshop on Knowledge Discovery in Databases, Washington, DC, pp. 227-240, 1993.
23. E. Y. Chang, K. Zhu, H. Wang, H. Bai, J. Li, Z. Qiu, and H. Cui. PSVM: Parallelizing support vector machines on distributed computers. In Proceedings of Neural Information Processing Systems, 2007.
24. R. Chen, K. Sivakumar, and H. Kargupta. Collective mining of bayesian networks from distributed heterogeneous data. Knowledge and Information Systems, 6(2):164-187, 2004.
25. W. Chen, Y. Song, H. Bai, C. Lin, and E. Chang. Parallel spectral clustering in distributed systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 99(RapidPosts), pp. 568-586, 2010.
26. S. Chib and E. Greenberg. Understanding the Metropolis-Hastings algorithm. The American Statistician, 49(4):327-335, 1995.
27. C. Chu, S. Kim, Y. Lin, Y. Yu, G. Bradski, A. Ng, and K. Olukotun. Map-Reduce for machine learning on multicore. In Proceedings of NIPS’06, pp. 281-288, Vancouver, BC, Canada, 2006.
28. A. Ng, M. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In Proceedings of Advances in Neural Information Processing Systems, 849-856, Vancouver, BC, Canada, 2001.
29. T. Mahule, K. D. Borne, S. Dey, S. Arora, and H. Kargupta. PADMINI: A peer-to-peer distributed astronomy data mining system and a case study. CIDU, 243-257, 2010.
30. K. Das, K. Bhaduri, K. Liu, and H. Kargupta. Distributed identification of top-l inner product elements and its application in a peer-to-peer network. IEEE Transactions on Knowledge and Data Engineering (TKDE), 20(4):475-488, 2008.
31. The DataGrid Project. http://eu-datagrid.web.cern.ch/eu-datagrid/default.htm.
32. S. Datta, K. Bhaduri, C. Giannella, R. Wolff, and H. Kargupta. Distributed data mining in peer-to-peer networks. IEEE Internet Computing Special Issue on Distributed Data Mining, 10(4):18-26, 2006.
33. S. Datta, C. Giannella, and H. Kargupta. K-Means clustering over large, dynamic networks. In Proceedings of SDM’06, pp. 153-164, Maryland, 2006.
34. W. Day and H. Edelsbrunner. Efficient algorithms for agglomerative hierarchical clustering methods. Journal of Classification, 1:7-24, 1984.
35. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. Commun. ACM, 51:107-113, 2008.
36. I. Dhillon and D. Modha. Adata-clustering algorithm on distributed memorymultiprocessors. In Revised Papers from Large-Scale ParallelDataMining, Workshop on Large-Scale Parallel KDD Systems, SIGKDD, pp. 245-260, Boston, MA, 2000.
37. H. Dutta, C. Giannella, K. D. Borne, and H. Kargupta. Distributed top-K outlier detection from astronomy catalogs using the DEMAC system. In SDM, 2007.
38. M. Eisenhardt, W. Müller, and A. Henrich. Classifying documents by distributed P2P clustering. In GI Jahrestagung, pp. 286-291, 2003.
39. I. Foster and C. Kesselman. The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, San Francisco, CA, USA, 2004.
40. A. Fred and A. Jain. Data clustering using evidence accumulation. In Proceedings of ICPR’02, p. 40276, Washington, DC, USA, 2002.
41. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: A statistical view of boosting. The Annals of Statistics, 38(2):337-374, 2000.
42. The Galaxy Evolution Explorer. http://www.galex.caltech.edu/about/overview.html.
43. S. Ghosh. Distributed Systems: An Algorithmic Approach. CRC Press, Boca Raton, FL, 2006.
44. C. Giannella, H. Dutta, K. Borne, R. Wolff, and H. Kargupta. Distributed data mining for astronomy catalogs. In Proceedings of Scientific Data Mining’06 Workshops, Bethesda, MD, USA, 2006.
45. C. Giannella, K. Liu, T. Olsen, and H. Kargupta. Communication efficient construction of deicision trees over heterogeneously distributed data. In Proceedings of ICDM’04, pp. 67-74, Brighton, UK, 2004.
46. G. Golub and C. Van Loan. Matrix Computations, 3rd ed. Johns Hopkins University Press, Baltimore, MD, USA, 1996.
47. What is grid? http://www.eu-degree.eu/DEGREE/General%\20questions/copy_of_what-isgrid.
48. GRIST: Grid data mining for astronomy. http://grist.caltech.edu/index.html.
49. W. Gropp, E. Lusk, and A. Skjellum. Using MPI: Portable Parallel Programmingwith theMessage Passing Interface. MIT Press, Cambridge, MA, 1996.
50. D. E. Hershberger and H. Kargupta. Distributed multivariate regression using wavelet-based collective data mining. Journal of Parallel and Distributed Computing, 61(3):372-400, 2001.
51. T. Hinke and J. Novotny. Data mining on NASA’s information power grid. In Proceedings of HPDC’00, p. 292, Pittsburgh, PA, USA, 2000.
52. W. Hoschek, F. J. Jaén-Martínez, A. Samar, H. Stockinger, and K. Stockinger. Data management in an international data grid project. In Proceedings of the First International Workshop on Grid Computing, pp. 77-90, London, UK, 2000.
53. JADE: Java Agent DEvelopment Framework. http://jade.tilab.com/.
54. M. Jelasity, A. Montresor, and O. Babaoglu. Gossip-based aggregation in large dynamic networks. ACM Transactions on Computer Systems (TOCS), 23(3):219-252, 2005.
55. R. Jin and G. Agrawal. Communication and memory efficient parallel decision tree construction. In SDM, pp. 119-129, San Francisco, CA, 2003.
56. E. L. Johnson and H. Kargupta. Collective, hierarchical clustering from distributed, heterogeneous data. In Revised Papers from Large-Scale Parallel Data Mining, Workshop on Large-Scale Parallel KDD Systems, SIGKDD, pp. 221-244, London, UK, 2000.
57. JXTA: Open source peer-to-peer protocol. https://jxta.dev.java.net/.
58. H. Kargupta, B. Park, D. Hershbereger, and E. Johnson. Collecttive data mining: A new perspective towards distributed data mining. In H. Kargupta and P. Chan, editors, Advances in Distributed and Parallel Knowledge Discovery. AAAI/MIT Press, pp. 133-184, Cambridge, MA, USA, 1999.
59. H. Kargupta and K. Sivakumar. Existential Pleasures of Distributed DataMining. DataMining:Next Generation Challenges and Future Directions. AAAI/MIT Press, Cambridge, MA, USA, 2004.
60. H. Kargupta and P. Chan, editors. Advances in Distributed and Parallel Knowledge Discovery. MIT Press, Cambridge, MA, USA, 2000.
61. H. Kargupta, W. Huang, K. Sivakumar, and E. L. Johnson. Distributed clustering using collective principal component analysis. Knowledge and Information Systems, 3(4):422-448, 2001.
62. H. Kargupta, Byung-Hoon Park, and H. Dutta. Orthogonal decision trees. IEEE Transactions on Knowledge and Data Engineering, 18(8):1028-1042, 2006.
63. D. Kempe and F. McSherry. A decentralized algorithm for spectral analysis. In Proceedings of STOC’04, pp. 561-568, Chicago, IL, USA, 2004.
64. D. Kempe, A. Dobra, and J. Gehrke. Computing aggregate information using gossip. In Proceedings of FOCS’03, Cambridge, 2003.
65. M. Klusch, S. Lodi, and G. L. Moro. Distributed clustering based on sampling local density estimates. In Proceedings of IJCAI’03, pp. 485-490, Mexico, August 2003.
66. W. Kowalczyk, M. Jelasity, and A. E. Eiben. Towards data mining in large and fully distributed peer-to-peer overlay networks. In Proceedings of BNAIC’03, pp. 203-210, University of Nijmegen, Nijmegen, Netherlands, 2003.
67. D. Krivitski, A. Schuster, and R. Wolff. A local facility location algorithm for large-scale distributed systems. Journal of Grid Computing, 5(4):361-378, 2007.
68. A. Lazarevic, D. Pokrajac, and Z. Obradovic. Distributed clustering and local regression for knowledge discovery in multiple spatial databases. In Proceedings of ESANN’00, pp. 129-134, Bruges, Belgium, April 2000.
69. K. Liu, H. Kargupta, and J. Ryan. Random projection-based multiplicative data perturbation for privacy preserving distributed data mining. IEEE Transactions on Knowledge and Data Engineering, 18:92-106, 2006.
70. LSST: Large Synoptic Survey Telescope. http://www.lsst.org/lsst.
71. P. Luo, H. Xiong, K. Lü, and Z. Shi. Distributed classification in peer-to-peer networks. In Proceedings of SIGKDD’07, pp. 968-976, San Jose, CA, USA, 2007.
72. S. M. McConnell and D. Skillicorn. Distributed data mining for astrophysical datasets. In Proceedings of Astronomical Data Analysis Software and Systems XIV, p. 360, 2005.
73. M. Mehyar, D. Spanos, J. Pongsajapan, S. H. Low, and R. Murray. Distributed averaging on peer-to-peer networks. In Proceedings of CDC’05, Spain, pp. 7446-7451, 2005.
74. C. F. Olson. Parallel algorithms for hierarchical clustering. Parallel Computing, 21:1313-1325, 1995.
75. B. Panda, J. Herbach, S. Basu, and R. Bayardo. Planet: Massively parallel learning of tree ensembles with mapreduce. Proceedings of the VLDB Endowment, 2:1426-1437, 2009.
76. B. Park, H. Kargupta, E. Johnson, E. Sanseverino, D. Hershberger, and L. Silvestre. Distributed, collaborative data analysis from heterogeneous sites using a scalable evolutionary technique. Applied Intelligence, 16(1):19-42, 2002.
77. The Röntgen Satellite. http://heasarc.gsfc.nasa.gov/docs/rosat/rosgof.html.
78. J. Shafer, R. Agrawal, and M. Mehta. SPRINT: A scalable parallel classifier for data mining. In Proceedings of the 22th International Conference on Very Large Data Bases, pp. 544-555, Bombay, India, 1996.
79. J. Shi and J. Malik. Normalized cuts and image segmentation. In Proceedings of CVPR’97, p. 731, San Juan, Puerto Rico, 1997.
80. S. J. Stolfo, A. L. Prodromidis, S. Tselepis, W. Lee, D.W. Fan, and P. K. Chan. JAM: Java agents for meta-learning over distributed databases. In Proceedings of KDD’97, pp. 74-81, Newport Beach, CA, 1997.
81. A. Strehl and J. Ghosh. Cluster ensembles-A knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research, 3:583-617, 2003.
82. R. Wolff, K. Bhaduri, and H. Kargupta. A generic local algorithm for mining data streams in large distributed systems. TKDE, 21(4):465-478, 2009.
83. R. Wolff and A. Schuster. Association rule mining in peer-to-peer systems. IEEE Transactions on Systems, Man and Cybernetics-Part B, 34(6):2426-2438, 2004.
84. M. J. Zaki. Parallel and distributed association mining: a survey. IEEEConcurrency, 7(4):14-25, 1999.
85. M. J. Zaki. Parallel and distributed data mining: An introduction. In Revised Papers from Large-Scale Parallel Data Mining, Workshop on Large-Scale Parallel KDD Systems, SIGKDD, pp. 1-23, London, UK, 2000.