An initial study of predictive machine learning analytics on large volumes of historical data for power system applications

Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

Volumn , Issue , 2014, Pages 952-959

  Zheng, Jiang ^a   Dagnino, Aldo ^a  

^a ABB INC   (United States)

Author keywords

Apache Spark; Big Data; Hadoop; Machine learning

Indexed keywords

ARTIFICIAL INTELLIGENCE; DATA HANDLING; ELECTRIC LOAD FORECASTING; ELECTRIC POWER SYSTEMS; LEARNING ALGORITHMS; LEARNING SYSTEMS;

ANALYTIC MODELS; FAULT EVENT; HADOOP; HISTORICAL DATA; INDUSTRIAL DATUM; INTELLIGENT MACHINE; LARGE VOLUMES; POWER SYSTEM APPLICATIONS;

BIG DATA;

EID: 84988273389     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BigData.2014.7004327     Document Type: Conference Paper

References (51)

1. H. K. Alfares and M. Nazeeruddin, "Electric load forecasting: literature survey and classification of methods," International Journal of Systems Science, Vol. 33, No. 1, pp. 23-34, 2002.
2. E. Almeshaiei and H. Soltan, "A methodology for Electric Power Load Forecasting," Alexandria Engineering Journal, Vol. 50, No. 2, pp. 137-144, June 2011.
3. Alpine Data Labs, http://alpinenow.com/plans/, retrieved on July 2014.
4. AMPLab, "SQL Big Data Benchmark" https://amplab.cs.berkeley.edu/benchmark/, retrieved on July 2014.
5. Apache Hadoop, http://hadoop.apache.org/, retrieved on July 2014.
6. Apache Hama, http://hama.apache.org/, retrieved on July 2014.
7. Apache Mahout, http://mahout.apache.org/, retrieved on July 2014.
8. Apache Mahout Team, "Powered by Mahout," https://mahout.apache.org/general/powered-by-mahout.html, retrieved on July 2014.
9. Apache Spark, http://spark.apache.org/, retrieved on July 2014.
10. Apache Spark Team, "MLlib Programming Guide," http://spark.incubator.apache.org/docs/latest/mllib-guide.html, retrieved on July 2014.
11. Ayasdi, http://www.ayasdi.com/, retrieved on July 2014.
12. A. R. Benson, D. F. Gleich, P. G. Constantine, and J. Demmel, "Direct QR factorizations for tall-And-skinny matrices in MapReduce architectures," Proceedings of the 2nd IEEE International Conference on Big Data, 2013.
13. A. R. Benson, D. F. Gleich, P. G. Constantine, and J. Demmel https://github.com/arbenson/mrtsqr, retrieved on July 2014.
14. Cloudera Oryx, https://github.com/cloudera/oryx, retrieved on July 2014.
15. Context Relevant, http://www.contextrelevant.com/, retrieved on July 2014.
16. A. Dagnino and D. Cox, "Industrial Analytics to Discover Knowledge from Instrumented Networked Machines", Proceedings of the 26th International Conference on Software Engineering and Knowledge Engineering (SEKE'14), Vancouver, Canada, July 2014.
17. A. Dagnino, K. Smiley, and L. Ramachandran, "Forecasting Fault Events in Power Distribution Grids Using Machine Learning," Proceedings of the 24th International Conference on Software Engineering and Knowledge Engineering (SEKE'12), San Francisco, CA, USA, pp. 458-463, July 2012.
18. Datameer, http://www.datameer.com/, retrieved on July 2014.
19. J. Dean, S. Ghemawat, "MapReduce: simplified Data processing on large clusters," Proceedings of the 6th conference on Symposium on Opearting Systems Design &Implementation, pp.10, December 2004.
20. J. W. Eaton et al. "GNU Octave", http://www.gnu.org/software/octave/ retrieved on July 2014.
21. I. Feinerer, S. Theussl, "hive: Hadoop InteractiVE," http://cran.rproject. org/web/packages/hive/index.html, retrieved on July 2014.
22. M. Hall, "Mining Big Data using Weka 3" http://www.cs.waikato.ac.nz/ml/weka/bigdata.html, retrieved on July 2014.
23. M. Hall, "Weka 3: Data Mining Software in Java," http://www.cs.waikato.ac.nz/ml/weka/, retrieved on July 2014.
24. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, "The WEKA Data mining software: an update," ACM SIGKDD Explorations Newsletter, Vol. 11, No. 1, June 2009.
25. R. Ihaka and R. Gentleman, "R: A language for Data analysis and graphics," Journal of Computational and Graphical Statistics, Vol. 5, No. 3, pp. 299-314, 1996.
26. T. Kraska, A. Talwalkar, J.Duchi, R. Griffith, M. Franklin, and M. I. Jordan, "MLbase: A Distributed Machine Learning System," Proceedings of the Conference on Innovative Data Systems Research, 2013.
27. A. Lim, L. Breiman, and A. Cutler, "bigrf: Big Random Forests: Classification and Regression Forests for Large Data Sets," http://cran.rproject. org/web/packages/bigrf/index.html, retrieved on July 2014.
28. Y. Low, D. Bickson, J. Gonzalez, C. Guestrin, A. Kyrola, and J. M. Hellerstein. "Distributed GraphLab: A framework for machine learning and Data mining in the cloud," Proceedings of the VLDB Endowment 5 No. 8, pp. 716-727, 2012.
29. Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. M. Hellerstein. "Graphlab: A new framework for parallel machine learning," arXiv preprint arXiv:1006.4990, 2010.
30. T. Lumley, "biglm: bounded memory linear and generalized linear models," http://cran.r-project.org/web/packages/biglm/index.html retrieved on July 2014.
31. J. Mannix, "Investigate possibility of integration with Neuroph neural-net library," https://issues.apache.org/jira/browse/MAHOUT-383, retrieved on July 2014.
32. X. Meng, M. W. Mahoney, "Robust Regression on MapReduce" Proceedings of the International Conference on Machine Learning (ICML'13), 2013.
33. Oracle, "Oracle Big Data Connectors User's Guide" http://docs.oracle.com/cd/E49465-01/doc.23/e49333/orch.htm, retrieved on July 2014.
34. Oracle, "Reference Documentation for ORAAH Version 2.3.1," http://download.oracle.com/otn/other/bigdata/ORAAH-2.3.1-Reference.pdf, pp. 85, retrieved on July 2014.
35. S. Owen, R. Anil, T. Dunning, and E. Friedman, "Mahout in Action," Manning Publications Co., Greenwich, CT, 2011.
36. Z. Prekopcsk, G. Makrai, T. Henk, and C. Gspr-Papanek, "Radoop: Analyzing Big Data with rapidminer and hadoop," Proceedings of the 2nd RapidMiner Community Meeting and Conference (RCOMM'11), 2011.
37. R Development Core Team, "R: A Language and Environment for Statistical Computing," Vienna: R Foundation for Statistical Computing 2004.
38. R Development Core Team, "R for Machine Learning," http://cran.rproject. org/web/views/MachineLearning.html, retrieved on July 2014.
39. Radoop, http://www.radoop.eu/, retrieved on July 2014.
40. RapidMiner, http://rapidminer.com/, retrieved on July 2014.
41. Revolution Analytics, "RHadoop Design Philosophy" https://github.com/RevolutionAnalytics/RHadoop/wiki/dev-rmr-Design-Philosophy, retrieved on July 2014.
42. Revolution R Enterprise, http://www.revolutionanalytics.com/products retrieved on July 2014.
43. S. Seo, E.J. Yoon, J. Kim, S. Jin, J. Kim, and S. Maeng, "HAMA: An Efficient Matrix Computation with the MapReduce Framework," Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science, pp. 721-726, November 2010.
44. Z. Sheikh, "Proposal to implement Neural Network with backpropagation learning on Hadoop", https://issues.apache.org/jira/browse/MAHOUT-364, retrieved on July 2014.
45. K. Shvachko, H. Kuang, S. Radia, R. and Chansler, "The Hadoop Distributed File System," Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST'10), pp. 1-10, May 2010.
46. E. Sparks, A. Talwalkar, V. Smith, J. Kottalam, X. Pan, J. Gonzalez, J. Gonzalez, M. Franklin, M. I. Jordan, and T. Kraska, "MLI: An API for Distributed Machine Learning," Proceedings of the International Conference on Data Mining
47. A. Talwalkar, T. Kraska, R. Griffith, J. Duchi, J. Gonzalez, D. Britz, X. Pan, V. Smith, E. Sparks, A. Wibisono, M. J. Franklin, and M. I. Jordan "MLbase: A Distributed Machine Learning Wrapper." Proceedings of the Big Learning Workshop at NIPS, 2012.
48. wise.io, http://about.wise.io/, retrieved on July 2014.
49. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica "Spark: cluster computing with working sets," Proceedings of the 2nd USENIX conference on Hot topics in cloud computing, pp. 10-10 Boston, MA, June 2010.
50. K. Zhu, H. Wang, H. Bai, J. Li, Z. Qiu, H. Cui, and E. Y. Chang "Parallelizing Support Vector Machines on Distributed Computers," Advances in Neural Information Processing Systems 20 (NIPS 2007).
51. K. Zhu, H. Wang, H. Bai, J. Li, Z. Qiu, H. Cui, and E. Y. Chang, "psvm: A parallel Version of support Vector machine," http://code.google.com/p/psvm/, retrieved on July 2014