Big data issues in smart grid – A review

Renewable and Sustainable Energy Reviews

Volumn 79, Issue , 2017, Pages 1099-1107

  Tu, Chunming ^a   He, Xi ^a   Shuai, Zhikang ^a   Jiang, Fei ^a  

^a HUNAN UNIVERSITY   (China)

Author keywords

Big data; Big data analytical applications; Cloud platform; Data mining; Smart grid

Indexed keywords

DATA MINING; ELECTRIC POWER TRANSMISSION NETWORKS; ENERGY UTILIZATION; SMART POWER GRIDS; SYSTEM STABILITY;

'CURRENT; ANALYTICAL APPLICATIONS; BIG DATA ANALYTICAL APPLICATION; CLOUD PLATFORMS; DATA TECHNOLOGIES; DISTRIBUTED ENERGIES; MONITOR SYSTEM; POWER GRIDS; SMART GRID; SYSTEM'S STABILITIES;

BIG DATA;

EID: 85019770020     PISSN: 13640321     EISSN: 18790690     Source Type: Journal    
DOI: 10.1016/j.rser.2017.05.134     Document Type: Review

References (131)

1. Chen HH, Chen S, Lan Y. Attaining a sustainable competitive advantage in the smart grid industry of China using suitable open innovation intermediaries. Renew Sustain Energy Rev 2016;62:1083–91.
2. Botte B, Cannatelli V, Rogai S. The telegestore project in Enel’s metering system[C]. In: Proceedings of the international conference and exhibition on electricity distribution. IET; 2005. p. 1–4.
3. National energy technology laboratory for the U.S. Department of energy, modern grid benefits, white paper; 2007.
4. Inovgrid. [Online]; 2010. Available: 〈http://www.inovcity.pt/en/pages/inovgrid. aspx〉.
5. Moma. [Online]; 2012. Available: 〈http://www.modellstadt-mannheim.de〉.
6. Farhangi H. The path of the smart grid. Power Energy Mag IEEE 2010;8(1):18–28.
7. DOE. Advancement of synchrophasor technology in ARRA Projects; 2016. 〈https://www.smartgrid.gov/recovery_act/program_publications.html〉.
8. Yang B, Yamazaki J. Big data analytic empowered grid applications- Is PMU a big data issue? In: Proceedings of the 2015 12th international conference on the European energy market (EEM). IEEE; 2015. p.1–4.
9. DOE. Summary of the North American synchrophasor initiative (NASPI) activity area; 2015. 〈https://www.naspi.org/documents〉.
10. Yuan J, Shen J, Pan L, et al. Smart grids in China. Renew Sustain Energy Rev 2014;37(3):896–906.
11. Alahakoon D, Yu X. Smart electricity meter data intelligence for future energy systems: a survey. IEEE Trans Ind Inf 2016;12(1):425–36.
12. Klump R, Agarwal P, Tate JE, Khurana H, Lossless compression of synchronized phasor measurements. In Proceedings of the IEEE power and energy society general meeting. Minneapolis, MN, USA; Jul. 2010. p. 1–7.
13. Electric Power Group. Wind farm oscillation detection and mitigation. 〈https:// www.smartgrid.gov/recovery_act/project_information.html〉.
14. Short TA. Advanced metering for phase identification, transformer identification, and secondary modeling. IEEE Trans Smart Grid 2013;4(2):651–8.
15. Overholt P, Kosterev D, Eto J, et al. Improving reliability through better models: using synchrophasor data to validate power plant models. IEEE Power Energy Mag 2014;12(3):44–51.
16. Luan W, Peng J, Maras M, et al. Smart meter data analytics for distribution network connectivity verification. IEEE Trans Smart Grid 2015;6(4), [1–1].
17. Berrisford AJ. A tale of two transformers: an algorithm for estimating distribution secondary electric parameters using smart meter data. Electrical and computer engineering (CCECE). In: Proceedings of the 2013 26th annual IEEE Canadian conference. IEEE; 2013. p. 1–6.
18. Hajnoroozi AA, Aminifar F, Ayoubzadeh H. Generating unit model validation and calibration through synchrophasor measurements. IEEE Trans Smart Grid 2015;6(1):441–9.
19. Bolognani S, Bof N, Michelotti D. et al. Identification of power distribution network topology via voltage correlation analysis. In: Proceedings of the IEEE conference on decision and control; 2013. p. 1659–64.
20. Erlinghagen S, Lichtensteiger B, Markard J. Smart meter communication standards in Europe – a comparison. Renew Sustain Energy Rev 2015;43:1249–62.
21. Zhou K, Yang S. Understanding household energy consumption behavior: the contribution of energy big data analytics. Renew Sustain Energy Rev 2016;56:810–9.
22. Tuballa ML, Abundo ML. A review of the development of Smart Grid technologies. Renew Sustain Energy Rev 2016;59:710–25.
23. Haider HT, See OH, Elmenreich W. A review of residential demand response of smart grid. Renew Sustain Energy Rev 2016;59:166–78.
24. Moretti M, Djomo SN, Azadi H, et al. A systematic review of environmental and economic impacts of smart grids. Renew Sustain Energy Rev 2017:68.
25. Ali SM, Jawad M, Khan B, et al. Wide area smart grid architectural model and control: a survey. Renew Sustain Energy Rev 2016;64:311–28.
26. Manyika J, Chui M, Brown B, et al. Big data: the next frontier for innovation, competition, and productivity. Analytics 2011.
27. Naimi AI, Westreich DJ. Big data: a revolution that will transform how we live, work, and think. Am J Epidemiol 2014;17(9):181–3.
28. Zhou K, Fu C, Yang S. Big data driven smart energy management: from big data to big insights. Renew Sustain Energy Rev 2016;56:215–25.
29. Schuelke-Leech BA, Barry B, Muratori M, et al. Big data issues and opportunities for electric utilities. Renew Sustain Energy Rev 2015;52(2):937–47.
30. Hu J, Vasilakos A. Energy big data analytics and security: challenges and opportunities. IEEE Trans Smart Grid 2016;7(5):2423–36, [2016, 7(5):1-1].
31. Mishra DP, Samantaray SR, Joos G. A combined wavelet and data-mining based intelligent protection scheme for microgrid. IEEE Trans Smart Grid 2016;7(5):2295–304.
32. Chen YC, Wang J, Dominguez-Garcia AD, et al. Measurement-based estimation of the power flow Jacobian matrix. IEEE Trans Smart Grid 2016;7(5):2507–15.
33. Endsley MR, Connors ES. Situation awareness: State of the art[C]. In: Proceedings of the IEEE power & energy society general meeting-conversion & delivery of electrical energy in century. IEEE; 2008. p. 1–4.
34. Aminifar F, Fotuhi-Firuzabad M, Shahidehpour, et al. A probabilistic multistage PMU placement in electric power systems. IEEE Trans Power Deliv 2011;26(2):841–9.
35. Gopakumar P, Chandra G, Reddy M, et al. Optimal placement of PMUs for the smart grid implementation in Indian power grid – a case study. Front Energy 2013;7(3):358–72.
36. Xu B, Abur A. Observability analysis and measurement placement for systems with PMUs. In: Proceedings of the IEEE PES power systems conference and exposition. Vol. 2; October 2004. p. 943–6.
37. Milosevic B, Begovic M. Nondominated sorting genetic algorithm for optimal phasor measurement placement. IEEE Trans Power Syst 2003;18(1):69–75.
38. Western interconnection synchrophasor program; June 2014. 〈https://www. smartgrid.gov/project/western〉.
39. Huang J, Wu N, Ruschmann M. Data-availability-constrained placement of PMUs and communication links in a power system. IEEE Syst J 2014;8(2):483–92.
40. Li Q, Cui T, Weng Y, Negi R, et al. An information-theoretic approach to PMU placement in electric power systems. IEEE Trans Smart Grid 2013;4(1):446–56.
41. Sodhi R, Srivastava SC, Singh SN. Multi-criteria decision-making approach for multistage optimal placement of phasor measurement units. IET Gener Transm Distrib 2011;5(2):181–90.
42. Domingos P, Hulten G. Mining high-speed data streams. In: Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining. 2000. p. 71–80.
43. Dahal N, Abuomar O, King R, et al. Event stream processing for improved situational awareness in the smart grid. Expert Syst Appl 2015;42(20):6853–63.
44. Ma S, Fang S, Yuan D, et al. The design of power security system in smart home based on the stream data mining. Adv Data Min Appl 2014:716–24.
45. Omitaomu OA. A control chart approach for representing and mining data streams with shape based similarity; 2014.
46. Basu C, Agrawal A, Hazra J. et al. Understanding events for wide-area situational awareness. In: Proceedings of the innovative smart grid technologies conference. IEEE; 2014. p. 1–5.
47. Peppanen J, Reno MJ, Thakkar M, et al. Leveraging AMI data for distribution system model calibration and situational awareness. IEEE Trans Smart Grid 2015;6(4):2050–9.
48. Schweppe FC, Wildes J. Power system static-state estimation,part I: exact model. IEEE Trans Power Appl Syst . 1970;PAS-89(1):120–5.
49. Zhao J, Zhang G, Das K, et al. Power system real-time monitoring by using PMU-based robust state estimation method. IEEE Trans Smart Grid 2015;7(1):1.
50. Pignati M, Zanni L, Sarri S. et al. A pre-estimation filtering process of bad data for linear power systems state estimators using PMUs. In: Proceedings of the power systems computation conference; 2014. p. 1–8.
51. Xie L, Chen Y, Kumar PR. Dimensionality reduction of synchrophasor data for early event detection: linearized analysis. IEEE Trans Power Syst 2014;29(6):2784–94.
52. DOE. Project information; 2016. 〈https://www.smartgrid.gov/recovery_act/ project_information.html〉.
53. Dahal OP, Brahma SM, Cao H. Comprehensive clustering of disturbance events recorded by phasor measurement units. IEEE Trans Power Deliv 2014;29(3):1390–7.
54. Chen Y, Xie L, Kumar PR. Power system event classification via dimensionality reduction of synchrophasor data. In: Proceedings of the IEEE, sensor array and multichannel signal processing workshop; 2014. p. 57–60.
55. IEEE Std C37.118.1™−2011. IEEE standard for synchrophasor measurements for power systems; 2011. p. 1–61.
56. IEEE Std C37.118.2™−2011. IEEE standard for synchrophasor measurements for power systems; 2011. p. 1–53.
57. Khan M, Li M, Ashton P. et al. Big data analytics on PMU measurements. In: Proceedings of the international conference on fuzzy systems and knowledge discovery; 2014.
58. Kezunovic M, Xie L, Grijalva S. The role of big data in improving power system operation and protection. In: Proceedings of the bulk power system dynamics and control – IX optimization, security and control of the emerging power grid (IREP), 2013 IREP symposium; 2013. p. 1–9.
59. Meier R, Cotilla-Sanchez E, Mccamish B, et al. Power system data management and analysis using synchrophasor data. Technol Sustain IEEE 2014:225–31.
60. Khan AR, Mahmood A, Safdar A, et al. Load forecasting, dynamic pricing and DSM in smart grid: a review. Renew Sustain Energy Rev 2016;54:1311–22.
61. Selakov A, Cvijetinović D, Milović L, et al. Hybrid PSO–SVM method for short-term load forecasting during periods with significant temperature variations in city of Burbank. Appl Soft Comput 2014;16(3):80–8.
62. Deihimi A, Orang O, Showkati H. Short-term electric load and temperature forecasting using wavelet echo state networks with neural reconstruction. Energy 2013;57(3):382–401.
63. Zhu L, Wu QH, Li MS. et al. Support vector regression-based short-term wind power prediction with false neighbors filtered. In: Proceedings of the international conference on renewable energy research and applications. IEEE; 2013. p. 740–4.
64. Rocio Cogollo M, Velasquez JD. Methodological advances in artificial neural networks for time series forecasting. IEEE Lat Am Trans 2014;12(4):764–71.
65. Huang SC, Lu CN, Lo YL. Evaluation of AMI and SCADA data synergy for distribution feeder modeling. IEEE Trans Smart Grid 2015;6(4), [1–1].
66. Berrisford A J. A tale of two transformers: an algorithm for estimating distribution secondary electric parameters using smart meter data; 2013. p. 1–6.
67. Kwac J, Rajagopal R. Data-driven targeting of customers for demand response. IEEE Trans Smart Grid, IEEE Trans Smart Grid 2016;7(5):2199–207.
68. Peppanen J, Reno MJ, Broderick RJ, et al. Distribution system model calibration with big data from AMI and PV inverters. IEEE Trans Smart Grid 2016;7(5):2497–506.
69. Pan S, Morris T, Adhikari U. Developing a hybrid intrusion detection system using data mining for power systems. IEEE Trans Smart Grid 2015;6(6):134–43.
70. Bayrak G, Kabalci E. Implementation of a new remote islanding detection method for wind–solar hybrid power plants. Renew Sustain Energy Rev 2016;58:1–15.
71. Li C, Higuma K, Watanabe M, et al. Monitoring and estimation of interarea power oscillation mode Based on application of CampusWAMS. Turk J Agric - Food Sci Technol 2008;3(1).
72. Dasgupta S, Paramasivam M, Vaidya U, et al. PMU-based model-free approach for real-time rotor angle monitoring. IEEE Trans Power Syst 2015;30(5):2818–9.
73. Liu Y, Zhan L, Zhang Y, et al. Wide-area measurement system development at the distribution level: an FNET/GridEye example. IEEE Trans Power Deliv 2015;31(2), [1–1].
74. Chai J, Liu Y, Guo J. et al. Wide-area measurement data analytics using FNET/ GridEye: a review[C]. In: Proceedings of the power systems computation conference; 2016. p. 1–6.
75. Kamwa J, Beland G, Trudel. et al. Wide-area monitoring and control at hydroquébec: past, present and future. In: Proceedings of the power engineering society general Meeting; 2006.
76. e-terraphasorpoint 3.0. 〈http://www.alstom.com/Global/Grid/Resources/ Documents/Automation/NMS/e-terraphasorpoint%203.0.pdf〉.
77. Siemens. 〈http://www.engerati.com/resources/efficient-network-integrationrenewable-energy-resources-distribution-level〉; 2015.
78. NARI. 〈http://www.naritech.cn/html/jie166.shtm〉; 2015.
79. Chang V, Wills G. A model to compare cloud and non-cloud storage of big data. Future Gener Comput Syst 2016;57:56–76.
80. Singh M, Ali A. Big data analytics with Microsoft HDInsight in 24h, Sams Teach Yourself: Big Data, Hadoop, and Microsoft Azure for better business intelligence; 2016.
81. Rusitschka S, Eger K, Gerdes C. Smart grid data cloud: a model for utilizing cloud computing in the smart grid domain. In: Proceedings of the IEEE international conference on smart grid communications; 2010. p. 483–8.
82. Birman KP, Ganesh L, Renesse RV. et al. Running smart grid control software on cloud computing architectures. In: Proceedings of the the workshop on computational needs for the next generation electric grid; 2012.
83. Baek J, Vu QH, Liu JK, et al. A secure cloud computing based framework for big data information management of smart grid. IEEE Trans Cloud Comput 2015;3(2):233–44.
84. Simmhan Y, Prasanna V, Aman S, et al. Cloud-based software platform for big data analytics in smart grids. Comput Sci Eng 2013;15(4):38–47.
85. Khan Mukhtaj. Hadoop performance modeling and job optimization for big data analytics. London: Brunel University; 2015.
86. Markovic DS, Zivkovic D, Branovic I, et al. Smart power grid and cloud computing. Renew Sustain Energy Rev 2013;24(1):566–77.
87. Fang B, Yin X, Tan Y, et al. The contributions of cloud technologies to smart grid. Renew Sustain Energy Rev 2016;59:1326–31.
88. Apache Hadoop. 〈http://hadoop.apache.org/〉; 2016.
89. 2MapReduce: incremental MapReduce for mining evolving big data. IEEE Trans Knowl Data Eng 2015;27(7):1.
90. Xing EP, Ho Q, Dai W. et al. Petuum: a new platform for distributed machine learning on big data. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining. ACM; 2015. p. 1–1.
91. Shyam R, Bharathi GHB, Sachin KS, et al. Apache spark a big data analytics platform for smart grid. 21; 2015. p. 171–8.
92. Apache Spark. 〈http://spark.apache.org/〉; 2016.
93. Zhou D, Guo J, Zhang Y, et al. Distributed data analytics platform for wide-area synchrophasor measurement systems. IEEE Trans Smart Grid 2016;7(5):2397–405.
94. Li D. Machine learning aided decision making and adaptive stochastic control in a hierarchical interactive smart grid. Dissertations & theses - gradworks; 2014.
95. Wu X, Zhu X, Wu GQ, et al. Data mining with big data. IEEE Trans Knowl Data Eng 2014;26(1):97–107.
96. Al-Jarrah OY, Yoo PD, Muhaidat S, et al. Efficient machine learning for big data: a review. Big Data Res 2015;2(3):87–93.
97. Khazaei J, Fan L, Jiang W, et al. Distributed prony analysis for real-world PMU data. Electr Power Syst Res 2016;133:113–20.
98. Shaker H, Zareipour H, Wood D. A data-driven approach for estimating the power generation of invisible solar sites. IEEE Trans Smart Grid 2016;7(5):2466–76.
99. Shaker H, Zareipour H, Wood D. Estimating power generation of invisible solar sites using publicly available data. IEEE Trans Smart Grid 2016;7(5):2456–65.
100. Taieb SB, Huser R, Hyndman RJ, et al. Forecasting uncertainty in electricity smart meter data by boosting additive quantile regression. IEEE Trans Smart Grid 2016;7(5):2448–55.
101. Green RC, Wang L, Alam M. Applications and trends of high performance computing for electric power systems: focusing on smart grid. IEEE Trans Smart Grid 2013;4(2):922–3.
102. Partridge M, Calvo RA. Fast dimensionality reduction and simple PCA. Intell Data Anal 1998;2(1–4):203–14.
103. Poekaew P, Champrasert P. Adaptive-PCA: an event-based data aggregation using principal component analysis for WSNs. In: Proceedings of the international conference on smart sensors and application. IEEE; 2015.
104. Nuchprayoon S. Electricity load classification using K-means clustering algorithm. In: Proceedings of the Brunei international conference on engineering and technology. IET; 2014. p. 1–5.
105. Yang H, Zhang J, Qiu J, et al. A practical pricing approach to smart grid demand response based on load classification. IEEE Trans Smart Grid 2016, [1–1].
106. Mu FL, Li HY. Power load classification based on spectral clustering of dual-scale. In: Proceedings of the IEEE international conference on control science and systems engineering. IEEE; 2014.
107. Ye M, Liu Y, Xiong W. et al. Voltage stability research of receiving-end network based on real-time classification load model. In: Proceedings of the IEEE international conference on mechatronics and automation; 2014. p. 1866-70.
108. Che JX, Wang JZ. Short-term load forecasting using a kernel-based support vector regression combination model. Appl Energy 2014;132(11):602–9.
109. Liu N, Tang Q, Zhang J, et al. A hybrid forecasting model with parameter optimization for short-term load forecasting of micro-grids. Appl Energy 2014;129:336–45.
110. Rahman S. Formation and analysis of a rule-based short-term load forecasting algorithm. In: Proceedings of the IEEE (Institute of electrical and electronics engineers), (USA). 78:5(5); 2015. p. 805–16.
111. Hernández L, Baladrón C, Aguiar JM, et al. Artificial neural networks for short-term load forecasting in microgrids environment. Energy 2014;75(C):252–64.
112. Nickel M, Murphy K, Tresp V, et al. A review of relational machine learning for knowledge graphs. Proc IEEE 2015;104(1):11–33.
113. Drumond LR, Diaz-Aviles E, Schmidt-Thieme L. et al. Optimizing multi-relational factorization models for multiple target relations. In: Proceedings of the ACM international conference; 2014. p. 191–200.
114. Schweppe F, Wildes J. Power system static-state estimation, part I, II, III. IEEE Trans Power Appl Syst . 1970;PAS-89(1):120–35.
115. Xu X, He X, Ai Q, et al. A correlation analysis method for power systems based on random matrix theory. IEEE Trans Smart Grid 2015.
116. Yang Q. Cross-domain data fusion. Computer 2016;49(4), [18–18].
117. Diou C, Stephanopoulos G, Panagiotopoulos P, et al. Large-scale concept detection in multimedia data using small training sets and cross-domain concept fusion. IEEE Trans Circuits Syst Video Technol 2011;20(12):1808–21.
118. Tsai CC, Chang-Chien LR, Chen IJ, et al. Practical considerations to calibrate generator model parameters using phasor measurements. IEEE Trans Smart Grid 2016:1–11.
119. Hana S. SAP AG, in-memory database, computer appliance. Vertpress; 2012.
120. Kennedy, Stephen James MCP. M.C.P. Massachusetts Institute of TechnologyTransforming big data into knowledge: experimental techniques in dynamic visualization. Massachusetts Institute of Technology; 2012.
121. Cuzzocrea A. Privacy and security of big data: current challenges and future research perspectives. In: Proceedings of the international workshop on privacy & secuirty of big data. ACM; 2014. p. 45–7.
122. Bertino E. Big data – security and privacy. In: Proceedings of the IEEE international congress on big data. IEEE; 2015.
123. Depuru SSSR, Wang L, Devabhaktuni V. Smart meters for power grid: challenges, issues, advantages and status. Renew Sustain Energy Rev 2011;15(6):2736–42.
124. Kabalci Y. A survey on smart metering and smart grid communication. Renew Sustain Energy Rev 2016;57:302–18.
125. Hashem IAT, Yaqoob I, Anuar NB, et al. The rise of “big data” on cloud computing: review and open research issues. Inf Syst 2015;47(47):98–115.
126. Jagadish HV, Gehrke J, Labrinidis A, et al. Big data and its technical challenges. Commun Acm 2014;57(7):86–94.
127. Schweppe FC, Rom DB. Power system static-state estimation, part II: approximate model. IEEE Trans Power Appl Syst 1970;PAS-89(1):125–30.
128. Schweppe FC. Power system static-state estimation, part III: implementation. IEEE Trans Power Appl Syst 1970;PAS-89(1):130–5.
129. Gol M, Abur A. A fast decoupled state estimator for systems measured by PMUs. IEEE Trans Power Syst 2015;30(5):2766–71.
130. Jokar P, Arianpoo N, Leung VCM. Electricity theft detection in AMI using customers' consumption patterns. IEEE Trans Smart Grid 2016;7(1):216–26.
131. Shuai Z, Sun Y, Shen Z, et al. Microgrid stability: classification and a review. Renew Sustain Energy Rev 2016;58:167–79.