A benchmarking approach for comparing data splitting methods for modeling water resources parameters using artificial neural networks

Water Resources Research

Volumn 49, Issue 11, 2013, Pages 7598-7614

  Wu, Wenyan ^a   May, Robert J ^a,b   Maier, Holger R ^a   Dandy, Graeme C ^a  

^a UNIVERSITY OF ADELAIDE   (Australia)

^b Veolia Eau Compagnie Generale des Eaux   (France)

Author keywords

artificial neural networks; benchmark; data splitting methods; data variability; validation

Indexed keywords

DATA SPLITTING; DATA VARIABILITY; DEVELOPMENT PROCESS; GENERALIZATION ABILITY; MODEL PERFORMANCE; PREDICTIVE PERFORMANCE; VALIDATION; VALIDATION ERRORS;

ERRORS; NEURAL NETWORKS; WATER RESOURCES;

BENCHMARKING;

ARTIFICIAL NEURAL NETWORK; BENCHMARKING; DATA ACQUISITION; DATA SET; ERROR ANALYSIS; ESTIMATION METHOD; OPTIMIZATION; PARAMETERIZATION; PERFORMANCE ASSESSMENT; WATER RESOURCE;

EID: 84887871464     PISSN: 00431397     EISSN: 19447973     Source Type: Journal    
DOI: 10.1002/2012WR012713     Document Type: Article

References (44)

1. Abrahart, R. J., F. Anctil, P. Coulibaly, C. W. Dawson, N. J. Mount, L. M. See, A. Y. Shamseldin, D. P. Solomatine, E. Toth, and, R. L. Wilby, (2012), Two decades of anarchy? Emerging themes and outstanding challenges for neural network river forecasting, Prog. Phys. Geogr., 36, 480-513.
2. Akaike, H., (1974), A new look at the statistical model identification, IEEE Trans. Autom. Control, 19 (6), 716-723.
3. Anctil, F., and, N. Lauzon, (2004), Generalisation for neural networks through data sampling and training procedures, with applications to streamflow predictions, Hydrol. Earth Syst. Sci., 8, 940-958, doi: 10.5194/hess-8-940-2004. (Pubitemid 40294956)
4. ASCE Task Committee on Application of Artificial Neural Networks in Hydrology (2000a), Artificial neural networks in hydrology. II: Hydrologic applications, J. Hydrol. Eng., 5 (2), 124-137.
5. ASCE Task Committee on Application of Artificial Neural Networks in Hydrology (2000b), Artificial neural networks in hydrology. I: Preliminary concepts, J. Hydrol. Eng., 5 (2), 115-123.
6. Baxter, C. W., S. J. Stanley, Q. Zhang, and, D. W. Smith, (2000), Developing artificial neural network process models: A guide for drinking water utilities, paper presented at the 6th Environmental Engineering Society Specialty Conference of the CSCE, pp. 376-383, Canadian Society for Civil Engineering (CSCE), London, Ontario, Canada.
7. Bowden, G. J., H. R. Maier, and, G. C. Dandy, (2002), Optimal division of data for neural network models in water resources applications, Water Resour. Res., 38 (2), 1010, doi: 10.1029/2001WR000266.
8. Bowden, G. J., H. R. Maier, and, G. C. Dandy, (2012), Real-time deployment of artificial neural network forecasting models: Understanding the range of applicability, Water Resour. Res., 48 (10), W10549, doi: 10.1029/2012WR011984.
9. Bowden, G. J., J. B. Nixon, G. C. Dandy, H. R. Maier, and, M. Holmes, (2006), Forecasting chlorine residuals in a water distribution system using a general regression neural network, Math. Comput. Modell., 44 (5-6), 469-484. (Pubitemid 43946635)
10. Breiman, L., (1996), Bagging Inputs, Mach. Learn., 24, 123-140.
11. Crow, E. L., F. A. Davis, and, M. W. Maxfield, (1960), Statistics Manual, Dover, New York.
12. Dawson, C. W., and, R. L. Wilby, (2001), Hydrological modelling using artificial neural networks, Prog. Phys. Geogr., 25 (1), 80-108. (Pubitemid 33028258)
13. Despagne, F., and, D. Luc Massart, (1998), Neural networks in multivariate calibration, Analyst, 123 (11), 157R-178R. (Pubitemid 28528625)
14. Elshorbagy, A., G. Corzo, S. Srinivasulu, and, D. P. Solomatine, (2010a), Experimental investigation of the predictive capabilities of data driven modeling techniques in hydrology-Part 1: Concepts and methodology, Hydrol. Earth Syst. Sci., 14, 1931-1941, doi: 10.5194/hess-14-1931-2010.
15. Elshorbagy, A., G. Corzo, S. Srinivasulu, and, D. P. Solomatine, (2010b), Experimental investigation of the predictive capabilities of data driven modeling techniques in hydrology-Part 2: Application, Hydrol. Earth Syst. Sci., 14, 1943-1961, doi: 10.5194/hess-14-1943-2010.
16. Haas, C. N., (2004), Neural networks provide superior description of Giardia lamblia inactivation by free chlorine, Water Res., 38 (14-15), 3449-3457, doi: 10.1016/j.watres.2004.05.001. (Pubitemid 38987724)
17. Jain, A., and, S. Srinivasulu, (2006), Integrated approach to model decomposed flow hydrograph using artificial neural network and conceptual techniques, J. Hydrol., 317 (3-4), 291-306, doi: 10.1016/j.jhydrol. 2005.05.022. (Pubitemid 41774118)
18. Kennard, R. W., and, L. A. Stone, (1969), Computer aided design of experiments, Technometrics, 11 (1), 137-148.
19. Khalil, A., M. N. Almasri, M. McKee, and, J. J. Kaluarachchi, (2005), Applicability of statistical learning algorithms in groundwater quality modeling, Water Resour. Res., 41, W05010, doi: 10.1029/2004WR 003608.
20. Kingston, G. B., (2006), Bayesian artificial neural networks in water resources engineering, PhD thesis, The Univ. of Adelaide, Adelaide, Australia.
21. Kingston, G. B., H. R. Maier, and, M. F. Lambert, (2005), Calibration and validation of neural networks to ensure physically plausible hydrological modeling, J. Hydrol., 314 (1-4), 158-176, doi: 10.1016/j.jhydrol. 2005.03.013. (Pubitemid 41727778)
22. Kohonen, T., (1998), The self-organizing map, Neurocomputing, 21 (1-3), 1-6, doi: 10.1016/s0925-2312(98)00030-7. (Pubitemid 28526808)
23. LeBaron, B., and, A. S. Weigend, (1998), A bootstrap evaluation of the effect of data splitting on financial time series, IEEE Trans. Neural Networks, 9 (1), 213-220. (Pubitemid 128743627)
24. Lendasse, A., V. Wertz, and, M. Verleysen, (2003), Model selection with cross-validations and bootstraps: Application to time series prediction with RBFN models, paper presented at the 2003 joint International Conference on Artificial Neural Networks and Neural Information Processing, Springer, Istanbul, Turkey.
25. Maidment, D. R., (Ed.) (1993), Handbook of Hydrology, McGraw-Hill, New York.
26. Maier, H. R., and, G. C. Dandy, (1996), The use of artificial neural networks for the prediction of water quality parameters, Water Resour. Res., 32 (4), 1013-1022, doi: 10.1029/96WR03529.
27. Maier, H. R., and, G. C. Dandy, (2000), Neural networks for the prediction and forecasting of water resources variables: A review of modelling issues and applications, Environ. Modell. Software, 15 (1), 101-124. (Pubitemid 30018318)
28. Maier, H. R., A. Jain, G. C. Dandy, and, K. P. Sudheer, (2010), Methods used for the development of neural networks for the prediction of water resource variables in river systems: Current status and future directions, Environ. Modell. Software, 25 (8), 891-909, doi: 10.1016/j.envsoft. 2010.02.003.
29. Maier, H. R., N. Morgan, and, C. W. K. Chow, (2004), Use of artificial neural networks for predicting optimal alum doses and treated water quality parameters, Environ. Modell. Software, 19 (5), 485-494, doi: 10.1016/s1364-8152(03)00163-4. (Pubitemid 38355873)
30. May, R. J., G. C. Dandy, H. R. Maier, and, J. B. Nixon, (2008a), Application of partial mutual information variable selection to ANN forecasting of water quality in water distribution systems, Environ. Modell. Software, 23 (10-11), 1289-1299, doi: 10.1016/j.envsoft.2008.03.008.
31. May, R. J., H. R. Maier, and, G. C. Dandy, (2010), Data splitting for artificial neural networks using SOM-based stratified sampling, Neural Networks, 23 (2), 283-294.
32. May, R. J., H. R. Maier, G. C. Dandy, and, T. Fernando, (2008b), Non-linear variable selection for artificial neural networks using partial mutual information, Environ. Modell. Software, 23 (10-11), 1312-1326, doi: 10.1016/j.envsoft.2008.03.007.
33. Meester, R., (2008), A Natural Introduction to Probability Theory, Birkhäuser, Basel-Boston-Berlin, Basel, Switzerland.
34. Mount, N. J., C. W. Dawson, and, R. J. Abrahart, (2013), Legitimising neural network river forecasting models: A new data-driven mechanistic modelling framework, Hydrol. Earth Syst. Sci. Discuss., 10, 145-187, doi: 10.5194/hessd-10-145-2013.
35. Press, W. H., S. A. Tuekolsky, W. T. Vetterling, and, B. P. Falnnery, (1992), Numerical Recipes: The Art of Scientific Computing, Cambridge Univ. Press, Cambridge.
36. Sahoo, A. K., M. J. Zuo, and, M. K. Tiwari, (2012), A data clustering algorithm for stratified data partitioning in artificial neural network, Exp. Syst. Appl., 39 (8), 7004-7014, doi: 10.1016/j.eswa.2012.01.047.
37. Shahin, M., H. R. Maier, and, M. B. Jaksa, (2004), Data division for developing neural networks applied to geotechnical engineering, J. Comput. Civil Eng., 18, 105-114, doi: 10.1061/(ASCE)0887-3801(2004)18:2(105). (Pubitemid 40475427)
38. Shu, C., and, D. H. Burn, (2004), Artificial neural network ensembles and their application in pooled flood frequency analysis, Water Resour. Res., 40, W09301, doi: 10.1029/2003WR002816.
39. Snee, R. D., (1977), Validation of regression models: Methods and examples, Technometrics, 19 (4), 415-428.
40. Sokal, R. R., and, F. J. Rohlf, (1994), Biometry, 3rd ed., Freeman, New York.
41. Specht, D. F., (1991), A general regression neural network, IEEE Trans. Neural Networks, 2 (6), 568-576.
42. Vesanto, J., (1999), SOM-based data visualization methods, Intell. Data Anal., 3 (2), 111-126, doi: 10.1016/s1088-467x(99)00013-x. (Pubitemid 33260739)
43. Wu, W., H. R. Maier, G. C. Dandy, and, R. May, (2012), Exploring the impact of data splitting methods on artificial neural network models, paper presented at the 10th International Conference on Hydroinformatics, 14-18 July, Hamburg, Germany.
44. Xu, Q.-S., and, Y.-Z. Liang, (2001), Monte Carlo cross validation, Chem. Intell. Lab. Syst., 56 (1), 1-11, doi: 10.1016/s0169-7439(00)00122-2. (Pubitemid 32224844)