Neural network and regression spline value function approximations for stochastic dynamic programming

Computers and Operations Research

Volumn 34, Issue 1, 2007, Pages 70-90

  Cervellera, Cristiano ^a   Wen, Aihong ^b   Chen, Victoria C P ^b  

^a INSTITUTE OF INTELLIGENT SYSTEMS FOR AUTOMATION   (Italy)

^b The University of Texas at Arlington   (United States)

Author keywords

Design of experiments; Inventory forecasting; Latin hypercube; Markov decision process; Orthogonal array; Statistical modeling; Water reservoir management

Indexed keywords

APPROXIMATION THEORY; DECISION MAKING; DYNAMIC PROGRAMMING; NEURAL NETWORKS; RANDOM PROCESSES; REGRESSION ANALYSIS; RESERVOIRS (WATER); VALUE ENGINEERING;

DESIGN OF EXPERIMENTS; INVENTORY FORECASTING; LATIN HYPERCUBE; MARKOV DECISION PROCESS; WATER RESERVOIR MANAGEMENT;

STOCHASTIC PROGRAMMING;

EID: 33746257756     PISSN: 03050548     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cor.2005.02.043     Document Type: Article

References (64)

1. Bellman R.E. Dynamic programming (1957), Princeton University Press, Princeton
2. Puterman M.L. Markov decision processes (1994), Wiley, New York, NY
3. Bertsekas D.P., and Tsitsiklis J.N. Dynamic programming and optimal control (2000), Athena Scientific, Belmont, MA
4. Gal S. Optimal management of a multireservoir water supply system. Water Resources Research 15 (1979) 737-748
5. Shoemaker C.A. Optimal integrated control of univoltine pest populations with age structure. Operations Research 30 (1982) 40-61
6. White D.J. Real applications of Markov decision processes. Interfaces 15 6 (1985) 73-83
7. White D.J. Further real applications of Markov decision processes. Interfaces 18 5 (1988) 55-61
8. Foufoula-Georgiou E., and Kitanidis P.K. Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems. Water Resources Research 24 (1988) 1345-1359
9. Culver T.B., and Shoemaker C.A. Dynamic optimal ground-water reclamation with treatment capital costs. Journal of Water Resources Planning and Management 123 (1997) 23-29
10. Chen VCP, Günther D, Johnson EL. Solving for an optimal airline yield management policy via statistical learning. Journal of the Royal Statistical Society, Series C 2003; 52(1):1-12.
11. Tsai J.C.C., Chen V.C.P., Chen J., and Beck M.B. Stochastic dynamic programming formulation for a wastewater treatment decision-making framework. Annals of Operations Research, Special Issue on Applied Optimization Under Uncertainty 132 (2004) 207-221
12. Johnson S.A., Stedinger J.R., Shoemaker C.A., Li Y., and Tejada-Guibert J.A. Numerical solution of continuous-state dynamic programs using linear and spline interpolation. Operations Research 41 (1993) 484-500
13. Eschenbach E.A., Shoemaker C.A., and Caffey H. Parallel processing of stochastic dynamic programming for continuous state systems with linear interpolation. ORSA Journal on Computing 7 (1995) 386-401
14. Bertsekas D.P., and Tsitsiklis J.N. Neuro-dynamic programming (1996), Athena Scientific, Belmont, MA
15. Chen V.C.P., Ruppert D., and Shoemaker C.A. Applying experimental design and regression splines to high-dimensional continuous-state stochastic dynamic programming. Operations Research 47 (1999) 38-53
16. Chen V.C.P. Application of MARS and orthogonal arrays to inventory forecasting stochastic dynamic programs. Computational Statistics and Data Analysis 30 (1999) 317-341
17. Chen V.C.P. Measuring the goodness of orthogonal array discretizations for stochastic programming and stochastic dynamic programming. SIAM Journal of Optimizations 12 (2001) 322-344
18. Friedman J.H. Multivariate adaptive regression splines (with discussion). Annals of Statistics 19 (1991) 1-141
19. Carey W.P., and Yee S.S. Calibration of nonlinear solid-state sensor arrays using multivariate regression techniques. Sensors and Actuators, B. Chemical B9 2 (1992) 113-122
20. Griffin W.L., Fisher N.I., Friedman J.H., and Ryan C.G. Statistical techniques for the classification of chromites in diamond exploration samples. Journal of Geochemical Exploration 59 3 (1997) 233-249
21. Kuhnert P.M., Do K.-A., and McClure R. Combining non-parametric models with logistic regression. an application to motor vehicle injury data. Computational Statistics and Data Analysis 34 3 (2000) 371-386
22. Shepherd K.D., and Walsh M.G. Development of reflectance spectral libraries for characterization of soil properties. Soil Science Society of America Journal 66 3 (2002) 988-998
23. Smets H.M.G., and Bogaerts W.F.L. Deriving corrosion knowledge from case histories: the neural network approach. Materials & Design 13 3 (1992) 149-153
24. Labossiere P., and Turkkan N. Failure prediction of fibre-reinforced materials with neural networks. Journal of Reinforced Plastics and Composites 12 12 (1993) 1270-1280
25. Abdelaziz A.Y., Irving M.R., Mansour M.M., El-Arabaty A.M., and Nosseir A.I. Neural network-based adaptive out-of-step protection strategy for electrical power systems. International Journal of Engineering Intelligent Systems for Electrical Engineering and Communications 5 1 (1997) 35-42
26. Chen V.C.P., and Rollins D.K. Issues regarding artificial beural network modeling for reactors and fermenters. Bioprocess Engineering 22 (2000) 85-93
27. Li Q.S., Liu D.K., Fang J.Q., Jeary A.P., and Wong C.K. Damping in buildings: its neural network model and AR model. Engineering Structures 22 9 (2000) 1216-1223
28. Liu J. Prediction of the molecular weight for a vinylidene chloride/vinyl chloride resin during shear-related thermal degradation in air by using a back-propagation artificial neural network. Industrial and Engineering Chemistry Research 40 24 (2001) 5719-5723
29. Pigram G.M., and Macdonald T.R. Use of neural network models to predict industrial bioreactor effluent quality. Environmental Science and Technology 35 1 (2001) 157-162
30. Kottapalli S. Neural network based representation of UH-60A pilot and hub accelerations. Journal of the American Helicopter Society 47 1 (2002) 33-41
31. Tang B. Orthogonal array-based Latin hypercubes. Journal of the American Statistical Association 88 (1993) 1392-1397
32. Chen V.C.P., Tsui K.-L., Barton R.R., and Allen J.K. A review of design and modeling in computer experiments. In: Khattree R., and Rao C.R. (Eds). Handbook of statistics. statistics in industry vol. 22 (2003), Elsevier Science, Amsterdam 231-261
33. Chen V.C.P., and Welch W.J. Statistical methods for deterministic biomathematical models. Proceedings of the 53rd session of the international statistical institute, Seoul, Korea (August 2001)
34. De Veaux R.D., Psichogios D.C., and Ungar L.H. Comparison of two nonparametric estimation schemes. MARS and neural networks. Computers & Chemical Engineering 17 8 (1993) 819-837
35. Pham D.T., and Peat B.J. Automatic learning using neural networks and adaptive regression. Measurement and Control 32 9 (1999) 270-274
36. Palmer KD. Data collection plans and meta models for chemical process flowsheet simulators. PhD dissertation, School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, USA, 1998.
37. Simpson T.W., Lin D.K.J., and Chen W. Sampling strategies for computer experiments. design and analysis. International Journal of Reliability and Applications 2 3 (2001) 209-240
38. Allen T., Bernshteyn M.A., and Kabiri K. Constructing meta-models for computer experiments. Journal of Quality Technology 35 (2003) 264-274
39. Baglietto M., Cervellera C., Parisini T., Sanguineti M., and Zoppoli R. Approximating networks for the solution of T-stage stochastic optimal control problems. Proceedings of the IFAC workshop on adaptation and learning in control and signal processing (2001)
40. Baglietto M, Cervellera C, Sanguineti M, Zoppoli R. Water reservoirs management under uncertainty by approximating networks and learning from data. In: Topics on participatory planning and managing water systems. Amsterdam: Elsevier Science Ltd., 2005, to appear.
41. Hadley G., and Whitin T.M. Analysis of inventory systems (1963), Prentice-Hall, Englewood Cliffs, NJ
42. Heath DC, Jackson PL. Modelling the evolution of demand forecasts with application to safety stock analysis in production/distribution systems. Technical report #989, School of Operations Research & Industrial Engineering, Cornell University, Ithaca, NY, 1991.
43. Archibald T.W., McKinnon K.I.M., and Thomas L.C. An aggregate stochastic dynamic programming model of multireservoir Systems. Water Resources Research 33 (1997) 333-340
44. Lamond B.F., and Sobel M.J. Exact and approximate solutions of affine reservoirs models. Operations Research 43 (1995) 771-780
45. Turgeon A. A decomposition method for the long-term scheduling of reservoirs in series. Water Resources Research 17 (1981) 1565-1570
46. Yakowitz S. Dynamic programming applications in water resources. Water Resources Research 18 (1982) 673-696
47. Chen V.C.P., Chen J., and Beck M.B. Statistical learning within a decision-making framework for more sustainable urban environments. Proceedings of the joint research conference on statistics in quality, industry, and technology, Seattle, WA (June 2000)
48. Rao C.R. Hypercubes of strength 'd' leading to confounded designs in factorial experiments. Bulletin of the Calcutta Mathematical Society 38 (1946) 67-78
49. Bose R.C., and Bush K.A. Orthogonal arrays of strength two and three. Annals of Mathematical Statistics 23 (1952) 508-524
50. Raghavarao D. Constructions and combinatorial problems in design of experiments (1971), Wiley, New York
51. Hedayat A.S., and Wallis W.D. Hadamard matrices and their applications. Annals of Statistics 6 (1978) 1184-1238
52. Dey A. Orthogonal fractional factorial designs (1985), Wiley, New York
53. Hedayat A.S., Stufken J., and Su G. On difference schemes and orthogonal arrays of strength t. Journal of Statistical Planning and Inference 56 (1996) 307-324
54. Lippmann RP. An introduction to computing with neural nets. IEEE ASSP Magazine (1987) 4-22.
55. Haykin S.S. Neural networks. a comprehensive foundation. 2nd ed. (1999), Prentice-Hall, Upper Saddle River, NJ
56. Cherkassky V., and Mulier F. Learning from data. concepts theory, and methods (1998), Wiley, New York
57. White H. Learning in neural networks. a statistical perspective. Neural Computation 1 (1989) 425-464
58. Barron A.R., Barron R.L., and Wegman E.J. Statistical learning networks. a unifying view. Computer science and statistics. Proceedings of the 20th symposium on the interface (1992) 192-203
59. Ripley B.D. Statistical aspects of neural networks. In: Barndorff-Nielsen O.E., Jensen J.L., and Kendall W.S. (Eds). Networks and chaos-statistical and probabilistic aspects (1993), Chapman & Hall, New York 40-123
60. Cheng B., and Titterington D.M. Neural networks. a review from a statistical perspective (with discussion). Statistical science 9 (1994) 2-54
61. Barron A.R. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory 39 (1993) 930-945
62. Rumelhart D.E., Hinton G.E., and Williams R.J. Learning internal representations by error propagation. In: Rumelhart D.E., and McClelland J.L. (Eds). Parallel distributed processing. explorations in the microstructure of cognition (1986), MIT Press, Cambridge, MA 318-362
63. Werbos P.K. The roots of backpropagation. from ordered derivatives to neural networks and political forecasting (1994), Wiley, New York
64. Hagan M.T., and Menhaj M. Training feedforward networks with the Marquardt algorithm. IEEE Transaction on Neural Networks 5 (1994) 989-993