New approximate dynamic programming algorithms for large-scale undiscounted Markov decision processes and their application to optimize a production and distribution system

European Journal of Operational Research

Volumn 249, Issue 1, 2016, Pages 22-31

  Ohno, Katsuhisa ^a   Boh, Toshitaka ^b   Nakade, Koichi ^c   Tamura, Takayoshi ^a  

^a AICHI INSTITUTE OF TECHNOLOGY   (Japan)

^b PANASONIC CORPORATION   (Japan)

^c NAGOYA INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Approximate dynamic programming algorithms; JIT based production and distribution system; Optimal control; The curses of dimensionality; Undiscounted Markov decision processes

Indexed keywords

COMPUTER SYSTEMS PROGRAMMING; ITERATIVE METHODS; JUST IN TIME PRODUCTION; MARKOV PROCESSES; OPTIMAL CONTROL SYSTEMS; POISSON DISTRIBUTION; STOCHASTIC CONTROL SYSTEMS; STOCHASTIC SYSTEMS;

APPROXIMATE DYNAMIC PROGRAMMING; DISTRIBUTION SYSTEMS; MARKOV DECISION PROCESSES; OPTIMAL CONTROLS; THE CURSES OF DIMENSIONALITY;

DYNAMIC PROGRAMMING;

EID: 84948715868     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2015.07.026     Document Type: Article

References (30)

1. Bellman R. Dynamic programming 1957 Princeton University Press Princeton
2. Bertsekas D.P., Tsitsiklis J.N. Neuro-dynamic programming 1996 Athena Scientific
3. Bertsekas D.P. Pathologies of temporal difference methods in approximate dynamic programming Proceedings of 2010 conference on decision and control, Atlanta, GA, 2010 December 2010
4. Bertsekas D.P. Approximate policy iteration: A survey and some new methods Journal of Control Theory and Application 9 3 2011 310 335
5. Bonvik A.M., Couch C.E., Gershwin S.B. A comparison of production-line control mechanisms International Journal of Production Research 35 3 1997 789 804
6. Buşoniu L., Babuška R., Schutter B.D., Ernst D. Reinforcement learning and dynamic programming using function approximators 2010 CRC Press Boca Raton
7. Cao X.-R. Stochastic learning and optimization - A sensitivity-based approach 2007 Springer NY
8. Chang H.S., Fu M.C., Hu J., Marcus S.I. Simulation-based algorithms for Markov decision processes 2007 Springer-Verlag
9. Clark A.J., Scarf H. Optimal policies for multi-echelon inventory problems Management Science 6 1960 475 490
10. Cooper W.L., Henderson S.G., Lewis M.E. Convergence of simulation-based policy iteration Probability in the Engineering and Informational Sciences 17 2003 213 234
11. Dallery Y., Liberopoulos G. Extended kanban control system: Combining kanban and base stock IIE Transactions 32 2000 369 386
12. Das T.K., Gosavi A., Mahadevan S., Marchalleck N. Solving semi-Markov decision problems using average reward reinforcement learning Management Science 45 1999 560 574
13. Desai V.V., Farias V.F., Moallemi C.C. Approximate dynamic programming via a smoothed linear program Operations Research 60 2012 655 674
14. Gosavi A., Bandla N., Das T.K. A reinforcement learning approach to a single-leg airline revenue management problem with multiple fare classes and overbooking IIE Transactions 34 2002 729 742
15. Gosavi A. Simulation-based optimization: Parametric optimization techniques and reinforcement learning 2003 Kluwer Academic
16. He Y., Fu M.C., Marcus S.I. A simulation- based policy iteration algorithm for average cost unichain Markov decision processes M. Laguna J.L.G. Velarde Computing tools for modeling, optimization and simulation 2000 Kluwer Academic 161 182
17. Katanyukul T., Duff W.S., Chong E.K.P. Approximate dynamic programming for an inventory problem: Empirical comparison Computers & Industrial Engineering 60 2011 719 743
18. Monden Y. Toyota production system 5th ed. 2012 CRC Press New York
19. Ohno K. Modified policy iteration algorithm with nonoptimality tests for undiscounted Markov decision process Working Paper 1985 Department of Information System and Management Science, Konan University, Japan
20. Ohno K., Ichiki K. Computing optimal policies for controlled tandem queueing systems Operations Research 35 1 1987 121 126
21. Ohno K., Nakashima K., Kojima M. Optimal numbers of two kinds of kanbans in a JIT production system International Journal of Production Research 33 5 1995 1387 1401
22. Ohno K., Yashima K., Ito T. Neuro-dynamic programming algorithms for computing optimal control of production lines Journal of Japan Industrial Management Association 54 5 2003 316 325 (in Japanese)
23. Ohno K., Ito T. An optimal control of a production and distribution system by neuro-dynamic programming and a comparison of pull systems Journal of Japan Industrial Management Association 55 4 2004 179 188 (in Japanese)
24. Ohno K. The optimal control of just-in-time-based production and distribution systems and performance comparisons with optimized pull systems European Journal of Operational Research 213 2011 124 133
25. Powell W.B. Approximate dynamic programming - Solving the curses of dimensionality 2007 Wiley-Interscience New Jersey
26. Powell W.B., Ma J. A review of stochastic algorithms with continuous value function approximation and some new approximate policy iteration algorithms for multidimensional continuous applications Journal of Control Theory and Applications 9 3 2011 336 352
27. Puterman M.L. Markov decision processes: Discrete stochastic dynamic programming 1994 John Wiley & Sons New York
28. Si J., Barto A., Powell W., Wunsch D. Handbook of learning and approximate dynamic programming 2004 IEEE Press
29. Spearman M.L., Woodruff D.L., Hopp W.J. CONWIP: A pull alternative to kanban International Journal of Production Research 28 5 1990 879 894
30. Sutton R.S., Barto A.G. Reinforcement learning 1998 MIT Press