Reinforcement learning for joint pricing, lead-time and scheduling decisions in make-to-order systems

European Journal of Operational Research

Volumn 221, Issue 1, 2012, Pages 99-109

  Li, Xueping ^a   Wang, Jiao ^a   Sawhney, Rapinder ^a  

^a University of Tennessee   (United States)

Author keywords

Pricing; Q learning; Reinforcement learning (RL); Scheduling; Semi Markov Decision Problem (SMDP); Simulation based optimization

Indexed keywords

CAPACITY CONSTRAINTS; DISCRETE-EVENT SIMULATION MODEL; EXPECTED PROFITS; FIXED COST; HEURISTIC POLICIES; INVENTORY HOLDING; JOINT PRICING; LEADTIME; MAKE TO ORDER; PLANNING HORIZONS; Q-LEARNING; Q-LEARNING ALGORITHMS; SCHEDULING DECISIONS; SEMI-MARKOV DECISION PROBLEMS; SIMULATION-BASED OPTIMIZATIONS; STOCHASTIC DEMAND; TARDINESS COST; VARIABLE COSTS;

BENCHMARKING; COST ACCOUNTING; COSTS; INDUSTRY; LEARNING ALGORITHMS; PROFITABILITY; SCHEDULING;

REINFORCEMENT LEARNING;

EID: 84860259623     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2012.03.020     Document Type: Article

References (55)

1. H. Ahn, M. Gumus, and P. Kaminsky Pricing and manufacturing decisions when demand is a function of price in multiple period Operations Research 55 6 2007 1039 1057
2. E.A. Boyd, and I.C. Bilegan Revenue management and e-commerce Management Science 49 10 2003 1363 1386
3. K. Charnsirisakskul, P. Griffin, and P. Keskinocak Pricing and scheduling decisions with leadtime flexibility European Journal of Operational Research 171 1 2006 153 169
4. M. Chen, and M. Chu The analysis of optimal control model in matching problem between manufacturing and marketing European Journal of Operational Research 150 2 2003 293 303
5. G. Chryssolouris, N. Giannelos, N. Papakostas, and D. Mourtzis Chaos theory in production scheduling CIRP Annals 53 1 2004 381 383
6. C. Darken, and J. Moody Towards fasters stochastic gradient search J. Moody, S. Hanson, R. Lippmann, Advances in Neural Information Processing Systems vol. 4 1992 Morgan Kaufmann San Mateo, CA 1009 1016
7. T.K. Das, A. Gosavi, S. Mahadevan, and N. Marchalleck Solving semi-markov decision problems using average reward reinforcement learning Management Science 45 4 1999 560 574
8. S. Deng, and C.A. Yano Joint production and pricing decision with setup costs and capacity constraints Management Science 52 5 2006 741 756
9. I. Duenya Single facility due date setting with multiple customer classes Management Science 41 4 1995 608 619
10. I. Duenya, and W.J. Hopp Quoting customer lead times Management Science 41 1 1995 43 57
11. F. Easton, and D. Moodie Pricing and lead time decision for make-to-order firms with contingent orders European Journal of Operational Research 116 2 1999 305 318
12. J. Eliashberg, and R. Steinberg Marketing-production joint decision-making J. Eliashberg, G.L. Lilien, Handbooks in Operations Research and Management Science Vol. 5 1993 North Holland Amsterdam 827 880
13. G. Feichtinger, and R. Hartl Optimal pricing and production in an inventory model European Journal of Operational Research 19 1 1985 45 46
14. S.M. Gilbert Coordination of pricing and multi-period production for constant priced goods European Journal of Operational Research 114 2 1999 330 337
15. S.M. Gilbert Coordination of pricing and multi-period production across multiple constant priced goods Management Science 46 12 2000 1602 1616
16. A. Gosavi A reinforcement learning algorithm based on policy iteration for average reward: empirical results with yield management and convergence analysis Machine Learning 55 1 2004 5 29
17. A. Gosavi Reinforcement learning for long-run average cost European Journal of Operations Research 155 3 2004 654 674
18. A. Gosavi, N. Bandla, and T.K. Das A reinforcement learning approach to a single leg airline revenue management problem with multiple fare classes and overbooking IIE Transactions 34 9 2002 729 742
19. N. Hall, and M. Magazine Maximizing the value of a space mission European Journal of Operational Research 78 2 1994 224 241
20. M.M. Hing, A. van Harten, and P.C. Schuur Reinforcement learning versus heuristics for order acceptance on a single resource Journal of Heuristic 13 2 2007 167 187
21. W. Hopp, and M. Spearman Factory Physics 2000 McGraw-Hill, Inc. Columbus, OH
22. L.P. Kaelbling, M.L. Littman, and A.P. Moore Reinforcement learning: a survey Journal of Artificial Intelligence Research 4 1996 237 285
23. P. Keskinocak, and S. Tayur Due date management policies S. Simchi-Levi, S.D. Wu, Z.M. Shen, Handbook of Quantitative Supply Chain Analysis: Modeling in the E-business Era 2004 Kluwer Academic Publishers Norwell, MA 485 553
24. P. Keskinocak, R. Ravi, and S. Tayur Scheduling and reliable lead time quotation for orders with availability intervals and lead time sensitive revenues Management Science 47 2 2001 264 279
25. D. Kim, and W.J. Lee Optimal joint pricing and lot sizing with fixed and variable capacity European Journal of Operational Research 109 1 1998 212 227
26. A.M. Law, and W.D. Kelton Simulation Modeling and Analysis 1991 McGraw-Hill, Inc. New York, NY
27. S.R. Lawrence Estimating flowtimes and setting due-dates in complex production systems IIE Transactions 27 5 1995 657 688
28. S. Leibs Ford heeds the profits CFO Magazine 16 9 2000 33 35
29. S. Lippman, and S. Ross The streetwalkers dilemma: a job shop model SIAM Journal of Applied Mathematics 20 3 1971 336 342
30. S. Mahadevan Average reward reinforcement learning: foundations, algorithms and empirical results Machine Learning 22 1 1996 159 195
31. S. Mahadevan, and G. Theocharous Optimizing production manufacturing using reinforcement learning Proceedings of the Eleventh International Florida Artificial Intelligence Research Society Conference 1998 AAAI Press 372 377
32. S. Mahadevan, N. Khaleeli, and N. Marchalleck Designing agent controllers using discrete-event markov models Proceeding of the AAAI Fall Symposium on Model-Directed Autonomous Systems 1997 MIT Cambridge
33. S. Mahadevan, N. Marchalleck, T. Das, and A. Gosavi Self-improving factory simulation using continuous-time average-reward reinforcement learning Proceedings of the Fourth International Machine Learning Conference 1997 Morgan Kaufmann San Mateo, CA 202 210
34. B. Miller A queueing reward system with several customer classes Management Science 16 3 1969 234 245
35. J. Nicholas Competitive Manufacturing Management: Montinuous Improvement, Lean Production, Customer-Focused Quality 1998 McGraw-Hill, Inc. Newyork, NY
36. K. Palaka, S. Erlebacher, and D.H. Kropp Lead-time setting, capacity utilization and pricing decision under lead-time dependent demand IIE Transactions 30 2 1998 151 163
37. C.D. Paternina-Arboleda, and T.K. Das Intelligent dynamic control of single-product serial production lines IIE Transaction 33 1 2001 65 77
38. D. Pekelman Simultaneous price-production decisions Operations Research 22 4 1974 788 794
39. P. Philipoom, L. Rees, and L. Wiegmann Using neural networks to determine internally-set due-date assinments for shop scheduling Decision Science 25 5-6 1994 825 851
40. J. Qiu, and R. Loulou Multiproduct production/inventory control under random demands IEEE Transactions on Automatic Control 40 2 1995 350 356
41. Rao, U.S.; Swaminathan, J.M.; Zhang, J.; 2000. Integrated demand and production management in a periodic, make-to-order setting with uniform guaranteed lead time and outsourcing. Working Paper, GSIA, Carnegie Mellon University, Pittsburgh, September.
42. K.C. So Price and time competition for service delivery, manufacturing and service Operations Management 2 4 2000 392 409
43. K.C. So, and J.-S. Song Price, delivery time guarantees and capacity selection European Journal of Operational Research 111 1 1998 28 49
44. T. Spengler, S. Rehkopf, and T. Volling Revenue management in make-to-order manufacturing - an application to the iron and steel industry OR Spectrum 29 1 2007 158 171
45. G.J. Stalk, and T.M. Hout Competing Against Time 1990 The Free Press
46. R.L. Sutton Learning to predict by the method of temporal differences Machine Learning 3 1 1988 9 44
47. Szepesvari, C.; Littman, M.L.; 1996. Generalized markov decision processes: dynamic-programming and reinforcement-learning algorithms. Tech. Rep.; Brown University, Providence, RI.
48. Vanthienen, L.G.; 1975. Simultaneous price-repoduction decision making with production adjustment costs. In: TIMS XX International Meeting. pp. 249-254.
49. Watkins, C.; 1989. Learning from delayed rewards. Ph.D. Thesis, University of Cambridge, England.
50. S. Webster Dynamic pricing and lead-time policies Decision Science 33 4 2002 579 599
51. T. Whitin Inventory control and price theory Management Science 2 1 1955 61 68
52. B. Widrow, and M.E.J. Hoff Adaptive switching circuits IRE WESCON Convention Record 4 1960 96 104
53. A. Wu, and D. Chiang The impact of estimation error on the dynamic order admission policy in B2B MTO environments Expert Systems with Applications 36 9 2009 11782 11791
54. C.A. Yano, and S.M. Gilbert Coordinated pricing and production/ procurement decisions: a review A. Chakravarty, J. Eliashberg, Managing Business Interfaces: Marketing Engineering and Manufacturing Perspectives 2004 Kluwer Academic Publishers Boston, MA
55. Zeng, D.; Sycara, K.; 1995. Using case-based reasoning as a reinforcement learning framework for optimization with changing criteria. In: Proceedings of the 7th International Conference on Tools with Artificial Intelligence, Takamatsu, Japan. pp. 56-62.