Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews

Volumn 36, Issue 1, 2006, Pages 92-106

  Raju Chinthalapati, V L ^a   Yadati, Narahari ^b   Karumanchi, Ravikumar ^c,d  

^a LONDON SCHOOL OF ECONOMICS   (United Kingdom)

^b INDIAN INSTITUTE OF SCIENCE   (India)

^c IBM INDIA RESEARCH LAB   (India)

^d GENERAL MOTORS CORPORATION   (United States)

Author keywords

Dynamic pricing; Inventory replenishments; Markovian game; Multi agent learning; Online retail markets; Price sensitive customers; Reinforcement learning (RL); Stochastic demands

Indexed keywords

ELECTRONIC COMMERCE; GAME THEORY; INVENTORY CONTROL; MARKETING; MARKOV PROCESSES; MULTI AGENT SYSTEMS; SALES;

DYNAMIC PRICING; INVENTORY REPLENISHMENTS; MARKOVIAN GAME; MULTI-AGENT LEARNING; ONLINE RETAIL MARKETS; REINFORCEMENT LEARNING (RL); STOCHASTIC DEMANDS;

LEARNING SYSTEMS;

EID: 33644903192     PISSN: 10946977     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCC.2005.860578     Document Type: Article

References (44)

1. G. A. J. O. Kephart and G. J. Tesauro, "Strategic pricebot dynamics," in Proc. 1st ACM Conf. Electronic Commerce EC-99, Denver, Colorado, USA, 1999.
2. V. S. Borkar, "Reinforcement learning in Markovian evolutionary games," Adv. Complex Syst., vol. 5, pp. 55-72, Denver, Colorado, USA, 2002.
3. C. Brooks, R. Fay, R. Das, J. K. MacKie-Mason, J. Kephart, and E. Durfee, "Automated strategy searches in an electronic goods market: learning and complex price schedules," in Proc. 1st ACM Conf. Electronic Commerce EC-99, 1999, pp. 31-40.
4. A. Carvalho and M. Puterman, "Dynamic pricing and reinforcement learning," Department of Statistics, University of British Columbia, Vancouver, Canada, Tech. Rep., 2003.
5. L. Chan, Z. J. M. Shen, D. Simchi-Levi, and J. Swann, "Review of dynamic and online pricing research to improve supply chain performance," Handbook on Supply Chain Analysis in the eBusiness Era, Kluwer, Norwell, MA, 2001.
6. L. Chan, D. Simchi-Levi, and J. Swann, "Dynamic pricing strategies for manufacturing with stochastic demand and discretionary sales," Industrial Engineering and Operations Research, Northwestern University, Evanston, IL, USA, Tech. Rep., 2002.
7. P. Dasgupta and R. Das, "Dynamic pricing with limited competitor information in a multi-agent economy," presented at the Conf. Cooperative Information Systems, pp. 299-310, 2000. [Online]. Available: citeseer.nj.nec.com/dasgupta01dynamic.html
8. J. DiMicco, A. Greenwald, and P. Maes, "Dynamic pricing strategies under a finite time horizon," in Proc. 3rd ACM Conf. Electronic Commerce EC-01, 2001, pp. 51-60.
9. J. M. DiMicco, A. Greenwald, and P. Maes, Learning Curve: A Simulation-Based Approach to Dynamic Pricing, 2002, [Online]. Available: citeseer.nj.nec.com/563289.html [Online].
10. W. Elmaghraby and P. Keskinocak, "Dynamic pricing: Research overview, current practices and future directions," Management Science, vol. 49, no. 10, pp. 1287-1309.
11. A. Federgruen, "On n-person stochastic games with denumerable state space," Adv. Appl. Probability, vol. 10, pp. 452-471, 1978.
12. A. Federgruen and A. Heching, "Combined pricing and inventory control under uncertainty," Oper. Res., vol. 47, pp. 454-475, 1999.
13. G. Gallego and G. van Ryzin, "Optimal dynamic pricing of inventories with stochastic demand over finite horizons," Manage. Sci., vol. 40, no. 8, pp. 999-1020, 1994.
14. M. Gupta, K. Ravikumar, and M. Kumar, "Adaptive strategies for price markdown in a multi-unit descending price auction: A comparative study," in Proc. IEEE Conf. Systems., Man, and Cybernetics, 2002, pp. 373-378.
15. S. Hart and A. Mas-Colell, "Uncoupled dynamics do not lead to Nash equilibrium," Amer. Econ. Rev., vol. 93, pp. 1830-1836, 2003.
16. P. Hong, R. P. McAfee, and A. Nayyar, "Equilibrium price dispersion with consumer inventories," J. Econ. Theory, vol. 22, pp. 1-15, 2001.
17. J. Hu and M. P. Wellman, "Multiagent reinforcement learning: theoretical framework and an algorithm," in Proc. 15th Int. Conf. Machine Learning, San Francisco, CA, 1998, pp. 242-250, [Online]. Available:, citeseer.nj.nec.com/hu98multiagent.html
18. J. Hu and Y. Zhang, "Online reinformcenet learning in multiagent systems," William E. Simon Graduate School of Business Administration, University of Rochester, Rochester, NY, USA, Tech. Rep., 2002.
19. S. Karlin and C. Carr, "Prices and optimal inventory policies," Studies in Applied Probability Manage. Sci., pp. 159-172, 1962.
20. J. O. Kephart and G. J. Tesauro, "Pseudo-convergent Q-learning by competitive pricebots," in Proc. 17th Int. Conf. Machine Learning, San Francisco, CA, 2000. pp. 463-470. [Online]. Available:, citeseer.nj.nec.com/306829.html
21. V. R. Konda and V. S. Borkar, "Actor-critic type learning algorithms for markov decision processes," SIAM J. Control Optim., vol. 38, pp. 94-123, 1999.
22. A. Lau and H. Lau, "The newsboy problem with price-dependent demand distribution," IIE Trans., vol. 20, pp. 168-175, 1988.
23. R. Lawrence, "A machine-learning approach to optimal bid pricing," IBM, Research Report, Jul. 2002.
24. D. Leslie and E. Collins, "Convergent multiple-timescales reinforcement learning algorithms in normal form games," Ann. Appl. Probability, vol. 13, pp. 1231-1251, 2003.
25. M. L. Littman, "Markov games as a framework for multi-agent reinforcement learning," in Proc. 11th Int. Conf. Machine Learning, San Francisco, CA, USA, 1994, pp. 157-163.
26. M. L. Littman, "Friend-or-foe q-learning in general-sum games," in Proc. 18th Int. Conf. Machine Learning, Williamstown, MA, USA, 2001, pp. 322-328.
27. J. Mcgill and G. van Ryzin, "Revenue management: Research overview and prospects," Transport. Sci., vol. 33, no. 2, pp. 233-256, 1999.
28. K. Ravikumar, G. Batra, and R. Saluja, "Multi-agent learning for dynamic pricing games of service markets," Communicated, 2002.
29. S. Salop and J. Stiglitz, "The theory of sales: A simple model of equilibrium price dispersion with identical agents," Amer. Econ. Rev., vol. 72, no. 5, pp. 1121-1130, 1982.
30. C. Shapiro and H. Varian, Information Rules. Cambridge, MA: HER Press, 1998.
31. B. Smith, D. Gunther, B. Rao, and R. Ratliff, "E-commerce and operations research in airline planning, marketing, and distribution," Interfaces, vol. 31, no. 2, 2001.
32. M. Smith, J. Bailey, and E. Brynjolfsson, Understanding Digital Markets: Review and Assessment. Cambridge, MA: MIT Press, 2000.
33. M. Sridharan and G. J. Tesauro, "Multi-agent q-learning and regression trees for automated pricing decisions," in Proc. 17th Int. Conf. Machine Learning, San Francisco, CA, 2000.
34. G. Stigler, "The economics of information," J. Political Econ., vol. 69, pp. 213-225, 1961.
35. J. Stiglitz, "Equilibrium in product markets with imperfect information," Amer. Econ. Rev. Proc., vol. 69, pp. 339-345, 1979.
36. J. Swann, "Dynamic pricing models to improve supply chain performance," Ph.D. Dissertation, IEMS, Northwestern University, Evanston, IL, USA, 2001.
37. J. Swann, "Flexible pricing policies: Introduction and a survey of implementation in various industries," General Motors Corporation, Contract Report # CR-99/04/ESL, Oct. 1999.
38. G. Tesauro and J. O. Kephart, "Pricing in agent economies using multiagent q-learning," in Proc. Workshop on Decision Theoretic and Game Theoretic Agents, London, U.K., 1999.
39. G. Tesauro and J. O. Kephart, "Pricing in agent economies using neural networks and multi-agent q-learning," in Proc. Workshop ABS-3: Learning About, From and with other Agents (Held in Conjunction with IJCAI'99), Stockholm, Sweden, 1999.
40. G. Thowsen, "A dynamic nonstationary inventory problem for a price/ quantity setting firm," Navel Res. Logistics Quart., vol. 22, pp. 461-476, 1975.
41. H. R. Varian, "A model of sales," Amer. Econ. Rev., pp. 651-659, 1980.
42. H. R. Varian, "Differential pricing and efficiency," First Monday, vol. 1, 1996. [Online]. Available: www.firstmonday.dk.
43. C. J. C. H. Watkins and P. Dayan, "Q-learning,"Machine Learning, vol. 8, pp. 279-292, 1992.
44. E. Zabel, "Monopoly and uncertainty," Rev. Econ. Studies, vol. 37, pp. 205-219, 1970.