Online Learning of Feasible Strategies in Unknown Environments

IEEE Transactions on Automatic Control

Volumn 62, Issue 6, 2017, Pages 2807-2822

  Paternain, Santiago ^a   Ribeiro, Alejandro ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Offline; primal dual methods

Indexed keywords

CONTROLLERS; COST FUNCTIONS; E-LEARNING;

CONSTRAINT VIOLATION; CONVEX CONSTRAINTS; COST DIFFERENCES; NUMERICAL EXPERIMENTS; OFFLINE; ONLINE LEARNING; OPTIMAL ACTIONS; PRIMAL-DUAL METHODS;

COSTS;

EID: 85028821757     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2016.2627401     Document Type: Article

References (40)

1. K. J. Arrow and L. Hurwicz, Studies in Linear and Nonlinear Programming. CA: Stanford University Press, 1958.
2. M. W. Hirsch, S. Smale, and R. L. Devaney, Differential Equations, Dynamical Systems, and An Introduction To Chaos, vol. 60. Academic Press, 2004.
3. M. Krstíc and H.-H. Wang, "Stability of extremum seeking feedback for general nonlinear dynamic systems, " Automatica, vol. 36, no. 4, pp. 595-601, 2000.
4. K. B. Ariyur andM. Krstic, Real-TimeOptimization By Extremum-Seeking Control. Wiley, 2003.
5. Y. Tan, D. Nesíc, and I. Mareels, "On non-local stability properties of extremum seeking control, " Automatica, vol. 42, no. 6, pp. 889-903, 2006.
6. A. Nedíc and A. Ozdaglar, "Subgradient methods for saddle-point problems, " J. Optim. Theory Appl., vol. 142, no. 1, pp. 205-228, 2009.
7. H. Uzawa, "Iterative methods for concave programming, " Studies in Linear and Nonlinear Programming, vol. 6, 1958.
8. D. Maistroskii, "Gradient methods for finding saddle points, " Matekon, vol. 14, no. 1, pp. 3-22, 1977.
9. D. Feijer and F. Paganini, "Stability of primal-dual gradient dynamics and applications to network optimization, " Automatica, vol. 46, no. 12, pp. 1974-1981, 2010.
10. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University press, 2004.
11. N. Atanasov, J. Le Ny, N. Michael, and G. J. Pappas, "Stochastic source seeking in complex environments, " in Proc. IEEE Int. Conf. Robotics Automation (ICRA), pp. 3013-3018, IEEE, 2012.
12. S.-I. Azuma, M. S. Sakar, and G. J. Pappas, "Stochastic source seeking by mobile robots, " IEEE Trans. Automatic Control, vol. 57, no. 9, pp. 2308-2321, 2012.
13. S.-J. Liu and M. Krstic, "Stochastic source seeking for nonholonomic unicycle, " Automatica, vol. 46, no. 9, pp. 1443-1453, 2010.
14. H. Robbins and S. Monro, "A stochastic approximation method, " Annals Mathematical Statistics, pp. 400-407, 1951.
15. M. Schmidt, N. L. Roux, and F. Bach, "Minimizing finite sums with the stochastic average gradient, " arXiv Preprint arXiv: 1309. 2388, 2013.
16. J. Konecnỳ and P. Richtárik, "Semi-stochastic gradient descent methods, " arXiv Preprint arXiv: 1312. 1666, 2013.
17. D. Blackwell, "An analog of the minimax theorem for vector payoffs, " Pacific J. Mathematics, vol. 6, no. 1, pp. 1-8, 1956.
18. V. Vapnik, The Nature of Statistical Learning Theory. Springer, 2000.
19. S. Shalev-Shwartz, "Online learning and online convex optimization, " Foundations and Trends in Machine Learning, vol. 4, no. 2, pp. 107-194, 2011.
20. M. Zinkevich, "Online convex programming and generalized infinitesimal gradient ascent, " in ICML, pp. 928-936, 2003.
21. E. Hazan, A. Agarwal, and S. Kale, "Logarithmic regret algorithms for online convex optimization, " Machine Learning, vol. 69, no. 2-3, pp. 169-192, 2007.
22. S. H. Low and D. E. Lapsley, "Optimization flow control-i: Basic algorithm and convergence, " IEEE/ACM Trans. Networking (TON), vol. 7, no. 6, pp. 861-874, 1999.
23. M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle, "Layering as optimization decomposition: A mathematical theory of network architectures, " Proc. IEEE, vol. 95, no. 1, pp. 255-312, 2007.
24. A. Chambolle and T. Pock, "A first-order primal-dual algorithm for convex problems with applications to imaging, " J. Math. Imaging Vision, vol. 40, no. 1, pp. 120-145, 2011.
25. M. Mahdavi, R. Jin, and T. Yang, "Trading regret for efficiency: Online convex optimization with long term constraints, " J. Machine Learning Research, vol. 13, no. Sep, pp. 2503-2528, 2012.
26. Y. Viossat and A. Zapechelnyuk, "No-regret dynamics and fictitious play, " J. Economic Theory, vol. 148, no. 2, pp. 825-842, 2013.
27. S. Sorin, "Exponential weight algorithm in continuous time, " Mathematical Programming, vol. 116, no. 1-2, pp. 513-528, 2009.
28. J. Kwon and P. Mertikopoulos, "A continuous-time approach to online optimization, " arXiv Preprint arXiv: 1401. 6956, 2014.
29. A. Y. Popkov, "Gradient methods for nonstationary unconstrained optimization problems, " Automat. Remote Control, vol. 66, no. 6, pp. 883-891, 2005.
30. M. Fazlyab, S. Paternain, V. M. Preciado, and A. Ribeiro, "Interior point method for dynamic constrained optimization in continuous time, " arXiv Preprint arXiv: 1510. 01396, 2015.
31. V. M. Zavala and M. Anitescu, "Real-time nonlinear optimization as a generalized equation, " SIAM J. Control Optim., vol. 48, no. 8, pp. 5444-5467, 2010.
32. V. S. Borkar, "Stochastic approximation, " Cambridge Books, 2008.
33. R. Kleinberg, A. Niculescu-Mizil, and Y. Sharma, "Regret bounds for sleeping experts and bandits, " Machine Learning, vol. 80, no. 2-3, pp. 245-272, 2010.
34. S. Kale, C. Lee, and D. Pál, "Hardness of online sleeping combinatorial optimization problems, " arXiv Preprint arXiv: 1509. 03600, 2015.
35. V. Kanade, H. B. McMahan, and B. Bryan, "Sleeping experts and bandits with stochastic action availability and adversarial rewards, " in Proc. Int. Conf. Artificial Intell. Statistics, pp. 272-279, 2009.
36. G. Neu and M. Valko, "Online combinatorial optimization with stochastic decision sets and adversarial losses, " in Adv. Neural Information Processing Syst., pp. 2780-2788, 2014.
37. D. Zhang and A. Nagurney, "On the stability of projected dynamical systems, " J. Optim. Theory Appl., vol. 85, pp. 97-124, Apr. 1995.
38. S. Hart and A. Mas-Colell, "A general class of adaptive strategies, " J. Economic Theory, vol. 98, no. 1, pp. 26-54, 2001.
39. H. P. Young, "The evolution of conventions, " Econometrica: J. Econometric Soc., pp. 57-84, 1993.
40. D. Mellinger and V. Kumar, "Minimum snap trajectory generation and control for quadrotors, " in Proc. IEEE Int. Conf. Robotics Autom. (ICRA), May 2011.