Structured evolution with compact architectures for scalable policy optimization

35th International Conference on Machine Learning, ICML 2018

Volumn 3, Issue , 2018, Pages 1562-1577

  Choromanski, Krzysztof ^a   Rowland, Mark ^b   Sindhwani, Vikas ^a   Tiirncr, Richard E ^b   Weller, Adrian ^b,c  

^a GOOGLE INC   (United States)

^b UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^c ALAN TURING INSTITUTE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; INFERENCE ENGINES; LEARNING SYSTEMS; MATRIX ALGEBRA; NETWORK ARCHITECTURE; RECREATIONAL FACILITIES;

BLACK-BOX OPTIMIZATION; COMPACT ARCHITECTURE; DERIVATIVEFREE OPTIMIZATIONS (DFO); GRADIENT APPROXIMATION; HIGH DIMENSIONAL SPACES; LINEAR TIME COMPLEXITY; SCALABLE ARCHITECTURES; THEORETICAL GUARANTEES;

OPTIMIZATION;

EID: 85057278161     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. Andoni, Alexandr, Indyk, Piotr, Laarhoven, Thijs, Razenshteyn, Ilya, and Schmidt, Ludwig. Practical and optimal LSH for angular distance. In Advances in Neural Information Processing Systems (NIPS). 2015.
2. Avron, Haim, Sindhwani, Vikas, Yang, Jiyan, and Mahoney, Michael W. Quasi-Monte Carlo feature maps for shiftinvariant kernels. The Journal of Machine Learning Research, 17(1):4096-4133, 2016.
3. Brockman, Greg, Cheung, Vicki, Pettersson, Ludwig, Schneider, Jonas, Schulman, John, Tang, Jie, and Zaremba, Wojciech. OpenAI Gym, 2016.
4. Caflisch, Russel E. Monte Carlo and quasi-Monte Carlo methods. Acta numerica, 7:1-49, 1998.
5. Choromanski, Krzysztof M., Rowland, Mark, and Weller, Adrian. The unreasonable effectiveness of structured random orthogonal embeddings. In Advances in Neural Information Processing Systems (NIPS). 2017.
6. Conn, Andrew R, Scheinberg, Katya, and Vicente, Luis N. Introduction to derivative-free optimization, volume 8. Siam, 2009.
7. Dick, Josef and Pillichshammer, Friedrich. Digital Nets and Sequences: Discrepancy Theory and Quasi-Monte Carlo Integration. Cambridge University Press, New York, NY, USA, 2010.
8. Han, Song, Mao, Huizi, and Dally, William J. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint:1510.00149, 2015.
9. Hansen, Nikolaus, Muller, Sibylle D., and Koumoutsakos, Petros. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es). Evol. Comput., 11(1):1-18, March 2003.
10. Hochreiter, Sepp and Schmidhuber, Jurgen. Flat minima. Neural Computation, 9(1):1-42, 1997.
11. Jamieson, Kevin G, Nowak, Robert, and Recht, Ben. Query complexity of derivative-free optimization. In Advances in Neural Information Processing Systems (NIPS), 2012.
12. Lehman, Joel, Chen, Jay, Clune, Jeff, and Stanley, Kenneth O. ES is more than just a traditional finite difference approximator. arXiv preprint:1712.06568, 2017.
13. Lillicrap, Timothy P, Hunt, Jonathan J, Pritzel, Alexander, Heess, Nicolas, Erez, Tom, Tassa, Yuval, Silver, David, and Wierstra, Daan. Continuous control with deep reinforcement learning. arXiv preprint:1509.02971, 2015.
14. More, Jorge J. and Wild, Stefan M. Benchmarking derivative-free optimization algorithms. SIAM Journal on Optimization, 20(1):172-191, 2009.
15. Nesterov, Yurii and Spokoiny, Vladimir. Random gradientfree minimization of convex functions. Found. Comput. Math., 17(2):527-566, April 2017. ISSN 1615-3375.
16. Sainath, Tara N, Kingsbury, Brian, Sindhwani, Vikas, Arisoy, Ebru, and Ramabhadran, Bhuvana. Low-rank matrix factorization for deep neural network training with high-dimensional output targets. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013.
17. Salimans, Tim, Ho, Jonathan, Chen, Xi, Sidor, Szymon, and Sutskever, Ilya. Evolution strategies as a scalable alternative to reinforcement learning. 2017.
18. Schulman, John, Levine, Sergey, Abbeel, Pieter, Jordan, Michael, and Moritz, Philipp. Trust region policy optimization. In International Conference on Machine Learning (ICML), 2015.
19. Schulman, John, Wolski, Filip, Dhariwal, Prafulla, Radford, Alec, and Klimov, Oleg. Proximal policy optimization algorithms. arXiv preprint:1707.06347, 2017.
20. Sindhwani, Vikas, Sainath, Tara, and Kumar, Sanjiv. Structured transforms for small-footprint deep learning. In Advances in Neural Information Processing Systems (NIPS), 2015.
21. Staines, Joe and Barber, David. Variational optimization. 2012.
22. Williams, Ronald J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229-256, 1992.
23. Yu, Felix X, Suresh, Ananda Theertha, Choromanski, Krzysztof M, Holtmann-Rice, Daniel N, and Kumar, Sanjiv. Orthogonal random features. In Advances in Neural Information Processing Systems (NIPS). 2016.