Quantum speedup for active learning agents

Physical Review X

Volumn 4, Issue 3, 2014, Pages

  Paparo, Giuseppe Davide ^a   Dunjko, Vedran ^b,c,d   Makmal, Adi ^b,c   Martin Delgado, Miguel Angel ^a   Briegel, Hans J ^b,c  

^a UNIVERSIDAD COMPLUTENSE DE MADRID   (Spain)

^b UNIVERSITY OF INNSBRUCK   (Austria)

^c ERICH SCHMID INSTITUTE OF MATERIALS SCIENCE   (Austria)

^d RUDER BO KOVI INSTITUTE   (Croatia)

Author keywords

Interdisciplinary physics; Quantum information; Quantum physics

Indexed keywords

INTELLIGENT AGENTS; QUANTUM OPTICS; QUANTUM THEORY;

INTELLIGENT BEHAVIOR; INTELLIGENT LEARNING; INTERDISCIPLINARY PHYSICS; QUADRATIC SPEEDUP; QUANTUM INFORMATION; QUANTUM PHYSICS; REALISTIC ENVIRONMENTS; TASK ENVIRONMENT;

ARTIFICIAL INTELLIGENCE;

EID: 84904537776     PISSN: None     EISSN: 21603308     Source Type: Journal    
DOI: 10.1103/PhysRevX.4.031002     Document Type: Article

References (62)

1. D. Deutsch, Quantum Theory, The Church-Turing Principle and the Universal Quantum Computer, Proc. R. Soc. A 400, 97 (1985)
2. D. Deutsch and R. Jozsa, Rapid Solution of Problems by Quantum Computation, Proc. R. Soc. A 439, 553 (1992)
3. L. K. Grover, Proceedings of the 28th Annual ACM Symposium on the Theory of Computing (STOC) (ACM, New York, 1996), pp. 212-219
4. P.W. Shor, in Proceedings of the 35th Symposium on Foundations of Computer Science, FOCS 1994 (IEEE, New York, 1994), p. 124-134
5. M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information (Cambridge University Press, Cambridge, England, 2000)
6. C. H. Bennett and D. P. DiVincenzo, Quantum Information and Computation, Nature (London) 404, 247 (2000)
7. M. Sasaki and A. Carlini, Quantum Learning and Universal Quantum Matching Machine, Phys. Rev. A 66, 022303 (2002)
8. H. Neven, V. S. Denchev, G. Rose, and W. G. Macready, Training a Binary Classifier with the Quantum Adiabatic Algorithm, arXiv:0811.0416
9. S. Lloyd, M. Mohseni, and P. Rebentrost, Quantum Algorithms for Supervised and Unsupervised Machine Learning, arXiv:1307.0411
10. D. Manzano, M. Pawłowski, and Č. Brukner, The Speed of Quantum and Classical Learning for Performing the kth Root of Not, New J. Phys. 11, 113018 (2009)
11. K. L. Pudenz and D. A. Lidar, Quantum Adiabatic Machine Learning, Quantum Inf. Process. 12, 2027 (2013)
12. E. Aïmeur, G. Brassard, and S. Gambs, Quantum Speed-up for Unsupervised Learning, Mach. Learn. 90, 261 (2013)
13. V. Braitenberg, Vehicles: Experiments in Synthetic Psychology (MIT Press, Cambridge, MA, 1986)
14. R. Brooks, Cambrian Intelligence: The Early History of the New AI (MIT Press, Cambridge, MA, 1999)
15. R. Pfeifer and C. Scheier, Understanding Intelligence (MIT Press, Cambridge, MA, 1999)
16. R. Pfeifer and J. C. Bongard, How the Body Shapes the Way We Think: A New View of Intelligence (MIT Press, Cambridge, MA, 2006)
17. D. Floreano and C. Mattiussi, Bio-inspired Artificial Intelligence: Theories, Methods, and Technologies (MIT Press, Cambridge, MA, 2008)
18. L.W. Barsalou, Grounded Cognition, Annu. Rev. Psychol. 59, 617 (2008)
19. H. J. Briegel and G. De las Cuevas, Projective Simulation for Artificial Intelligence, Sci. Rep. 2, 400 (2012)
20. S. J. Russel and P. Norvig, Artificial Intelligence-A Modern Approach (Prentice-Hall, New Jersey, 2003), 2nd ed
21. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA, 1998), 1st ed
22. AIBO, adapted from http://commons.wikimedia.org/wiki/ Category:Aibo#mediaviewer/File:Aibo_img_0807.jpg, created by Rama, and licensed under the CC-BY-SA-2.0-FR
23. M. Szegedy, in Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science, 2004 (IEEE Computer Society, Washington, DC, 2004), pp. 32-41
24. F. Magniez, A. Nayak, J. Roland, and M. Santha, Search via Quantum Walk, SIAM J. Comput. 40, 142 (2011)
25. Examples of such applications include the problems of navigation, or human-robot interaction. We note that the focus on such classical environments is by no means a restriction of our scheme. For a quantum environment, the actions that the agent can perform could, e.g., be quantum measurements (as components of his actuators), and percepts the measurement outcomes-such scenarios are certainly not without interest. The results we present in this work apply equally to such environments. However, we will not explore the applications of learning agents in quantum environments in this paper
26. Even if we were to allow superpositions of actions, the amount of control the agent must have over the degrees of freedom of the environment, in order to apply quantum query algorithms, may be prohibitive. This problem constitutes one of the fundamental operative distinctions between quantum algorithms, where full control is assumed, and quantum agents, where such control is limited
27. In embodied agents, these physical processes may, e.g., realize some internal representation of the environment, which the agent itself has to develop as it interacts with the environment. For example, in the context of artificial neural networks, such internal models are known as selforganizing maps and, more specifically, sensorimotor maps [58,59]
28. In the PS framework, one formally distinguishes between real percepts and actions s and a and their internal representations, denoted as μ{eth}s{Thorn} and μ{eth}a{Thorn}, which comprise clips. For our purposes, by abuse of notation, we will omit the mapping μ. For more details, see Appendix C
29. In the PS, the hopping probabilities themselves are encoded in the so-called h matrix, which is an unnormalized representation of the transition matrix [19]
30. Each agent is equipped with input and output couplers, which translate, through sensors and actuators (see Fig. 1), real percepts to the internal representations of percepts, and internal representations of actions to real actions
31. In this paper, we do not consider logarithmically contributing terms in the complexity analysis; thus, we use the Õ -level analysis of the limiting behavior (instead of the standard "big-O" O)
32. F. Magniez, M. Santha, and M. Szegedy, in Proceedings of the Sixteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2005 (Society for Industrial and Applied Mathematics, Philadelphia, 2005), p. 1109-1117
33. A. Ambainis, in Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science, October 17-19, 2004 (IEEE Computer Society Press, Los Alamitos, CA, 2004), p. 22-31
34. F. Magniez, A. Nayak, P. C. Richter, and M. Santha, On the Hitting Times of Quantum versus Random Walks, Algorithmica 63, 91 (2012)
35. The optimality claim holds provided that no additional mechanisms, except for diffusion and checking, are available to the agent
36. K. Temme, T. J. Osborne, K. G. H. Vollbrecht, D. Poulin, and F. Verstraete, Quantum Metropolis Sampling, Nature (London) 471, 87 (2011)
37. M.-H. Yung and A. Aspuru-Guzik, A Quantum-Quantum Metropolis Algorithm, Proc. Natl. Acad. Sci. U.S.A. 109, 754 (2012)
38. R. D. Somma, S. Boixo, H. Barnum, and E. Knill, Quantum Simulations of Classical Annealing Processes, Phys. Rev. Lett. 101, 130504 (2008)
39. P. Wocjan and A. Abeyesinghe, Speedup via Quantum Sampling, Phys. Rev. A 78, 042336 (2008)
40. *, so V can be constructed from U by conjugating it with the swap operator of the two registers. Here, we present the construction for the general case of irreducible, aperiodic Markov chains
41. Here, we note that concrete applications of quantum walks specified by nonsymmetric Markov chains have, to our knowledge and aside from this work, only been studied in Refs. [60,61], by two of the authors and other collaborators, in a significantly different context
42. s is very small
43. M. Boyer, G. Brassard, P. Høyer, and A. Tapp, Tight Bounds on Quantum Searching, Fortschr. Phys. 46, 493 (1998)
44. We note that some of the only logarithmically contributing terms, which appear in a more detailed analysis omitted here, can be further avoided using more complicated constructions, as has, for instance, been done in Ref. [24]
45. A. Aspuru-Guzik and P. Walther, Photonic Quantum Simulators, Nat. Phys. 8, 285 (2012)
46. R. Blatt and C. F. Roos, Quantum Simulations with Trapped Ions, Nat. Phys. 8, 277 (2012)
47. T. Schaetz, C. R. Monroe, and T. Esslinger, Focus on Quantum Simulation, New J. Phys. 15, 085009 (2013)
48. D. H. Wolpert, The Lack of A Priori Distinctions between Learning Algorithms, Neural Comput. 8, 1341 (1996)
49. S. Droste, T. Jansen, and I. Wegener, Optimization with Randomized Search Heuristics-The (a)nfl Theorem, Realistic Scenarios, and Difficult Functions, Theor. Comput. Sci. 287, 2002 (1997)
50. We acknowledge that the interpretation of these results in the sense of their practical impact on the field is not without controversy. Nonetheless, the validity of the mathematical statements is not contested. See Ref. [62] for more details
51. That is, a comparison can be made with no further assumptions on the environment beyond the trivial ones -that the percept and action sets are compatible with the environment and that the environment provides a rewarding scheme
52. A. Sinclair, Algorithms for Random Generation and Counting: A Markov Chain Approach (Springer, New York, 1993), Vol. 7
53. J. Mautner, A. Makmal, D. Manzano, M. Tiersch, and H. J. Briegel, arXiv:1305.1578. To appear in New Generation Computing (2014)
54. H. J. Briegel, On Creative Machines and the Physical Origins of Freedom, Sci. Rep. 2, 522 (2012)
55. F. Magniez, M. Santha, and M. Szegedy, Quantum Algorithms for the Triangle Problem, SIAM J. Comput. 37, 413 (2007)
56. E. Bernstein and U. Vazirani, in Proceedings of the 25th Annual ACM Symposium on Theory of Computing (ACM, New York, 1993), pp. 11-20
57. Additionally, the quality of the approximation can be, probabilistically, arbitrarily raised by one round of the phase-estimation algorithm for the walk operator, provided that no phase is detected. Should this process fail, the unsearching process is repeated
58. T. Kohonen, Self-Organizing Maps (Springer-Verlag, Berlin, 1995)
59. M. Toussaint, A Sensorimotor Map: Modulating Lateral Interactions for Anticipation and Planning, Neural Comput. 18, 1132 (2006)
60. G. D. Paparo and M. Martin-Delgado, Google in a Quantum Network, Sci. Rep. 2, 444 (2012)
61. G. Paparo, M. Mueller, F. Comellas, and M. Martin-Delgado, Quantum Google in a Complex Network, Sci. Rep. 3, 2773 (2013)
62. An introduction to discussions regarding the no free lunch theorems, and their interpretations, can be found at http:// no-free-lunch.org/