Meta-learning within Projective Simulation

IEEE Access

Volumn 4, Issue , 2016, Pages 2110-2122

  Makmal, Adi ^a   Melnikov, Alexey A ^a,b   Dunjko, Vedran ^a   Briegel, Hans J ^a  

^a UNIVERSITY OF INNSBRUCK   (Austria)

^b ERICH SCHMID INSTITUTE OF MATERIALS SCIENCE   (Austria)

Author keywords

adaptive algorithm; learning; Machine learning; meta learning; quantum mechanics; random processes; reinforcement learning

Indexed keywords

ADAPTIVE ALGORITHMS; ARTIFICIAL INTELLIGENCE; AUTONOMOUS AGENTS; ECONOMIC AND SOCIAL EFFECTS; LEARNING SYSTEMS; QUANTUM THEORY; RANDOM PROCESSES;

DEVELOPED MODEL; LEARNING; LEARNING PARAMETERS; METALEARNING; RANDOM WALK PROCESS; SIMULATION APPROACH; TASK ENVIRONMENT; UNIVERSAL APPROACH;

REINFORCEMENT LEARNING;

EID: 84979833309     PISSN: None     EISSN: 21693536     Source Type: Journal    
DOI: 10.1109/ACCESS.2016.2556579     Document Type: Article

References (49)

1. S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, 3rd ed. Englewood Cliffs, NJ, USA: Prentice-Hall, 2010.
2. P. Brazdil, C. G. Carrier, C. Soares, and R. Vilalta, Metalearning: Appli-cations to Data Mining. Berlin, Germany: Springer, 2009.
3. D. H.Wolpert andW. G. Macready, "No free lunch theorems for optimization," IEEE Trans. Evol. Comput., vol. 1, no. 1, pp. 67-82, Apr. 1997.
4. S.W.Wilson, "Classifier fitness based on accuracy," Evol. Comput., vol. 3, no. 2, pp. 149-175, 1995.
5. T. Schaul and J. Schmidhuber, "Metalearning," Scholarpedia, vol. 5, no. 6, p. 4650, 2010.
6. C. Giraud-Carrier, R. Vilalta, and P. Brazdil, "Introduction to the special issue on meta-learning," Mach. Learn., vol. 54, no. 3, pp. 187-193, Mar. 2004.
7. S. Thrun and L. Pratt, Eds., Learning to Learn. New York, NY, USA: Springer, 1998.
8. W. Duch and K. Grudziski, "Meta-learning via search combined with parameter optimization," in Intelligent Information Systems (Advances in Soft Computing), vol. 17, M. Kaopotek, S. Wierzcho, and M. Michalewicz, Eds. Heidelberg, Germany: Physica-Verlag HD, 2002, pp. 13-22.
9. P. B. Brazdil, C. Soares, and J. P. da Costa, "Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results," Mach. Learn., vol. 50, no. 3, pp. 251-277, Mar. 2003.
10. P. Zhao and B. Yu, "On model selection consistency of lasso," J. Mach. Learn. Res., vol. 7, pp. 2541-2563, Nov. 2006.
11. M. M. Adankon and M. Cheriet, "Model selection for the LS-SVM. Application to handwriting recognition," Pattern. Recognit., vol. 42, no. 12, pp. 3264-3270, Dec. 2009.
12. S. M. Abdulrahman, P. Brazdil, J. N. van Rijn, and J. Vanschoren, "Algorithm selection via meta-learning and sample-based active testing," in Proc. Meta-Learn. Algorithm Selection Workshop ECMLPKDD, 2015, pp. 55-66.
13. L. Todorovski and S. Deroski, "Combining classifiers with meta decision trees," Mach. Learn., vol. 50, no. 3, pp. 223-249, Mar. 2003.
14. S. Ishii, W. Yoshida, and J. Yoshimoto, "Control of exploitation-exploration meta-parameter in reinforcement learning," Neural Netw., vol. 15, nos. 4-6, pp. 665-687, Jun./Jul. 2002.
15. N. Schweighofer and D. Doya, "Meta-learning in reinforcement learning," Neural Netw., vol. 16, no. 1, pp. 5-9, Jan. 2003.
16. A. Eriksson, G. Capi, and K. Doya, "Evolution of meta-parameters in reinforcement learning algorithm," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., vol. 1. Oct. 2003, pp. 412-417.
17. K. Kobayashi, H. Mizoue, T. Kuremoto, and M. Obayashi, "A metalearning method based on temporal difference error," in Neural Infor-mation Processing (Lecture Notes in Computer Science), vol. 5863, C. Leung, M. Lee, and J. H. Chan, Eds. Berlin, Germany: Springer, 2009, pp. 530-537.
18. M. Tokic, F. Schwenker, and G. Palm, "Meta-learning of exploration and exploitation parameters with replacing eligibility traces," in Partially Supervised Learning (Lecture Notes in Computer Science), vol. 8183, Z.-H. Zhou and F. Schwenker, Eds. Berlin, Germany: Springer, 2013, pp. 68-79.
19. Y. Bengio, "Gradient-based optimization of hyperparameters," Neural Comput., vol. 12, no. 8, pp. 1889-1900, 2000.
20. R. Bardenet and B. Kégl, "Surrogating the surrogate: Accelerating Gaussian-process-based global optimization with a mixture cross-entropy algorithm," in Proc. 27th Int. Conf. Mach. Learn., 2010, pp. 55-62.
21. J. Bergstra and Y. Bengio, "Random search for hyper-parameter optimization," J. Mach. Learn. Res., vol. 13, pp. 281-305, Feb. 2012.
22. M. Reif, F. Shafait, and A. Dengel, "Meta-learning for evolutionary parameter optimization of classifiers," Mach. Learn., vol. 87, no. 3, pp. 357-380, Jun. 2012.
23. C. Thornton, F. Hutter, H. H. Hoos, and K. Leyton-Brown, "Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms," in Proc. 19th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2013, pp. 847-855.
24. M. R. Smith, L. Mitchell, C. Giraud-Carrier, and T. Martinez, "Recommending learning algorithms and their associated hyperparameters," in Proc. Meta-Learn. Algorithm Selection Workshop ECAI, 2014, pp. 39-40.
25. M. Feurer, J. T. Springenberg, and F. Hutter, "Initializing Bayesian hyperparameter optimization via meta-learning," in Proc. 29th AAAI Conf. Artif. Intell., 2015, pp. 1128-1135.
26. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press, 1998.
27. Y. Achbany, F. Fouss, L. Yen, A. Pirotte, and M. Saerens, "Tuning continual exploration in reinforcement learning: An optimality property of the Boltzmann strategy," Neurocomputing, vol. 71, nos. 13-15, pp. 2507-2520, Aug. 2008.
28. J. Schmidhuber, "Completely self-referential optimal reinforcement learners," in Artificial Neural Networks: Formal Models and Their Applica-tions (Lecture Notes in Computer Science), vol. 3697. Berlin, Germany: Springer, 2005, pp. 223-233.
29. H. J. Briegel and G. De las Cuevas, "Projective simulation for artificial intelligence," Sci. Rep., vol. 2, p. 400, Mar. 2012.
30. J. Mautner, A. Makmal, D. Manzano, M. Tiersch, and H. J. Briegel, "Projective simulation for classical learning agents: A comprehensive investigation," New Generat. Comput., vol. 33, no. 1, pp. 69-114, Jan. 2015.
31. A. A. Melnikov, A. Makmal, and H. J. Briegel, "Projective simulation applied to the grid-world and the mountain-car problem," Artif. Intell. Res., vol. 3, no. 3, pp. 24-34, 2014. [Online]. Available: http://arxiv.org/abs/1405.5459
32. Ø. F. Bjerland, "Projective simulation compared to reinforcement learning," M.S. thesis, Dept. Comput. Sci., Univ. Bergen, Bergen, Norway, 2015.
33. A. A. Melnikov, A. Makmal, V. Dunjko, and H. J. Briegel. (2015). "Projective simulation with generalization." [Online]. Available: http://arxiv.org/abs/1504.02247
34. S. Hangl, E. Ugur, S. Szedmak, and J. Piater. (2016). "Hierarchical haptic manipulation for complex skill learning." [Online]. Available: http://arxiv.org/abs/1603.00794
35. R. Pfeifer and C. Scheier, Understanding Intelligence. Cambridge, MA, USA: MIT Press, 1999.
36. R. Motwani and P. Raghavan, Randomized Algorithms. New York, NY, USA: Cambridge Univ. Press, 1995, ch. 6.
37. R. P. Feynman and A. R. Hibbs, Quantam Mechanics and Path Integrals (International Series in Pure and Applied Physics). New York, NY, USA: McGraw-Hill, 1965.
38. Y. Aharonov, L. Davidovich, and N. Zagury, "Quantum random walks," Phys. Rev. A, vol. 48, pp. 1687-1690, Aug. 1993.
39. D. Aharonov, A. Ambainis, J. Kempe, and U. Vazirani, "Quantum walks on graphs," in Proc. 33rd Annu. ACM Symp. Theory Comput. (STOC), New York, NY, USA, 2001, pp. 50-59.
40. A. M. Childs, R. Cleve, E. Deotto, E. Farhi, S. Gutmann, and D. A. Spielman, "Exponential algorithmic speedup by a quantum walk," in Proc. 35th Annu. ACM Symp. Theory Comput. (STOC), New York, NY, USA, 2003, pp. 59-68.
41. J. Kempe, "Discrete quantum walks hit exponentially faster," Probab. Theory Rel., vol. 133, no. 2, pp. 215-235, Oct. 2005.
42. H. Krovi, F. Magniez, M. Ozols, and J. Roland, "Quantum walks canfind a marked element on any graph," Algorithmica, vol. 74, no. 2, pp. 851-907, Feb. 2015.
43. G. D. Paparo, V. Dunjko, A. Makmal, M. A. Martin-Delgado, and H. J. Briegel, "Quantum speedup for active learning agents," Phys. Rev. X, vol. 4, p. 031002, Jul. 2014.
44. V. Dunjko, N. Friis, and H. J. Briegel, "Quantum-enhanced deliberation of learning agents using trapped ions," New J. Phys., vol. 17, no. 2, p. 023006, Feb. 2015.
45. N. Friis, A. A. Melnikov, G. Kirchmair, and H. J. Briegel, "Coherent controlization using superconducting qubits," Sci. Rep., vol. 5, p. 18036, Aug. 2015.
46. M. L. Anderson and T. Oates, "A review of recent research in metareasoning and metalearning," AI Mag., vol. 28, no. 1, pp. 7-16, 2007.
47. G. A. Rummery and M. Niranjan, "On-line Q-learning using connectionist systems," Dept. Eng., Univ. Cambridge, Cambridge, U.K., Tech. Rep. CUED/F-INFENG/TR 166, 1994.
48. C.-C. Wang, S. R. Kulkarni, and H. V. Poor, "Bandit problems with side observations," IEEE Trans. Autom. Control, vol. 50, no. 3, pp. 338-355, Mar. 2005.
49. R. S. Sutton, "Integrated architectures for learning, planning, and reacting based on approximating dynamic programming," in Proc. 7th Int. Conf. Mach. Learn., 1990, pp. 216-224.