Interactive machine learning for health informatics: when do we need the human-in-the-loop?

Brain Informatics

Volumn 3, Issue 2, 2016, Pages 119-131

  Holzinger, Andreas ^a,b  

^a MEDICAL UNIVERSITY OF GRAZ   (Austria)

^b GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

Author keywords

Health informatics; Interactive machine learning

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLUSTERING ALGORITHMS; COMPUTATIONAL COMPLEXITY; HEALTH; HEURISTIC ALGORITHMS; REINFORCEMENT LEARNING; SPEECH RECOGNITION;

AUTOMATIC APPROACHES; AUTOMATIC MACHINES; AUTONOMOUS VEHICLES; HEALTH INFORMATICS; HEURISTIC SELECTIONS; INTERACTIVE MACHINE LEARNING; PREFERENCE LEARNING; SUB-SPACE CLUSTERING;

BIG DATA;

ACTIVE LEARNING; ARTICLE; ARTIFICIAL INTELLIGENCE; BAYESIAN LEARNING; HUMAN; INTERACTIVE MACHINE LEARNING; LEARNING ALGORITHM; MACHINE LEARNING; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; MEDICAL INFORMATICS; METHODOLOGY; OBJECT MANIPULATION; PREFERENCE LEARNING; PRIORITY JOURNAL; REINFORCEMENT LEARNING; SUPPORT VECTOR MACHINE;

EID: 85048560245     PISSN: 21984018     EISSN: 21984026     Source Type: Journal    
DOI: 10.1007/s40708-016-0042-6     Document Type: Article

References (98)

1. Samuel AL (1959) Some studies in machine learning using the game of checkers. IBM J Res Dev 3(3):210–229
2. Jordan MI, Mitchell TM (2015) Machine learning: trends, perspectives, and prospects. Science 349(6245):255–260
3. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
4. Bayes T (1763) An essay towards solving a problem in the doctrine of chances (posthumous communicated by Richard Price). Philos Trans 53:370–418
5. Barnard GA, Bayes T (1958) Studies in the history of probability and statistics: IX. Thomas Bayes’s essay towards solving a problem in the doctrine of chances. Biometrika 45(3/4):293–315
6. Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference, and prediction, 2nd edn. Springer, New York
7. Murphy KP (2012) Machine learning: a probabilistic perspective. MIT press, Cambridge
8. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489
9. Zhong N, Liu JM, Yao YY, Wu JL, Lu SF, Qin YL, Li KC, Wah B (2007) Web intelligence meets brain informatics. In: Zhong N, Liu JM, Yao YY, Wu JL, Lu SF, Li KC (eds) Web intelligence meets brain informatics., Lecture Notes in Artificial Intellience 4845Springer, Berlin, pp 1–31
10. Holzinger A (2014) Trends in interactive knowledge discovery for personalized medicine: cognitive science meets machine learning. IEEE Intell Inform Bull 15(1):6–14
11. Mitchell TM (1997) Machine learning. McGraw Hill, New York
12. Petz G, Karpowicz M, Frschu H, Auinger A, Sttesk V, Holzinger A (2015) Computational approaches for mining users opinions on the web 2.0. Inf Process Manag 51(4):510–519
13. Holzinger A, Dehmer M, Jurisica I (2014) Knowledge discovery and interactive data mining in bioinformatics-state-of-the-art, future challenges and research directions. BMC Bioinform 15(S6):I1
14. Spinrad N (2014) Google car takes the test. Nature 514(7523):528–528
15. Holzinger A (2014) Biomedical informatics: discovering knowledge in big data. Springer, New York
16. Holzinger A (2013) Human–computer interaction and knowledge discovery (HCI-KDD): what is the benefit of bringing those two fields to work together? In: Cuzzocrea A, Kittl C, Simos DE, Weippl E, Lida X (eds) Multidisciplinary research and practice for information systems., Springer Lecture Notes in Computer Science LNCS 8127Springer, Heidelberg, pp 319–328
17. Settles B (2011) From theories to queries: Active learning in practice. In: Guyon I, Cawley G, Dror G, Lemaire V, Statnikov A (eds) Active Learning and Experimental Design Workshop 2010, vol 16, JMLR Proceedings, Sardinia, pp 1–18
18. Fahlman SE, Hinton GE, Sejnowski TJ (1983) Massively parallel architectures for Al: NETL, Thistle, and Boltzmann machines. In: Genesereth MR (ed) AAAI-83. AAAI, Washington, DC, pp 109–113
19. Hubel DH, Wensveen J, Wick B (1995) Eye, brain, and vision. Scientific American Library, New York
20. Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Mach Learn 42(1–2):177–196
21. Holzinger A, Schantl J, Schroettner M, Seifert C, Verspoor Karin (2014) Biomedical text mining: state-of-the-art, open problems and future challenges. In: Holzinger A, Jurisica I (eds) Interactive knowledge discovery and data mining in biomedical informatics., Lecture Notes in Computer Science LNCS 8401, volume 8401Springer, Berlin, pp 271–300
22. Akgul CB, Rubin DL, Napel S, Beaulieu CF, Greenspan H, Acar B (2011) Content-based image retrieval in radiology: current status and future directions. j digit imaging 24(2):208–222
23. Gigerenzer G, Gaissmaier W (2011) Heuristic decision making. Annu Rev Psychol 62:451–482
24. Wilson AG, Dann C, Lucas, Xing EP (2015) The human kernel. arXiv preprint arXiv:1510.07389
25. Kieseberg P, Schantl J, Früwirt P, Weippl E, Holzinger A (2015) Witnesses for the doctor in the loop. In: Guo Y, Friston K, Aldo F, Hill S, Peng H (eds) Brain informatics and health., Lecture Notes in Artificial Intelligence LNAI 9250Springer, heidelberg, pp 369–378
26. Atzmüller M, Baumeister J, Puppe F (2006) Introspective subgroup analysis for interactive knowledge refinement. In Sutcliffe G, Goebel R (ed), FLAIRS nineteenth international florida artificial intelligence research society conference. AAAI Press, pp 402–407
27. G Toderici, H Aradhye, M Paca, L Sbaiz, J Yagnik (2010) Finding meaning on YouTube: tag recommendation and category discovery. In IEEE conference on computer vision and pattern recognition (CVPR 2010), IEEE, pp 3447–3454
28. Sturm W, Schreck T, Holzinger A, Ullrich T (2015) Discovering medical knowledge using visual analytics a survey on methods for systems biology and omics data. In Bühler K, Linsen L, John NW (eds) Eurographics workshop on visual computing for biology and medicine, Eurographics EG, pp 71–81
29. Müller E, Assent I, Krieger R, Jansen T, Seidl T (2008) Morpheus: interactive exploration of subspace clustering. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining KDD 08, ACM, pp 1089–1092
30. Hund M, Sturm W, Schreck T, Ullrich T, Keim D, Majnaric L, Holzinger A (2015) Analysis of patient groups and immunization results based on subspace clustering. In: Guo Y, Friston K, Aldo F, Hill S, Peng H (eds) Brain informatics and health., Lecture Notes in Artificial Intelligence LNAI 9250, vol 9250Springer International Publishing, Cham, pp 358–368
31. Shepard RN (1962) The analysis of proximities: multidimensional scaling with an unknown distance function. Psychometrika 27(2):125–140
32. Goldstone RL, Day S, Son JY (2010) Comparison. In: Glatzeder B, Goel V, Müller A (eds) Towards a theory of thinking. Springer, Berlin, pp 103–121
33. Awasthi P, Balcan M-F, Voevodski K (2014) Local algorithms for interactive clustering. In: Proceedings of the 31th international conference on machine learning, pp 550–558
34. Balcan M-F, Blum A (2008) Clustering with interactive feedback. In: Freund Y, Gyrfi L, Turn G, Zeugmann T (eds) Proceedings of the algorithmic learning theory: 19th international conference, ALT 2008, Budapest, Hungary. Springer, Berlin. pp 316–328. 13–16 Oct 2008
35. Lathrop RH (1994) The protein threading problem with sequence amino-acid interaction preferences is np-complete. Protein Eng 7(9):1059–1068
36. Good BM, Andrew I (2013) Crowdsourcing for bioinformatics. Bioinformatics 29(16):1925–1933
37. Cooper S, Khatib F, Treuille A, Barbero J, Lee J, Beenen M, Leaver-Fay A, Baker D, Popovic Z (2010) Predicting protein structures with a multiplayer online game. Nature 466(7307):756–760
38. Jane Budge E, Maria Tsoti S, James Howgate D, Sivakumar S, Jalali M (2015) Collective intelligence for translational medicine: Crowdsourcing insights and innovation from an interdisciplinary biomedical research community. Ann Med 47(7):570–575
39. Aggarwal CC (2005) On k-anonymity and the curse of dimensionality. In: Proceedings of the 31st international conference on Very large data bases VLDB, pp 901–909
40. Samarati P, Sweeney L (1998) Generalizing data to provide anonymity when disclosing information. In: Mendelzon AO, Paredaens J (eds) PODS ’98 17th ACM SIGACT-SIGMOD-SIGART symposium on principles of database systems, ACM, p 188
41. Sweeney L (2002) Achieving k-anonymity privacy protection using generalization and suppression. Int J Uncertain Fuzziness Knowl Based Syst 10(5):571–588
42. Machanavajjhala A, Kifer D, Gehrke J, Venkitasubramaniam M (2007) l-diversity: privacy beyond k-anonymity. ACM Trans Knowl Discov Data 1(1):1–52
43. Li N, Li T, Venkatasubramanian S (2007) t-Closeness: privacy beyond k-anonymity and l-diversity. In: IEEE 23rd international conference on data engineering, ICDE 2007, IEEE, pp 106–115
44. Nergiz ME, Clifton C (2010) Delta-presence without complete world knowledge. IEEE Trans Knowl Data Eng 22(6):868–883
45. Turing AM (1950) Computing machinery and intelligence. Mind 59(236):433–460
46. Littman ML (2015) Reinforcement learning improves behaviour from evaluative feedback. Nature 521(7553):445–451
47. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT press, Cambridge
48. Szepesvri C (2010) Algorithms for reinforcement learning. Synth Lect Artif Intell Mach Learn 4(1):1–103
49. Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285
50. Busoniu L, Babuska R, De Schutter B (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern Part C 38(2):156–172
51. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533
52. Thomaz A, Hoffman G, Breazeal C (2005) Real-time interactive reinforcement learning for robots. In: Oblinger D, Lau T, Gil Y, Bauer M (eds) AAAI 2005 workshop on human comprehensible machine learning. AAAI Press, Menlo Park, pp 9–13
53. Suay HB, Chernova S (2011) Effect of human guidance and state space size on interactive reinforcement learning. In: 20th IEEE international symposium on robot and human interactive communication. IEEE, pp 1–6
54. Yang M, Yang YX, Wang W, Ding HY, Chen J (2014) Multiagent-based simulation of temporal-spatial characteristics of activity-travel patterns using interactive reinforcement learning. Math Probl Eng. doi:10.1155/2014/951367
55. Holland S, Ester M, Kiessling W (2003) Preference mining: a novel approach on mining user preferences for personalized applications. In: Lavrac N, Gamberger D, Todorovski L, Blockeel H (eds) Proceedings LNAI of the knowledge discovery in databases: PKDD 2003, vol 2838. Springer, Berlin, pp 204–216
56. Trotman A (2005) Learning to rank. Inf Retr 8(3):359–381
57. Liu Tie-Yan (2011) Learning to rank for information retrieval. Springer, Berlin
58. Agarwal S, Cortes C, Herbrich R (2005) Learning to rank. In: Weiss Y, Schlkopf B, Platt JC (eds) NIPS Workshop
59. Fürnkranz J, Hüllermeier E (2010) Preference learning. Springer, Berlin
60. Hunt EB (1962) Concept learning: an information processing problem. Wiley, Hoboken
61. Feldman J (2000) Minimization of Boolean complexity in human concept learning. Nature 407(6804):630–633
62. Lake BM, Salakhutdinov R, Tenenbaum JB (2015) Human-level concept learning through probabilistic program induction. Science 350(6266):1332–1338
63. Brochu E, Freitas ND, Ghosh A (2007) Active preference learning with discrete choice data. In: Platt JC, Koller D, Singer Y, Roweis ST (eds) Advances in neural information processing systems 20 (NIPS 2007). Neural Information Processing Systems Foundation, pp 409–416
64. Chu W, Ghahramani Z (2005) Preference learning with gaussian processes. In: Proceedings of the 22nd international conference on machine learning. ACM, pp 137–144
65. Thurstone LL (1927) A law of comparative judgment. Psychol Rev 34(4):273–286
66. Bush RR, Mosteller F (1951) A mathematical model for simple learning. Psychol Rev 58(5):313–323
67. Fürnkranz J, Hüllermeier E, Cheng W, Park S-H (2012) Preference-based reinforcement learning: a formal framework and a policy iteration algorithm. Mach Learn 89(1–2):123–156
68. Zhao Y, Kosorok MR, Zeng D (2009) Reinforcement learning design for cancer clinical trials. Stat Med 28(26):3294–3315
69. Knox WB, Stone P, Breazeal C (2013) Training a robot via human feedback: a case study. In: Herrmann G, Pearson M, Lenz A, Bremner P, Spiers A, Leonards U (eds) Social robotics. Lecture Notes in Artificial Intelligence LNAI 8239. Springer, Heidelberg, pp 460–470
70. Wilson A, Fern A, Tadepalli P (2012) A Bayesian approach for policy learning from trajectory preference queries. In: Pereira F, Burges C, Bottou L, Weinberger K (eds) Advances in neural information processing systems 25 (NIPS 2012), pp 1133–1141
71. Jain A, Wojcik B, Joachims T, Saxena A (2013) Learning trajectory preferences for manipulators via iterative improvement. In: Advances in neural information processing systems, pp 575–583
72. Yimam SM, Biemann C, Majnaric L, Sabanovic S, Holzinger A (2015) Interactive and iterative annotation for biomedical entity recognition. In: Yike G, Karl F, Faisal A, Sean H, Hanchuan P (eds) Brain Informatics and Health. Lecture Notes in Artificial Intelligence LNAI 9250. Springer, Cham, pp 347–357
73. Settles B (2012) Active learning. Morgan and Claypool, San Rafael
74. Cohn DA, Ghahramani Z, Jordan MI (1996) Active learning with statistical models. J Artif Intell Res 4:129–145
75. Warmuth MK, Liao J, Raetsch G, Mathieson M, Putta S, Lemmen C (2003) Active learning with support vector machines in the drug discovery process. J Chem Inf Comp Sci 43(2):667–673
76. Yue Y, Joachims T (2009) Interactively optimizing information retrieval systems as a dueling bandits problem. In: Proceedings of the 26th annual international conference on machine learning (ICML), ACM, pp 1201–1208
77. Yue Y, Broder J, Kleinberg R, Joachims T (2012) The k-armed dueling bandits problem. J Comput Syst Sci 78(5):1538–1556
78. Viappiani P, Boutilier C (2010) Optimal bayesian recommendation sets and myopically optimal choice query sets. In: Lafferty JD, Williams CKI, Shawe-Taylor J, Zemel RS, Culotta A (eds) Advances in neural information processing systems 23. Curran, pp 2352–2360
79. Valiant LG (1984) A theory of the learnable. Commun ACM 27(11):1134–1142
80. Baxter J (2000) A model of inductive bias learning. J Artif Intell Res 12:149–198
81. Evgeniou T, Pontil M (2004) Regularized multi-task learning. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 109–117
82. Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244
83. Parameswaran S, Weinberger KQ (2010) Large margin multi-task metric learning. In: Lafferty J, Williams C, Shawe-Taylor J, Zemel R, Culotta A (eds) Advances in neural information processing systems 23 (NIPS 2010), pp 1867–1875
84. McCloskey M, Cohen NJ (1989) Catastrophic interference in connectionist networks: the sequential learning problem. In: Bower GH (ed) The psychology of learning and motivation, vol 24. Academic Press, San Diego, pp 109–164
85. French Robert M (1999) Catastrophic forgetting in connectionist networks. Trends Cogn Sci 3(4):128–135
86. Goodfellow IJ, Mirza M, Xiao D, Courville A, Bengio Y (2015) An empirical investigation of catastrophic forgeting in gradient-based neural networks. hyperimagehttp://arxiv.org/abs/1312.6211v3arXiv:1312.6211v3
87. Pan SJ, Yang QA (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359
88. Taylor ME, Stone P (2009) Transfer learning for reinforcement learning domains: a survey. J Mach Learn Res 10:1633–1685
89. Sycara KP (1998) Multiagent systems. AI Mag 19(2):79
90. Lynch NA (1996) Distributed algorithms. Morgan Kaufmann, San Francisco
91. DeGroot MH (1974) Reaching a consensus. J Am Stat Assoc 69(345):118–121
92. Benediktsson JA, Swain PH (1992) Consensus theoretic classification methods. IEEE Trans Syst Man Cybern 22(4):688–704
93. Weller SC, Mann NC (1997) Assessing rater performance without a gold standard using consensus theory. Med Decis Mak 17(1):71–79
94. Olfati-Saber R, Fax JA, Murray RM (2007) Consensus and cooperation in networked multi-agent systems. In: Proceedings of the IEEE, vol 95(1), pp 215–233
95. Roche B, Guegan JF, Bousquet F (2008) Multi-agent systems in epidemiology: a first step for computational biology in the study of vector-borne disease transmission. BMC Bioinform 9:435
96. Kok JR, Vlassis N (2006) Collaborative multiagent reinforcement learning by payoff propagation. J Mach Learn Res 7:1789–1828
97. Freund R, Neubauer M, Summerer M, Gruber S, Schaffer J, Swoboda R (2000) A hybrid system for the recognition of hand-written characters. In: Ferri FJ, Iñesta JM, Amin A, Pudil P (eds) Advances in pattern recognition, LNCS, vol 1876. Springer, Heidelberg, pp 67–76. doi:10.1007/3-540-44522-6_7
98. Parunak HVD, Nielsen P, Brückner S, Alonso R (2007) Hybrid multi-agent systems: integrating swarming and BDI agents. In: Brueckner SA., Hassas S, Jelasity M, Yamins D (eds) Engineering self-organising systems, LNCS, vol 4335. Springer, Berlin, pp 1–14. doi:10.1007/978-3-540-69868-5_1