On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products

Big Data

Volumn 5, Issue 3, 2017, Pages 246-255

  Varshney, Kush R ^a   Alemzadeh, Homa ^b  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b University of Virginia   (United States)

Author keywords

cyber physical systems; data products; decision science; machine learning; safety

Indexed keywords

ALGORITHM; DECISION MAKING; ECONOMICS; MACHINE LEARNING; SAFETY;

ALGORITHMS; DECISION MAKING; MACHINE LEARNING; SAFETY;

EID: 85049006775     PISSN: 21676461     EISSN: 2167647X     Source Type: Journal    
DOI: 10.1089/big.2016.0051     Document Type: Article

References (54)

1. Conn A. 2015. The AI wars: The battle of the human minds to keep artificial intelligence safe. Available online at http://futureoflife. org/2015/ 12/17/the-ai-wars-the-battle-of-the-human-minds-to-keep-artificialintelligence-safe (accessed September 8, 2017).
2. Ferrell T. Engineering safety-critical systems in the 21st century. IEEE Central Virginia Section, Engineers Week Dinner Meeting. Charlottesville, VA, 2010.
3. Varshney KR. Engineering safety in machine learning. In: Proceeding of Information Theory and Application Workshop, La Jolla, CA, 2016.
4. Möller N, Hansson SO. Principles of engineering safety: Risk and uncertainty reduction. Reliab Eng Syst Safe. 2008;93:798-805.
5. Möller N. The concepts of risk and safety. In: Roeser S, Hillerbrand R, Sandin P, Peterson M (Eds.): Handbook of Risk Theory, Dordrecht, Netherlands: Springer, 2012. pp. 55-85.
6. Senge R, Bösner S, Dembczynski K, et al. Reliable classification: Learning classifiers that distinguish aleatoric and epistemic uncertainty. Inf Sci. 2014;255:16-29.
7. Vapnik V. Principles of risk minimization for learning theory. Adv Neur Inf Process Syst. 1992;4:831-838.
8. Wagstaff KL. Machine learning that matters. In: Proceedings of International Conference on Machine Learning, Edinburgh, United Kingdom, June-July 2012, pp. 529-536.
9. Alemzadeh H. Data-driven resiliency assessment of medical cyberphysical systems. Ph. D. dissertation, University of Illinois, Urbana-Champaign, Urbana, IL, 2016.
10. Stanley J, Tunkelang D. 2016. Doing data science right-your most common questions answered. Available online at http:// firstround. com/review/doing-data-science-right-your-most-commonquestions-answered (accessed September 8, 2017).
11. Olteanu A, Talamadupula K, Varshney KR. The limits of abstract evaluation metrics: The case of hate speech detection. In: Proceedings on ACM Web Science Conference, Troy, NY, 2017, pp. 405-406.
12. Alemzadeh H, Raman J, Leveson N, et al. Adverse events in robotic surgery: A retrospective study of 14 years of FDA data. PLoS One. 2016;11:1-20.
13. Knight J. Fundamentals of dependable computing for software engineers. Boca Raton, FL: CRC Press, 2012.
14. Shimodaira H. Improving predictive inference under covariate shift by weighting the log-likelihood function. J Stat Plan Inference. 2000;90:227-244.
15. Daume H III, Marcu D. Domain adaptation for statistical classifiers. J Artif Intell Res 2006;26:101-126.
16. Caruana R, Lou Y, Gehrke J, et al. Elhadad, intelligible models for healthcare: predicting pneumonia risk and hospital 30-day readmission. In: Proceedings on ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Sydney, Australia, 2015, pp. 1721-1730.
17. Freitas AA. Comprehensible classification models-A position paper. SIGKDD Explorations 2013;15:1-10.
18. Rudin C. Algorithms for interpretable machine learning. In: Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, New York, NY, 2014, p. 1519.
19. Athey S, Imbens GW. 2015. Machine learning methods for estimating heterogeneous causal effects. Available online at http://arxiv. org/pdf/ 1504. 01132. pdf (accessed September 8, 2017).
20. Welling M. 2015. Are ML and statistics complementary? In: IMS-ISBA Meeting on 'Data Science in the Next 50 Years'.
21. Wang F, Rudin C. 2015. Causal falling rule lists. Available online at http:// arxiv. org/pdf/1510. 05189. pdf (accessed September 8, 2017).
22. Chakarov A, Nori A, Rajamani S, et al. 2016. Debugging machine learning tasks. Available online at http://arxiv. org/pdf/1603. 07292. pdf (accessed September 8, 2017).
23. Petrik M, Luss R. Interpretable policies for dynamic product recommendations. In: Proceedings of Conference on Uncertainty Artificial Intelligence, Jersey City, NJ, 2016, p. 74.
24. Provost F, Fawcett T. Robust classification for imprecise environments. Mach Learn. 2001;42:203-231.
25. Davenport MA, Baraniuk RG, Scott CD. Tuning support vector machines for minimax and Neyman-Pearson classification. IEEE Trans Pattern Anal Mach Intell. 2010;32:1888-1898.
26. Hajian S, Domingo-Ferrer J. A methodology for direct and indirect discrimination prevention in data mining. IEEE Trans Knowl Data Eng. 2013;25:1445-1459.
27. Feldman M, Friedler SA, Moeller J, et al. Certifying and removing disparate impact. In: Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Sydney, Australia, 2015, pp. 259-268.
28. Barocas S, Selbst AD. Big data's disparate impact. California Law Rev. 2016;104.
29. The U. S. EEOC. 1979. Uniformguidelines on employee selection procedures.
30. Varshney KR, Prenger RJ, Marlatt TL, et al. Practical ensemble classification error bounds for different operating points. IEEE Trans Knowl Data Eng. 2013;25:2590-2601.
31. Attenberg J, Ipeirotis P, Provost F. Beat the machine: challenging humans to find a predictive model's ''unknown unknowns. '' ACM J Data Inf Qual. 2015;6:1.
32. Weiss GM. Mining with rarity: A unifying framework. SIGKDD Explorations Newsletter 2004;6:7-19.
33. Sahuguet A, Krauss J, Palacios L, Sangokoya D. Open civic data: Of the people, by the people, for the people. Bull Tech Comm Data Eng. 2014;37:15-26.
34. Shaw E. 2015. Improving service and communication with open data: A history and how-to. Ash Center, Harvard Kennedy School, Tech. Rep. Available online at http://datasmart. ash. harvard. edu/news/article/ improving-service-and-communication-with-open-data-702 (accessed August 31, 2017).
35. Kapoor S, Mojsilovic A, Strattner JN, Varshney KR. From open data ecosystems to systems of innovation: A journey to realize the promise of open data. In: Proceedings of Data for Good Exchange Conference, New York, NY, 2015.
36. Schirner G, Erdogmus D, Chowdhury K, Padir T. The future of human-inthe-loop cyber-physical systems. Computer. 2013;36-45.
37. Kassahun Y, Yu B, Tibebu AT, et al. Surgical robotics beyond enhanced dexterity instrumentation: A survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int J Comput Assist Radiol Surg. 2016;11:553-568.
38. Lin HC, Shafran I, Murphy TE, et al. Automatic detection and segmentation of robot-assisted surgical motions. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005, pp. 802-810.
39. Lin HC, Shafran I, Yuh D, Hager GD. Towards automatic skill evaluation: Detection and segmentation of robot-assisted surgical motions. Comput Aided Surg. 2006;11:220-230.
40. Reiley CE, Plaku E, Hager GD. Motion generation of robotic surgical tasks: Learning from expert demonstrations. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, 2010, pp. 967-970.
41. Shademan A, Decker RS, Opfermann JD, et al. Supervised autonomous robotic soft tissue surgery. Sci Translational Med. 2016;8:37ra64-337ra64.
42. Alemzadeh H, Chen D, Lewis A, et al. Systems-theoretic safety assessment of robotic telesurgical systems. In: Proceedings of International Conference on Computer Safety, Reliability, and Security, 2015, pp. 213-227.
43. Azimian H, Naish MD, Kiaii B, Patel RV. A chance-constrained programming approach to preoperative planning of robotic cardiac surgery under tasklevel uncertainty. IEEE Trans Biomed Health Inf. 2015;19:612-1898.
44. Rayej S. 2014. How do self-driving cars work? Available online at http:// robohub. org/how-do-self-driving-cars-work/ (accessed September 8, 2017).
45. Lowy J. 2016. Driver killed in self-driving car accident for first time. Available online at www. pbs. org/newshour/rundown/driver-killed-inself-driving-car-accident-for-first-time (accessed September 8, 2017).
46. Duchi J, Glynn P, Johari R. 2016. Uncertainty on uncertainty, robustness, and simulation. SAIL-Toyota Center for AI Research, Stanford University, Tech. Rep. Available online at http://aicenter. stanford. edu/uncertaintyon-uncertainty-robustness-and-simulation (accessed August 31, 2017).
47. Zhu Y, Janapa Reddi V. Cognitive computing safety: The new horizon for reliability. IEEE Micro. 2017;37:15-21.
48. Koopman P, Wagner M. Challenges in autonomous vehicle testing and validation. SAE Int J Transportation Saf. 2016;4:2016-01-0128.
49. Brynjolfsson E, Hitt L, Kim H. Strength in numbers: How does data-driven decision-making affect firm performance? In: Proceedings of International Conference on Information System, Shanghai, China, 2011, p. 13.
50. Singh M, Varshney KR, Wang J, et al. An analytics approach for proactively combating voluntary attrition of employees. In: Proceedings of IEEE International Conference on Data Mining Workshops, Brussels, Belgium, 2012, pp. 317-323.
51. Wei D, Varshney KR. Robust binary hypothesis testing under contaminated likelihoods. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Brisbane, Australia, 2015, pp. 3407-3411.
52. Malioutov DM, Varshney KR. Exact rule learning via Boolean compressed sensing. In: Proceedings of International Conference Machine Learning, Atlanta, GA, 2013, pp. 765-773.
53. Gerard H, Rao K, Simithraaratchy M, et al. Predictive modeling of customer repayment for sustainable pay-as-you-go solar power in rural India. In: Proceedings of Data for Good Exchange Conf., New York, NY2015.
54. Goodman B, Flaxman S. European Union regulations on algorithmic decision-making and a'right to explanation'. In: Proceedings of ICML Workshop Human Interpretability, New York, NY. 2016, pp. 26-30.