A Safe, Secure, and Predictable Software Architecture for Deep Learning in Safety-Critical Systems

IEEE Embedded Systems Letters

Volumn 12, Issue 3, 2020, Pages 78-82

  Biondi, Alessandro ^a   Nesti, Federico ^a   Cicero, Giorgiomaria ^a   Casini, Daniel ^a   Buttazzo, Giorgio ^a  

^a SCUOLA SUPERIORE SANT ANNA   (Italy)

Author keywords

Deep learning; deep neural networks (DNNs); fault Tolerance; machine learning; predictability; safety; safety critical systems; security

Indexed keywords

ACCIDENT PREVENTION; COMPUTER ARCHITECTURE; COMPUTER CONTROL SYSTEMS; COMPUTER SOFTWARE; CONTROL SYSTEMS; DEEP NEURAL NETWORKS; FAULT DETECTION; FAULT TOLERANCE; IMAGE RECOGNITION; LEARNING SYSTEMS; MAN MACHINE SYSTEMS; NETWORK ARCHITECTURE; OBJECT DETECTION; SECURITY SYSTEMS; SOFTWARE ARCHITECTURE;

HUMAN-LEVEL PERFORMANCE; LEARNING TECHNIQUES; PREDICTABILITY; PROPOSED ARCHITECTURES; SAFETY CRITICAL APPLICATIONS; SAFETY CRITICAL SYSTEMS; SECURITY; VIRTUAL MACHINE MONITORS;

DEEP LEARNING;

EID: 85081294095     PISSN: 19430663     EISSN: 19430671     Source Type: Journal    
DOI: 10.1109/LES.2019.2953253     Document Type: Article

References (21)

1. MISRA. Accessed: Nov. 26, 2019. [Online]. Available: https://www.misra.org.uk/
2. "Collision between a car operating with automated vehicle control systems and a tractor-semitrailer truck near williston, Florida May 7, 2016, " Nat. Transp. Safety Board, Washington, DC, USA, Rep. NTSB/HAR-17/02, 2017.
3. C. Szegedy et al., "Intriguing properties of neural networks, " arXiv:1312.6199v4 [cs.CV], 2014. [Online]. Available: https://arxiv.org/abs/1312.6199
4. X. Huang et al., "Safety and trustworthiness of deep neural networks: A survey, " arXiv:1812.08342v2 [cs.LG], 2018. [Online]. Available: https://arxiv.org/abs/1812.08342v2
5. S. W. X. Huang, M. Kwiatkowska, and M. Wu, "Safety verification of deep neural networks, " in Proc. 29th Int. Conf. Comput.-Aided Verification (CAV), 2017, pp. 3-29.
6. Y. Tian, K. Pei, S. Jana, and B. Ray, "Deeptest: Automated testing of deep-neural-network-driven autonomous cars, " in Proc. 40th Int. Conf. Softw. Eng., 2018, pp. 303-314.
7. K. Pei, Y. Cao, J. Yang, and S. Jana, "Deepxplore: Automated whitebox testing of deep learning systems, " in Proc. 26th Symp. Oper. Syst. Princ., 2017, pp. 1-18.
8. Y. Sun, X. Huang, D. Kroening, J. Sharp, M. Hill, and R. Ashmore, "Testing deep neural networks, " arXiv:1803.04792 [cs.LG], 2019. [Online]. Available: https://arxiv.org/abs/1803.04792
9. L. Sha, "Using simplicity to control complexity, " IEEE Softw., vol. 18, no. 4, pp. 20-28, Jul. 2001.
10. B. P. Douglass, Real-Time Design Patterns: Robust Scalable Architecture for Real-Time Systems. Boston, MA, USA: Addison-Wesley Professional, 2002.
11. H. Zhou, S. Bateni, and C. Liu, "S3DNN: Supervised streaming and scheduling for GPU-Accelerated real-Time DNN workloads, " in Proc. IEEE Real Time Embedded Technol. Appl. Symp. (RTAS), Apr. 2018, pp. 190-201.
12. M. Yang et al., "Re-Thinking CNN frameworks for time-sensitive autonomous-driving applications: Addressing an industrial challenge, " in Proc. IEEE Real Time Embedded Technol. Appl. Symp. (RTAS), Apr. 2019, pp. 305-317.
13. Y. Luo, A. K. Saberi, T. Bijlsma, J. J. Lukkien, and M. van den Brand, "An architecture pattern for safety critical automated driving applications: Design and analysis, " in Proc. Annu. IEEE Int. Syst. Conf. (SysCon), Apr. 2017, pp. 1-7.
14. S. Bak, T. T. Johnson, M. Caccamo, and L. Sha, "Real-Time reachability for verified simplex design, " in Proc. IEEE Real Time Syst. Symp., 2014, pp. 138-148.
15. N. Burow et al., "Control-flow integrity: Precision, security, and performance, " ACM Comput. Surveys, vol. 50, no. 1, Apr. 2017, Art. no. 16.
16. D. Seto, B. Krogh, L. Sha, and A. Chutinan, "The simplex architecture for safe online control system upgrades, " in Proc. Amer. Control Conf., Jun. 1998, pp. 3504-3508.
17. S. Bak, K. Manamcheri, S. Mitra, and M. Caccamo, "Sandboxing controllers for cyber-physical systems, " in Proc. IEEE/ACM 2nd Int. Conf. Cyber Phys. Syst., 2011, pp. 3-12.
18. K. Vivekanandan, G. Garcia, H. Yun, and S. Keshmiri, "A simplex architecture for intelligent and safe unmanned aerial vehicles, " in Proc. IEEE Int. Conf. Embedded Real Time Comput. Syst. Appl., 2016, pp. 69-75.
19. A. Biondi, A. Balsini, M. Pagani, E. Rossi, M. Marinoni, and G. Buttazzo, "A framework for supporting real-Time applications on dynamic reconfigurable FPGAs, " in Proc. IEEE Real Time Syst. Symp. (RTSS), Dec. 2016, pp. 1-12.
20. Nvidia TensortRT. Accessed: Nov. 26, 2019. [Online]. Available: https://developer.nvidia.com/tensorrt
21. F. Abdi, C.-Y. Chen, M. Hasan, S. Liu, S. Mohan, and M. Caccamo, "Guaranteed physical security with restart-based design for cyberphysical systems, " in Proc. 9th ACM/IEEE Int. Conf. Cyber Phys. Syst., 2018, pp. 10-21