Special Session

39.3 - Automated Systematic Testing of Deep Learning Systems

10:30 AM–12:00 PM Jun 27, 2018 (America - Los Angeles)

Room 3020

Machine Learning (ML) systems are increasingly deployed in safety- and security-critical domains including self-driving cars and malware detection, where the correctness and predictability of a system’s behavior for corner case inputs are of great importance. However, state-of-the-art ML systems, despite their impressive capabilities, often have unexpected/incorrect behaviors in corner cases as demonstrated by the recent fatal collision involving Tesla's autopilot system. Unfortunately, traditional software testing techniques are not well-suited for testing modern ML systems like deep neural networks. In this talk, I will highlight the major challenges and possible solutions to developing systematic testing techniques that can ensure the safety and security of ML systems. I will share promising results from DeepXplore and DeepTest, two of our recent projects demonstrating complementary approaches for systematically testing state-of-the-art deep neural networks with corner-case inputs.