Machine Learning (ML) systems are increasingly deployed in safety- and security-critical domains including self-driving cars and malware detection, where the correctness and predictability of a system’s behavior for corner case inputs are of great importance. However, state-of-the-art ML systems, despite their impressive capabilities, often have unexpected/incorrect behaviors in corner cases as demonstrated by the recent fatal collision involving Tesla's autopilot system. Unfortunately, traditional software testing techniques are not well-suited for testing modern ML systems like deep neural networks. In this talk, I will highlight the major challenges and possible solutions to developing systematic testing techniques that can ensure the safety and security of ML systems. I will share promising results from DeepXplore and DeepTest, two of our recent projects demonstrating complementary approaches for systematically testing state-of-the-art deep neural networks with corner-case inputs.