DFarm: Massive-Scaling Dynamic Android App Analysis on Real Hardware

Dynamic analysis is an important tool for assessing software quality during testing. It not only helps analysts identify performance bottlenecks and functional errors, but also provides a means for finding security vulnerabilities. For example, analysts can determine the servers to which a mobile app connects, which sensitive data it transfers, and which cryptographic protocols it uses for the transfer. While many approaches for monitoring a running Android app exist, most work silently assumes that a suitable execution environment is available. When analyzing hundreds of apps at the same time, however, a single phone on the analyst's desk is not enough. Emulators are not always an alternative as we show, because apps can behave differently on real hardware. In this paper, we discuss the challenges for providing a large-scale testing environment with real Android devices on physical hardware. We further present DFarm, a software and hardware system to configure and control hundreds of Android phones in a private testing cloud. We discuss electrical wiring, USB and WiFi connectivity, automatic configuration, and load balancing. We evaluate DFarm on a range between 1 and more than 70 devices. We show that it provides near-linear scaling for dynamic app analysis when adding new devices, while retaining the original device's computation and network performance.