CrossTest: A cross-domain physical testbed environment for cybersecurity performance evaluations

Network based intrusion detection systems (NIDS) play a vital role in protecting valuable assets or applications in a wide range of industrial domains. Especially commercial NIDS providers need to address very specific requirements and challenges for theses domains, such as supporting a variety of different network protocols. Despite such challenges, most commercial NIDS vendors offer one solution for multiple industrial domains. In contrast, most NIDSs proposed by researchers are evaluated on only a few domain-specific datasets due to the lack of publicly available industrial datasets. Therefore, conclusions about the applicability of research-oriented NIDS across industrial domains cannot be made. Domain-agnostic threat detection methods are required when advanced persistent threats (APT) are evolving across multiple sectors. This research work presents a cross-domain physical cybersecurity testbed environment, CrossTest, for the development and evaluation of domain-agnostic threat detection methods. For this purpose, two testbeds were designed, one for the energy and another for the production domain. Multiple cyber-attacks were implemented in both testbeds and network traffic was recorded as PCAP files. The dataset containing PCAP files with corresponding description will be made publicly available upon request. Furthermore, we demonstrate the evaluation of an open source network traffic analysis tool, Malcolm, with CrossTest. The evaluation identified major issues that are briefly described in this work.