Construction-sAIt: Multi-modal AI-driven technologies for construction site monitoring

In the construction industry, unexpected accidents, thefts, and vandalism can cause high costs due to construction delays. Also, logistic monitoring is of central importance to ensure a smooth construction process, especially for largescale projects. Manual attempts to secure the safety of construction workers and to monitor traffic flows and deliveries of construction materials are often costly and inefficient. As a promising solution, audio-visual analysis algorithms combined with distributed sensor networks across construction sites can help to automate these processes. Inspired by human perception, the combination of multiple sensor modalities in a multi-modal approach is most promising since it allows to balance the individual weaknesses of a pure visual or acoustic analysis in particular application scenarios.
In this paper, we will focus on several use-cases, where AI-driven machine listening and computer vision algorithms can be used to automatically analyze streaming audio and video recordings taken at different locations on a construction site. These use-cases include for instance the monitoring of the construction process by recognizing work steps such as drilling, sawing, and hammering, as well as the detection and classification of passing vehicles for logistic monitoring.