Replacing the Region Proposal Network of a Mask-RCNN with an Event Camera to increase the Speed of Video File Processing

This Master thesis tests the possibility to replace the Regional Proposal Network (RPN) of a Faster or Mask R-CNN with an event camera. The goal is to accelerate the analysis of frames of a video. First, a mount for two cameras is designed, using a designing system, that allows both cameras to be adjusted to each other so that they capture the same scene, and the event data of the event camera can be superimposed to the RGB image. The Detectron2 library is used, in which changes are made to replace the RPN. These changes will be described. Then the best algorithm to clean the event camera data from background noise and to cluster the events is experimentally determined. The criteria used are speed and accuracy. These clusters replace the RPN finally. Through the experiments it is shown that for the cleaning of the event data the IsolationForest algorithm is best suited with the parameter "contamination" set to auto. For the actual clustering process, DBSCAN is selected. Here the parameters ms (min_samples) = 5, eps (distance between the datapoints) = 6 and leaf_size = 20 have been evaluated as optimal. Finally, a direct comparison is made between the original RPN and the replacement with the event camera. The criteria were again accuracy and speed. The result is that the replacement is as good as the original in terms of accuracy. In terms of speed, however, a distinction must be made as to whether a CPU or GPU is used. A Mask R-CNN with the event camera as replacement for the RPN is faster on the CPU, but clearly slower when using the GPU.