Performance Analysis of Classification Algorithms for Millimeter-wave Imaging

A detailed analysis of the performance of traditional machine learning and deep learning techniques applied on a representative classification problem of millimeter-wave (mmW) images is presented in this paper. The algorithms chosen for this analysis are the k-Nearest Neighbors (KNN), Random Forest (RF) and Convolutional Neural Network (CNN). All algorithms presented here are modeled using 'keras' library inside TensorFlow and 'scikit-learn' module. The dataset for training and testing are generated via a developed near-field coded aperture computational imaging (CI) physical model. The use of a physical model of an imaging system that implements CI techniques instead of an experimental set-up makes the whole dataset generation process facile and less time consuming. The training data, in case of the RF and KNN algorithms, are presented in tabular form whereas for the CNN technique, the synthesized images from the physical model itself are used for training. The models are tested with both synthesized as well as experimental data, generated from the physical model and a mmW handheld imager, respectively. Upon testing, it is observed that the KNN and RF algorithms are able to classify the test samples with accuracies of 82% and 87%, respectively, whereas an accuracy of 90% is observed in case of the CNN classifier. Also, an inference speed test is conducted on all the three algorithms. It was observed that CNN is the fastest to predict classes for all of the test samples with a frame rate of 3.8 ms/sample whereas RF is the slowest, with a frame rate of 65.9 ms/sample. These findings establish the fact that when it comes to image classification, CNN based classifiers perform better than any traditional machine learning algorithms with more accurate and faster predictions, paving the way for various real-time applications such as automatic threat detection.