Evaluation of binary keypoint descriptors

In this paper an evaluation of state-of-the-art binary keypoint descriptors, namely BRIEF, ORB, BRISK and FREAK, is presented. In contrast to previous evaluations we used the Stanford Mobile Visual Search (SMVS) data set because binary descriptors are mainly used in mobile applications. This large data set does provide a lot of characteristic transformations for mobile devices, but no ground truth data. The often used Oxford data set is used only for validation purposes. We use ratio-test and RANSAC (RANdom SAmple Consensus) for evaluation and present results for accuracy, precision and average number of best matches as performance metrics. The validity of the results is also checked by evaluating these binary keypoint descriptors on Oxford data set. The obtained results show that BRISK is the keypoint descriptor which gives highest percentage of precision and largest number of best matches among all the binary descriptors. Next to BRISK is FREAK, which offers comparably good result.