Automated Hand Joint Classification of Psoriatic Arthritis Patients Using Routinely Acquired Near Infrared Fluorescence Optical Imaging

Near infrared fluorescence optical imaging (NIR-FOI) is a relatively new imaging modality to diagnose arthritis in the hands. The acquired data has two spatial dimensions and one temporal dimension, which visualizes the time dependent distribution of an administered color agent. In accordance with previous work, we hypothesize that the distribution process allows a joint-wise classification into inflammatory affected and unaffected. In this work, we present the first approach to objectively classify hand joint NIR-FOI image stacks by designing, training, and testing a neural network. Previously presented model architectures for spatio-temporal classification do not yield satisfying results when trained on NIR-FOI data. A recall value of 0.812 of the over- and a recall value of 0.652 of the underrepresented class is achieved, the model’s robustness tested against small variations and its attention visualized in activation maps. Even though these results leave room for further improvement, they also indicate, that the model architecture can capture the latent features of the data. We are confident, that more available data will lead to a robust classification model and can support medical doctors in using NIR-FOI as a diagnostic tool for PsA.