RheumaVIT: Transformer-Based Model for Automated Scoring of Hand Joints in Rheumatoid Arthritis

Alexander Stolpovsky, Elizaveta Dakhova, Polina Druzhinina, Polina Postnikova, Daniil Kudinsky, Alexander Smirnov, Anastasia Sukhinina, Alexander Lila, Anvar Kurmukov; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 2522-2531

Rheumatoid arthritis (RA) is an autoimmune disease that causes chronic inflammation, joint destruction, and extra-articular manifestations. Radiography is the standard imaging modality for diagnosing and monitoring joint damage in RA. However, the commonly used Sharp method and its variants, which evaluate radiographic progression, are time-consuming and subjective. Automated joint evaluation using deep neural networks can address these challenges. This study introduces RheumaVIT, a novel vision transformer-based pipeline for automatically scoring hand joints affected by RA. The method consists of two stages: a regression model for joint localization and a transformer-based architecture for assessing erosion and joint space narrowing (JSN). Our approach demonstrates superior accuracy (up to 12% higher for erosion and 2% higher for JSN) compared to existing state-of-the-art methods. Moreover, it has a promising ability to detect common patterns of erosion and JSN through roll-out interpretation. To promote further research, we are open-sourcing our clinical collection since there is no annotated dataset on RA available in the public domain. Our paper contributes to the progress of automated joint assessment in rheumatoid arthritis, offering potential applications in both clinical practice and research.