Learning Anatomy-Disease Entangled Representation

Fatemeh Haghighi, Michael B. Gotway, Jianming Liang; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 4129-4141

Human experts demonstrate proficiency not only in disentangling anatomical structures from disease conditions but also in intertwining anatomical and disease information to accurately diagnose a variety of disorders. However deep learning models despite their prowess in acquiring intricate representation often struggle to learn representation where distinct semantic aspects of the data (both anatomy and pathology) are entangled particularly in medical images which present a rich array of anatomical structures and potential pathological conditions. We envision that a deep model when trained to comprehend medical images akin to human perception would offer powerful representation with higher generalizability robustness and interpretability. To realize this vision we have developed LeADER a framework for learning anatomy-disease entangled representation from medical images. As a proof of concept we have trained LeADER on 1M chest radiographs gathered from 10 public datasets. Experimental results across 11 medical tasks compared to 8 baselines in zero-shot linear probing limited data regimes and full fine-tuning settings demonstrate LeADER's superior performance over the Google CXR Foundation Model large-scale medical models and fully/self-supervised baselines across diverse downstream tasks. This enhanced performance is attributed to the significance of entangling anatomy-specific and disease-specific representations via our framework which enables the simultaneous acquisition of both anatomical and disease knowledge yet overlooked in existing supervised/self-supervised learning methods. All code and models are available at GitHub.com/JLiangLab/LeADER.