\section{Introduction}
\label{sec:intro}

Multi-label image classification (\texttt{MLIC}) seeks to predict all relevant object categories present in a single image. 
Unlike single-label recognition, \texttt{MLIC} must model \emph{semantic dependencies} among labels (e.g., \textit{person–bicycle–helmet}) and contextual interactions between multiple visual regions. 
Recent convolutional and transformer-based architectures have achieved strong results by learning data-driven attention over image features~\cite
{liu2021query2label,ridnik2021mldecoder,cho2022CoFormer}. 
However, these methods still struggle to generalize beyond dataset statistics, as they rely heavily on co-occurrence patterns while overlooking \emph{external knowledge} that can guide reasoning. 
Human perception naturally integrates visual cues with prior knowledge -- we expect traffic lights near cars or plates beside forks.
Motivated by this, knowledge-guided learning has emerged as a promising direction~\cite{kgmlsurvey2024,kgcv2022}.
Existing approaches typically encode knowledge graphs (\texttt{KG}) such as \texttt{ConceptNet55}~\cite{speer2017conceptnet} or \texttt{WordNet}~\cite{miller1995wordnet} into static adjacency priors and combine them with deep visual features through graph convolutional networks (\texttt{GCN})~\cite{dewi2025knowledge, chen2019multi}.
% However, these fixed propagation rules limit expressiveness and fail to capture asymmetric or higher-order correlations among labels.
However, the predefined message-passing schemes in conventional graph models constrain their representational capacity and fail to capture asymmetric or high-order dependencies among labels.
At the same time, purely visual attention mechanisms remain agnostic to explicit semantic relations, often resulting in redundant activations and misclassified co-occurring objects.

Recent transformer-based approaches address part of this challenge by adaptively extracting label-specific local features for multiple objects within an image.
While earlier methods often relied on globally average-pooled CNN features for simplicity, which may weaken spatial grounding ~\cite{chen2019learning, ridnik2021asymmetric}, our model preserves spatial cues via Transformer token-level attention and uses global pooling only as a context signal (Eq.~2).
The cross-attention mechanism of the Transformer decoder enables adaptive feature pooling guided by label queries, where each label acts as a query attending to relevant image regions for accurate binary classification~\cite{vaswani2017attention}. 
The related work by You et al.~\cite{you2020cross} adopts a similar idea but computes attention using cosine similarity with negative clipping and identical key-value features, which restricts its ability to learn fine-grained and discriminative features. 
\textit{Inspired by the label-as-query design of \texttt{Query2Label}~\cite{liu2021query2label}, we extend this paradigm by introducing knowledge-guided reasoning through a graph attention mechanism that explicitly models semantic dependencies among labels. 
Unlike \texttt{Query2Label}, which focuses solely on visual attention, our proposed \texttt{KGGAT} jointly leverages semantic priors from an external knowledge graph and an adaptive cross-attention mechanism to capture richer inter-label dependencies and improve interpretability.}

To address the limitations of both static \texttt{GCN}-based reasoning and {spatially coarse Transformer attention}, we propose {\texttt{KGGAT}} for multi-label image classification. As illustrated in Figure~\ref{fig:KGGCN1}, \texttt{KGGAT} unifies visual representation learning and semantic reasoning within a single framework.
A convolutional backbone (e.g., \texttt{TResNetL} or \texttt{ResNet}) first extracts spatial image features, which are then processed by Transformer decoders using cross-attention with label embeddings to capture label-specific visual cues.
\textit{In parallel, the knowledge-guided graph attention module leverages external knowledge graphs to model semantic dependencies among labels and refine their relationships adaptively.} \texttt{KGGAT} bridges visual learning and knowledge reasoning through an image-conditioned graph attention mechanism that dynamically refines label relations. Each class is represented as a node, and edges are initialized from an external knowledge matrix derived from sources such as \texttt{ConceptNet55} or the \texttt{OpenAI} knowledge base.
During inference, the graph attention layer adaptively updates these edges based on both semantic priors and visual evidence, enabling selective message passing between semantically related labels.
Through this joint reasoning process, \texttt{KGGAT} highlights informative label relations while suppressing noisy correlations, producing interpretable and semantically consistent predictions.
% To further enhance interpretability, we introduce  \emph{knowledge-consistency regularization} that aligns learned attention weights with prior semantic relations, stabilizing training and preserving conceptual consistency while adapting to image-specific contexts.

We evaluate \texttt{KGGAT} on the \texttt{MS-COCO} and \texttt{VOC 2007} benchmarks. 
Our model surpasses previous \texttt{GCN}- and transformer-based \texttt{MLIC} methods, including \texttt{Query2Label}~\cite{liu2021query2label}, \texttt{CoFormer}~\cite{cho2022CoFormer}, and \texttt{ML-Decoder}~\cite{ridnik2021mldecoder}. 
\texttt{KGGAT} achieves {$89.21$\% mAP} on \texttt{MS-COCO} and {$96.9$\% mAP} on \texttt{VOC 2007}, establishing new state-of-the-art performance among graph-based methods while providing clear label-to-region attention visualizations. 
Comprehensive ablations demonstrate the benefits of our adaptive graph reasoning and knowledge-consistency design in improving both accuracy and interpretability.
Our main contributions are three-fold:
\begin{itemize}
 \item We propose a unified {Knowledge-Guided Graph Attention Network (\texttt{KGGAT})} that integrates external knowledge graphs with Transformer-based visual reasoning for multi-label image classification, achieving state-of-the-art performance on \texttt{MS-COCO} and \texttt{VOC 2007} benchmarks.
 \item We design an \textit{image-conditioned graph attention mechanism} that adaptively refines label relations by combining semantic priors and image-specific visual evidence, enabling context-aware reasoning among correlated labels.
\item We introduce a \textit{knowledge-consistency loss} that enforces semantic alignment between learned attention maps and external knowledge, improving both interpretability and generalization.
\end{itemize}


