\section{Experiments}
\label{sec:experiment} 
% \subsection{Datasets and Evaluation Metrics}
% We evaluate \texttt{KGGAT} on two standard multi-label benchmarks: \texttt{MS-COCO}~\cite{lin2014microsoft} and \texttt{VOC 2007}~\cite{everingham2015pascal}. These datasets are widely adopted in MLIC and provide established evaluation protocols and strong baseline coverage (including Q2L-style models), enabling fair and direct comparison. \texttt{MS-COCO} contains $\sim$120k images across $80$ categories with complex scenes; following~\cite{liu2021query2label,cho2022CoFormer}, we use $82{,}081$ images for training and $40{,}504$ for validation (avg.~$2.9$ labels/image). \texttt{VOC 2007} comprises $9{,}963$ images over $20$ categories, with $5{,}011$ for train/val and $4{,}952$ for test, providing a simpler yet complementary benchmark for modeling label correlations. We limit our empirical claims to these two benchmarks; evaluation on larger, relation-rich datasets such as Visual Genome is an important direction for future work.
% \textbf{Metrics.}
% We report mean Average Precision (\texttt{mAP}), Overall Precision/Recall/F1 (\texttt{OP}/\texttt{OR}/\texttt{OF1}), and Per-class Precision/Recall/F1 (\texttt{CP}/\texttt{CR}/\texttt{CF1}). 
% All models predict independent label probability scores for each label, and the top-3 highest scores are used for fair comparison.

\subsection{Datasets and Evaluation Metrics}
We evaluate \texttt{KGGAT} on two standard MLIC benchmarks: \texttt{MS-COCO}~\cite{lin2014microsoft} and \texttt{VOC 2007}~\cite{everingham2015pascal}, which offer established protocols and strong baseline coverage for fair comparison. \texttt{MS-COCO} contains $\sim$120k images over $80$ categories; following~\cite{liu2021query2label,cho2022CoFormer}, we use $82{,}081$ for training and $40{,}504$ for validation (avg.~$2.9$ labels/image). \texttt{VOC 2007} has $9{,}963$ images over $20$ categories, with $5{,}011$ for train/val and $4{,}952$ for test. We limit our empirical claims to these benchmarks; evaluation on Visual Genome is left for future work.
\textbf{Metrics.}
We report \texttt{mAP}, \texttt{OP}/\texttt{OR}/\texttt{OF1}, and \texttt{CP}/\texttt{CR}/\texttt{CF1}. All models predict independent label probability scores, and the top-3 scores are used for fair comparison.
\subsection{Implementation Details}
\texttt{KGGAT} is implemented in PyTorch and trained on a single \texttt{NVIDIA RTX A6000}. 
We use a \texttt{TResNetL}~\cite{ridnik2021asymmetric} backbone pretrained on \texttt{ImageNet22K}. 
Images are resized to $448{\times}448$ with random crop, flip, and color jitter. 
The Transformer decoder uses two layers with eight attention heads and 512-dim embeddings; label queries match class counts ($C{=}80$ for \texttt{MS-COCO}, $C{=}20$ for \texttt{VOC 2007}). 
The \texttt{GAT} module employs a single multi-head layer, and $\mathbf{S}$ is initialized from \texttt{CN55},\texttt{OA} or \texttt{OA-FU}. 
We use \texttt{AdamW} ($\text{lr}{=}10^{-4}$, weight decay $10^{-4}$, batch size $64$) with cosine scheduling for $80$ epochs. 
The trade-off $\lambda_{KC}$ in Eq.~\ref{eq:total_loss} is set to $0.1$.

\subsection{Comparison with State-of-the-Art}
\subsubsection{Performance on MS-COCO Dataset} 
Table \ref{table:COCO} summarizes the performance of representative multi-label image classification approaches on the \texttt{MS-COCO} dataset.
Earlier graph-based methods such as \texttt{ML-GCN} \cite{chen2019multi}, \texttt{CMA} \cite{you2020cross}, and \texttt{TSGCN} \cite{xu2020joint}  achieve \texttt{mAP} values around $83–84$\%, while attention-based variants (\texttt{MCAR} \cite{gao2021learning}, \texttt{CSRA} \cite{zhu2021residual} ) show limited gains.
Transformer-based frameworks including \texttt{Q2L-R101} \cite{liu2021query2label} and \texttt{MambaML} \cite{zhu2025mambaml} further improve results to about $85–86$\% \texttt{mAP}.
When trained with a stronger backbone and large-scale \texttt{ImageNet22k} pretraining, \texttt{Q2L-TResL} reports $89.20\%$  \texttt{mAP}.


\textit{In our reproduction, using the official configuration (\texttt{TResNetL} backbone, \texttt{ASL} loss, input $448 \times 448$), we obtained $89.08\%$  \texttt{mAP}, which is consistent with the reported value ($\Delta$ = $0.12$) with $120$M parameters.}
Building upon this verified baseline, our proposed \texttt{KGGAT} achieves $89.21\%$  \texttt{mAP}, $84.43\%$  \texttt{CR}, and $86.19\%$ \texttt{OR}, representing the highest overall performance among all compared methods while maintaining comparable model size ($124$M parameters).
The consistent improvement over the reproduced and reported \texttt{Q2L-TResL} baselines demonstrates that incorporating knowledge-guided graph attention further enhances label–feature reasoning beyond conventional transformer attention.

\subsubsection{Performance on VOC 2007 Dataset}
Table~\ref{table:VOC} presents class-wise average precision (\texttt{AP}) and mean average precision (\texttt{mAP}) on the \texttt{~VOC~2007} dataset. 
Earlier graph-based methods such as \texttt{ML-GCN}~\cite{chen2019multi} reach around 94\%,\texttt{mAP}, while transformer-based frameworks including \texttt{C-TMS} \cite{wu2023transformer} and \texttt{MambaML} \cite{zhu2025mambaml}  achieve approximately $96$\% \texttt{mAP}. 
Recent approaches such as \texttt{HSQ-CvN} \cite{yin2024hybrid} ($96.40$\% \texttt{mAP}) and \texttt{Q2L-TResL}~\cite{liu2021query2label} ($96.10$\% \texttt{mAP}) serve as strong baselines. 
Our proposed \texttt{KGGAT} attains {$96.85$\%,\texttt{mAP}}, surpassing all previous methods with consistent gains across several object categories (e.g., \emph{cat}~$99.13$\%, \emph{plane}~$99.97$\%, \emph{bird}~$99.16$\%,
\emph{dog}~$99.62$\%). Although the absolute improvement is modest ($+0.45$\, \texttt{mAP} over \texttt{HSQ-CvN}), it demonstrates that knowledge-guided graph attention provides a stable and generalizable enhancement even on this highly saturated dataset.


\subsection{Ablation Studies of Architectural Components and Loss Functions}

Table~\ref{table:COCO1} summarizes the ablation results of key components and loss functions on \texttt{MS-COCO} and \texttt{VOC 2007}. 
\texttt{KG} denotes knowledge graph integration, \texttt{GCN} the graph convolutional network, \texttt{GAT} the graph attention network, \texttt{ASL} the asymmetric loss, and \texttt{CSL} the contrastive semantic loss. 
All models use the \texttt{TResNetL22k} backbone with $448{\times}448$ input resolution. 
On \texttt{MS-COCO}, combining \texttt{KG}, \texttt{GAT}, and \texttt{ASL} improves reasoning quality, increasing \texttt{mAP} from $89.09$\% to $89.21$\%. 
On \texttt{VOC 2007}, the same setup achieves $96.85$\% \texttt{mAP}. 
Performance consistently rises when \texttt{KG} and \texttt{GAT} are included, confirming the effectiveness of graph-based reasoning for modeling inter-label dependencies. 
The addition of \texttt{ASL} further stabilizes training under class imbalance, complementing the semantic propagation from \texttt{KG} and \texttt{GAT}. 
Overall, their joint contribution significantly enhances semantic reasoning and multi-label prediction accuracy.

\subsection{Ablation Studies of Fusion and Loss Design}

Table~\ref{table:COCO2} shows the effects of fusion strategies and loss settings on \texttt{MS-COCO} and \texttt{VOC 2007}. 
Among Add (+), Multiply (×), and Concat+Linear (CL), the Add strategy performs best, reaching $89.21$\% and $96.85$\% \texttt{mAP}, respectively. 
Element-wise addition enables smoother feature blending and better semantic alignment. 
For losses, \texttt{ASL} handles label imbalance effectively, while \texttt{CSL} enhances semantic discrimination. 
Combining Add fusion with \texttt{ASL} yields the optimal balance between accuracy and robustness in \texttt{KGGAT}.

\subsection{Ablation Studies of Knowledge Sources}

% Table~\ref{table:COCO3} shows the effect of different knowledge sources on GAT-based multi-label classification for the \texttt{MS-COCO} dataset. 
% Three semantic priors are evaluated: \texttt{ConceptNet55} (\texttt{CN55}), OpenAI (\texttt{OA}), and their combined version, OpenAI-Fused Union (\texttt{OA-FU}). 
% Among them, the OA-FU configuration yields the best overall performance, achieving an \texttt{mAP} of $89.17$\% and the highest precision and recall metrics (\texttt{OP} = $83.37$\%, \texttt{OF1} = $84.32$\%, \texttt{CR} = $83.46$\%, \texttt{CP} = $82.95$\%, \texttt{OR} =$ 73.54$\%). 
% These results indicate that the fused-union strategy effectively integrates structured relations from \texttt{CN55} and contextual knowledge from QA, resulting in richer and more balanced semantic connectivity among label embeddings. 
% Consequently, the \texttt{OA-FU} knowledge source enables stronger inter-label reasoning and better generalization within the \texttt{KGGAT} framework.
% Similarly, Table~\ref{table:VOC3} reports the results on \texttt{VOC 2007}, where the \texttt{OA-FU} source further improves the \texttt{mAP} by $0.02$\% ($96.83$\% to $96.85$\%). 
% This confirms that combining human-curated and \texttt{LLM}-derived knowledge enhances the semantic reasoning capability of the GAT model, strengthening inter-label relationships and improving overall generalization.

Table~\ref{table:COCO3} compares knowledge sources on \texttt{MS-COCO}: \texttt{ConceptNet55} (\texttt{CN55}), \texttt{OpenAI} (\texttt{OA}), and their fused version (\texttt{OA-FU}). 
\texttt{OA-FU} achieving an \texttt{mAP} of $89.17$\% and the highest precision and recall metrics (\texttt{OP} = $83.37$\%, \texttt{OF1} = $84.32$\%, \texttt{CR} = $83.46$\%, \texttt{CP} = $82.95$\%, \texttt{OR} =$ 73.54$\%), showing that fusing structured (\texttt{CN55}) and contextual (\texttt{OA}) knowledge yields richer and more balanced semantic connectivity. 
On \texttt{VOC 2007} (Table~\ref{table:VOC3}), \texttt{OA-FU} further improves \texttt{mAP} from 96.83\% to 96.85\%, confirming that combining human-curated and \texttt{OpenAI}-derived knowledge strengthens inter-label reasoning and overall generalization in \texttt{KGGAT}.

% \begin{figure}[htbp]
%     \centering
%     % Row 1
%     \includegraphics[width=0.43\textwidth]{GCa.png}
%     \includegraphics[width=0.43\textwidth]{GCb.png}
%     % \includegraphics[width=0.43\textwidth]{GCc.png}
%     \includegraphics[width=0.43\textwidth]{GCd.png}
%     \includegraphics[width=0.43\textwidth]{GCe.png}
%     \caption{Visualization of attention maps in the \texttt{KGGAT} framework on several examples from the MS-COCO testing set.}
%     \label{fig:kggat_attention}
% \end{figure}


\begin{figure}[htbp]
    \centering
    % ----- Subfigure (a) -----
    \begin{subfigure}[t]{0.43\textwidth}
        \centering
        \includegraphics[width=\linewidth]{GCa.png}
        \includegraphics[width=\linewidth]{GCf.png}
        \caption{\texttt{KGGAT}}
        
        \label{fig:q2l_attn}
    \end{subfigure}
    \hfill
    % ----- Subfigure (b) -----
    \begin{subfigure}[t]{0.43\textwidth}
        \centering
        \includegraphics[width=\linewidth]{GCa1.png}
        \includegraphics[width=\linewidth]{GCf1.png}
        \caption{\texttt{Query2Label}}
        \label{fig:kggat_attn}
    \end{subfigure}
    
\caption{
Qualitative comparison of Grad-CAM heatmaps between 
(a) \texttt{KGGAT} and (b) \texttt{Query2Label}~\cite{liu2021query2label} on \texttt{MS-COCO} examples. 
\texttt{KGGAT}, guided by external knowledge, produces more accurate and context-aware localization, 
correctly identifying the \textit{handbag} ($0.89$) missed by \texttt{Query2Label} ($0.12$) 
and attending to multiple co-occurring objects (\textit{person}, \textit{cake}, \textit{tie}). 
All heatmaps are generated using Grad-CAM and are best viewed in color.
}
    \label{fig:q2l_vs_kggat}
\end{figure}

% \subsection{Discussion}
% \texttt{KGGAT} unifies visual learning and knowledge-guided reasoning with minimal overhead: refinement is performed on the
% label graph ($C{=}80/20$) with top-$k$ neighbors for $T$ steps (Alg.~1), while the backbone dominates runtime. It achieves
% $89.21$\% mAP on \texttt{MS-COCO} and $96.85$\% on \texttt{VOC 2007}, surpassing transformer and graph baselines. The adaptive
% graph attention refines label relations using image cues, and the knowledge-consistency loss preserves semantic validity; the
% fused \texttt{OA-FU} prior yields the best balance between coverage and reliability.
% \noindent\textbf{Relation to multi-modal LLMs.}
% MLLMs (e.g., GPT-4o, Qwen3-VL) target open-world, prompt-based reasoning, whereas \texttt{KGGAT} targets \emph{supervised} MLIC
% with fixed labels ($C{=}80/20$) and mAP/OP/OR; thus we compare against established MLIC baselines under the same protocol.\texttt{KGGAT} is a post-decoder refinement and cannot resolve strong backbone visual ambiguities; it helps most when
% image-supported label relations provide complementary context.

% \subsection{Visualization and Interpretability}

% Figure~\ref{fig:q2l_vs_kggat} compares attention maps of \texttt{KGGAT} and \texttt{Query2Label}~\cite{liu2021query2label}. 
% By fusing visual cues with external knowledge, \texttt{KGGAT} yields sharper and more consistent attention. 
% In the first image, it correctly highlights the \textit{handbag} (0.89) with the \textit{bed} and \textit{chair}, 
% whereas \texttt{Query2Label} misses the handbag ($0.12$). 
% In the second, \texttt{KGGAT} attends to correlated regions—\textit{person} ($1.00$), \textit{cake} ($1.00$), \textit{tie} ($0.98$)—while \texttt{Query2Label} weakly responds to the \textit{tie} ($0.21$). 
% These results show that knowledge guidance improves both precision and interpretability.
% \texttt{KGGAT} learns spatially coherent, semantically grounded attention: its image-conditioned graph aligns labels with relevant regions, while knowledge-consistency regularization preserves commonsense relations—enhancing transparency and reasoning in multi-label prediction.

\subsection{Discussion and Interpretability}

\texttt{KGGAT} combines visual learning and knowledge-guided reasoning with limited overhead, since refinement operates only on the label graph ($C{=}80/20$) with top-$k$ neighbors for $T$ steps, while the backbone remains the main runtime cost. It achieves $89.21$\% mAP on \texttt{MS-COCO} and $96.85$\% on \texttt{VOC 2007}, outperforming transformer- and graph-based baselines. Adaptive graph attention refines label dependencies using image cues, while the knowledge-consistency loss preserves semantic validity. The fused \texttt{OA-FU} prior provides the best trade-off between coverage and reliability.

\noindent\textbf{Relation to multi-modal LLMs.}
Unlike multi-modal LLMs (e.g., GPT-4o, Qwen3-VL), which address open-world prompt-based reasoning, \texttt{KGGAT} targets \emph{supervised} MLIC with fixed labels and standard metrics (mAP/OP/OR). Thus, we compare against established MLIC baselines under the same protocol. As a post-decoder refinement module, \texttt{KGGAT} cannot fully correct strong backbone ambiguities; it is most effective when label relations provide useful contextual cues.

\noindent\textbf{Visualization and interpretability.}
Figure~\ref{fig:q2l_vs_kggat} shows that \texttt{KGGAT} produces sharper and more consistent attention than \texttt{Query2Label}~\cite{liu2021query2label}. In one example, it correctly highlights the \textit{handbag} (0.89), which \texttt{Query2Label} misses ($0.12$). In another, it better captures correlated regions for \textit{person} (1.00), \textit{cake} (1.00), and \textit{tie} (0.98), while \texttt{Query2Label} responds weakly to the \textit{tie} ($0.21$). These results show that knowledge guidance improves both prediction quality and interpretability.
Overall, \texttt{KGGAT} learns spatially coherent, semantically grounded attention by aligning labels with relevant regions while preserving commonsense relations.


% \begin{figure}[t]
%   % \centering

%   % === Column headers ===
%   % \begin{tabular}{cc}
%   %   \textbf{Knowledge Source} & \textbf{Output} \\
%   % \end{tabular}
%   % \\[4pt]

%   % === First row ===
%   \subfloat[ConceptNet-55]{%
%     \includegraphics[width=0.22\textwidth]{CN55.png}}
%   \hfill
%   \subfloat[Output from ConceptNet-55]{%
%     \includegraphics[width=0.22\textwidth]{kg1.jpg}}
%   \\[6pt]

%   % === Second row ===
%   \subfloat[OpenAI Fused Union (OA-FU)]{%
%     \includegraphics[width=0.22\textwidth]{hybrid.png}}
%   \hfill
%   \subfloat[Output from OA-FU]{%
%     \includegraphics[width=0.22\textwidth]{kg2.jpg}}

%   \caption{Comparison between ConceptNet-55 and OpenAI Fused Union (OA-FU) knowledge sources and their corresponding multi-label image classification outputs on complex scenes. Each pair shows the knowledge graph (left) and the corresponding image-level prediction (right).}
%   \label{fig:knowledge_output}
% \end{figure}
