\input{tab/sota}

\section{Experiments}
\label{sec:experiments}

In this section, details on the datasets employed along with implementation settings are presented. We evaluate our proposed method on standard few-shot benchmarks and compare results with recent state-of-the-art methods. 

\paragraph{Datasets} To verify the proposed method, we address few shot classification under two scenarios: generic object recognition and fine-grained image classification. For this purpose, we employ the \textit{mini}ImageNet~\cite{Ravi2017OptimizationAA,deng2009imagenet}, \textit{tiered}ImageNet~\cite{ren2018meta}, CIFAR-FS~\cite{bertinetto2018meta}, and CUB-200-2011~\cite{wah2011caltech} datasets. 

\textbf{\textit{mini}ImageNet} consists of a subset of 100 object classes from ImageNet~\cite{deng2009imagenet} with 600 images per class. We follow the setting proposed by Ravi \etal\cite{Ravi2017OptimizationAA} to randomly select $64$ base, $16$ validation, and $20$ novel classes, respectively. \textbf{\textit{tiered}ImageNet} is a larger subset of ImageNet with 351/97/160 sub-classes for training/validation/testing stemming from 20/6/8 super-classes. It is a more challenging dataset as the splits are disjoint in terms of super-classes and requires better generalization. \textbf{CUB-200-2011} consists of $200$ classes with a total of $11,788$ images. Following the protocols of Hilliard \etal\cite{hilliard2018few}, the dataset is randomly split into $100$ base, $50$ validation, and $50$ novel classes. \textbf{CIFAR-FS} is built upon CIFAR-100 and has $100$ classes, each containing $600$ images. We use the same split of 64/16/20 as in Bertinetto \etal\cite{bertinetto2018meta}.

\paragraph{Implementation Details} Following recent works~\cite{zhang2020deepemd,kang2021relational,Hou2019CrossAN}, we adopt ResNet12~\cite{he2016deep} as the backbone network. The backbone take images of size $84\times84$ as input and produces embeddings $z_{q,s} \in \mathbb{R}^{640}$ after global pooling. We train our model in the `$5$-way $1$-shot' and `$5$-way $5$-shot' setting with standard normalization and augmentation techniques as in prior work \cite{kang2021relational}. Moreover, our attention block is shared between adaptation steps and only uses a single head (h=1) with dimensions of the fully-connected layers set to $640$, and $d(\cdot{})$ is the euclidean function. In training, Adam optimizer was used with a learning rate of $0.003$ without decay or scheduling. $1$-shot models were trained for $300$ epochs with $200$ tasks sampled per epoch, resulting in $60,000$ tasks. On the other hand, $5$-shot models were trained for $200$ epochs totaling $40,000$ tasks similar to Chen \etal\cite{chen2019closer}. During evaluation, we meta-test 15 query samples per class in each episode and report the average accuracy with 95\% confidence interval over $2,000$ randomly sampled test episodes.  


\input{tab/sota2}
\paragraph{Comparison with State-of-the-Art Methods} Tables \ref{tab:1} and \ref{tab:2} compare the proposed method with current state-of-the-art few-shot methods \cite{kang2021relational,wertheimer2021few,Hou2019CrossAN,zhang2020deepemd,ye2020few}. Our approach shows consistent improvements over several methods in all evaluated settings. Notably, we observe significant gains in both $1$- and $5$-shot settings, highlighting the effectiveness of our approach despite its simplicity. For example, on \textit{mini}ImageNet, our approach shows $+3\%$ and $+4\%$ gain in $1$-shot and $5$-shot settings over the best method. We observed similar results on the challenging \textit{tiered}ImageNet with $+1\%$ gains, except for CUB where our approach shows results on par with current state-of-the-art methods. While closely related method RENet~\cite{kang2021relational} employs spatial relational learning of query/support features, our technique solely benefits from discriminative co-adaptation to enhance performance. Recent work FRN~\cite{wertheimer2021few} equally shows competitve performance, however FRN reformulates few-shot classification to leverage reconstruction of related features, and requires a two-stage process with pre-training followed by episodic fine-tuning. In contrast, our approach can be trained end-to-end from scratch with episodic training.

As shown in Table \ref{tab:2}, our method is highly competitive to the closely related method FEAT \cite{ye2020few}. Note that FEAT is another transfer learning method leveraging self-attention on support samples only, and uses extra regularization terms in the episodic training step. Herein, the results validate our initial hypothesis that discriminative learning alone can still show gains over existing baselines. On the other hand, while our approach does not report the best result on CUB-200, we are on par with the best method FRN and produce the best performance on CIFAR-FS ($\approx 5\%$). In addition, while approaches such as Boosting~\cite{gidaris2019boosting} use self-supervision and auxiliary losses with extra unlabeled samples for semi-supervised few-shot learning, we show that a smaller backbone is still competitive. 


