\section{Introduction}
\label{sec:intro}
% 1
Deep learning has achieved remarkable success in numerous computer vision tasks, reaching human-level performance in domains with sufficiently large-scale labeled training data. However, large-scale data is not always readily available due to costly curation and labeling. Hence, there is a ongoing research effort to design models that can learn to solve tasks using a limited number of labeled examples to alleviate this requirement. While several approaches such as transfer-, semi- and unsupervised learning have shown reasonable performance, learning to generalize with an extremely limited number of labeled samples is still a challenge. Motivated by the ability of humans to rapidly adapt to new tasks using prior knowledge, the field of few-shot learning shows promise in achieving this goal. 
% 2
\input{fig/teaser}

Few-shot learning~\cite{koch2015siamese, Ravi2017OptimizationAA, Snell2017PrototypicalNF} aims to classify unlabeled query samples from unseen classes using only limited labeled support examples. Recent approaches have leveraged the meta-learning paradigm where transferable knowledge across a collection of tasks is learned and propagated to improve generalization. In particular, one of the promising approaches have used the meta-learning framework to optimize model parameters~\cite{Finn2017ModelAgnosticMF, Ravi2017OptimizationAA, nichol2018first} with a few gradient steps, thus enabling neural network classifiers to quickly adapt to unseen classes. Other works employ similarity information between images (\Fig{teaser}(a)), augmentation of training data with generative techniques~\cite{Hariharan2017LowShotVR,Schwartz2018DeltaencoderAE}, or two-stage approaches~\cite{chen2019closer, raghu2019rapid, yang2020free, shen2021partial} that involve first pre-training the model on existing known classes and later use a meta-finetuning strategy. In the general meta-setup, tasks defined in the training phase mimic the testing phase to encourage generalization of models. 
% 3

Though pre-training strategies show promising results, Chen \etal\cite{chen2019closer} argued that fine-tuning shows marginal improvements. Moreover, the standard technique of obtaining a base representation with global-average pooling is considered limited, since it is sensitive to object pose and discards key semantic details, and may be difficult to learn generalizable embeddings without being distracted by spurious features~\cite{kang2021relational,hong2021reinforced}. Thus, recent works focus on learning finer distinctions using spatial features only via spatial attention mechanisms~\cite{Hou2019CrossAN} or other forms of relational learning~\cite{wertheimer2021few,kang2021relational} between query and support features to enhance performance. Nevertheless, methods leveraging this setting \ie, spatial feature-based learning, still report marginal improvements over purely discriminative approaches with the exception of some works that provide insight into the model reasoning procedure by visualizing object attended regions for interpretability.
% 4

Here, we argue that meta-learning with instance embeddings only is still viable and can improve few-shot performance when attending to the relevant features. For example, Ye \etal\cite{ye2020few} attempt to address this and show that optimally discriminative features can be meta-learned using a set-to-set function based on the transformer model~\cite{vaswani2017attention} (\Fig{teaser}(b)). The self-attention mechanism in the Transformer presents several desirable properties for set based problems \ie, permutation invariance, interpolation, and contextualization. As a consequence, any meta-learner can leverage such a set-based transformation to adapt instance embeddings relative to other classes for improved prototype generation, or to produce better embeddings. However, while Ye \etal\cite{ye2020few} showed improvements, embedding adaptation is performed on support samples only without leveraging query information, a key distinction with recent spatial techniques using cross-attention~\cite{wertheimer2021few,Hou2019CrossAN,kang2021relational} to learn better correspondences. Moreover, the authors use a transfer-based approach requiring pre-training on base classes before meta-training.      

In this work, we propose an end-to-end meta-learning strategy trained from scratch to co-attend to both support and query features using self-attention (\Fig{teaser}(c)). This is useful to focus on common object features and avoid distractors for better matching. In particular, we introduce a single shared attention module that first projects global pooled base representations and computes attention scores per $k$-shot task via scaled-dot product. Given the scores per shot, we take the mean of the scores to obtain an attention pooled feature (dot product of features and scores) that is later added to the initial projected features before producing the final re-weighted features. Concretely, to improve support features \ie, \textit{query} to \textit{support}, attention scores between query and support features are employed to pool the initial support features producing a support prototype concatenated with the initial features, and vice versa for \textit{support} to \textit{query} to produce a query prototype that improves the initial query features. This strategy is conceptually equivalent to spatial based cross-attention that implicitly re-weights the base spatial maps to attend to relevant object regions. Finally, we employ a nearest neighbor classifier on the refined features for few-shot classification. The contributions of this paper can be summarized as follows:

\begin{itemize}[leftmargin=*]
\setlength\itemsep{-.3em}
\item To improve model-based embedding adaptation, we propose to cross-attend support and query embeddings by re-weighting each instance relative to the other via self-attention.
\item We reveal that implicitly re-weighting features with their prototypes (support/query) via self-attention improves metric based few-shot performance and adds minimal over-head in learnable parameters.
\item Extensive evaluation of the proposed components verify the effectiveness of our method. Moreover, we show competitive results over state-of-the-art methods on several benchmark datasets.
\end{itemize}
