# PROTOTYPICAL CONTRASTIVE LEARNING FOR IMPROVED FEW SHOT AUDIO CLASSIFICATION
## Abstract
Few-shot learning has emerged as a powerful paradigm for training models with limited labeled data, addressing challenges in scenarios where large-scale annotation is impractical. While extensive research has been conducted in the image domain, few-shot learning in audio classification remains relatively underexplored. In this work, we investigate the effect of integrating supervised contrastive loss into prototypical few shot training for audio classification. In detail, we demonstrate that angular loss further improves the performance compared to the standard contrastive loss. Our method leverages SpecAugment followed by a self-attention mechanism to encapsulate diverse information of augmented input versions into one unified embedding. We evaluate our approach on MetaAudio, a benchmark including five datasets with predefined splits, standardized preprocessing, and a comprehensive set of few-shot learning models for comparison. The proposed approach achieves state-of-the-art performance in a 5-way, 5-shot setting.
Index Terms — Few shot, audio classification, contrastive learning,
## 1 Introduction
In today’s rapidly evolving fields of machine learning and artificial intelligence, there is a growing demand for models that can generalize effectively from limited training data. Traditional machine learning algorithms typically depend on large amount of labeled data to achieve high performance, making them less effective in real-world scenarios where such data are often scarce or difficult to obtain. In contrast, Few-Shot Learning (FSL) focuses on enabling models to achieve high performance with only a few labeled examples. This approach is especially valuable in scenarios where the data is limited or unevenly distributed, allowing models to quickly adapt to new tasks with minimal prior information.
Existing methodologies in few-shot learning can be broadly categorized into metric learning and optimization-based methods. Metric learning approaches utilize well-defined similarity measures to compare query samples with support examples, enabling classification based on embedding proximity. These methods often use techniques such as ProtoNets and MatchingNets [1, 2], to learn discriminative embeddings by utilizing various distance metrics. On the other hand, optimization-based methods focus on efficiently adapting model parameters based on limited data, leveraging meta-learning strategies such as gradient-based updates [3, 4] or task-specific adaptation mechanisms [5].
Although extensive research has been conducted in the image domain, few-shot learning in the audio domain remains relatively underexplored. Several studies focus on the task of sound event detection, aiming to identify specific acoustic events in an audio file using only a few examples [6, 7]. In the context of few shot audio classification, researchers have leveraged Prototypical Networks for the tasks of speaker recognition [8] and sound event classification [9]. Chou et al. [10] incorporate an attention-based similarity mechanism into metric learning architectures to effectively match transient sound events. On the other hand, Zhang et al. [11] use attentional graph neural networks for the same task. To address the lack of a standard benchmark in audio few-shot learning, MetaAudio [12] evaluates the most widely used few-shot learning algorithms across five publicly available datasets. Additionally, it provides predefined train-test splits, ensuring consistency in data preparation, partitioning, and backbone feature extraction.
Contrastive learning has recently gained significant attention for its effectiveness in learning robust representations. The idea is to minimize an appropriate distance metric to cluster together augmented versions (positives) of the input while distinguishing them from other samples (negatives) on the embedding space [13]. Building on this foundation, Khosla et al. proposed a supervised variation of the contrastive loss [14] extending the original formulation by leveraging label information to pull together samples from the same class while pushing apart those from different classes. Wang et al. [15] introduced the angular loss, which improves the traditional triplet loss by enforcing a stricter angular margin between positive and negative pairs, leading to more discriminative feature representations. In few-shot image classification, several works have employed contrastive learning as an auxiliary objective [16], while others have integrated it directly into the training phase of few-shot models [17, 18]. However, in the audio domain, the integration of contrastive learning into few-shot learners remains unexplored.
To address this gap, the present work builds upon the architecture proposed in [18], with the aim of developing a model specifically designed for audio classification. Our approach introduces key modifications, including the integration of SpecAugment method for spectrogram augmentations [19] and the replacement of the original contrastive loss with angular loss, in order to evaluate its effectiveness. For our experiments, the MetaAudio benchmark [12] is utilized to validate our approach in a 5-way, 5-shot setting across five audio datasets (ESC-50, FSDKaggle2018, VoxCeleb1, Nsynth, BirdClef2020). To facilitate the evaluation of the proposed method, the performance of the models included in the benchmark (ProtoNets [1], MAML [3], MAML-Curvature [4]) is also being measured. Overall, the contribution of the present work holds as follows:
- To the best of our knowledge, this is the first work to combine few-shot loss with supervised contrastive loss for audio classification in a few-shot setting.
- By replacing contrastive loss with angular loss, the proposed method achieves state-of-the-art results on the majority of datasets. This enables a straightforward approach such as ProtoNets to achieve competitive results compared to optimization-based algorithms, without requiring gradient updates during inference.
Finally, it is worth highlighting that this work promotes reproducibility, by thoroughly documenting all stages of our methodology and ensuring that the dataset splits, pre-processing steps, and backbone models align with those proposed in the MetaAudio benchmark. In the following GitHub repository https://github.com/magcil/audio-few-shot-learning we provide instructions to reproduce the experiments, allowing further experimentation with our approach.
## 2 Methods
### 2.1 Few Shot Classification Setting
Let $D=≤ft\{(x_i,y_i)\right\}_i=1^N$ be a collection of samples and labels. Denote by $C$ the set of all classes. Let $C=C_train∪ C_val∪ C_test$ be a partition of $C$ . Define $D_train=\{(x_i,y_i)\mid y_i∈ C_train\}$ , and $D_val,D_test$ , accordingly. The goal of FSL is to recognize samples from new categories by leveraging knowledge from the base training set $D_train$ . To achieve this, FSL typically employs an episodic training strategy. In detail, $n$ classes $C_n⊂ C_train$ , and a small number of $k+q$ samples per class are sampled from $D_train$ to form the support set $S=\{(x^s_i,y^s_i)\mid y^s_i∈ C_n,i=1,…,n× k\}$ and the query set $Q=\{(x^q_i,y^q_i)\mid y^q_i∈ C_n,i=1,…,n× q\}$ . Together $S$ and $Q$ form an episode where $S∩ Q=∅$ . During training, episodes are randomly sampled from $D_train$ ; the support set provides labeled examples used as a reference for learning, while the query set consists of unlabeled samples from the same classes. The model predicts labels for the query set based on the support set, and the loss is computed using these predictions. During inference, episodes are randomly sampled from $D_test$ containing previously unseen classes. The model uses only a few labeled examples per class from the support set to predict the labels of query examples.
### 2.2 Architecture
Our architecture is based on [18], with several modifications for few-shot audio classification. The input data, instead of images, consists of single-channel mel spectrograms $x$ of shape $F× T$ , where $F$ are the frequency bins, and $T$ the time bins. The model architecture consists of four main modules: the Augmentation Module, which generates three augmented versions of the original input; the Embedding Module where features of each input and its augmentations are computed; the FSL Module, where ProtoNets are used to compute the few-shot learning loss; and the Contrastive Module, which applies two versions of supervised contrastive loss for improved representation separation.
Augmentation Module (AM): To enrich the few shot batch with more information, we augment every input spectrogram $x_i^s,x_i^q$ by using time masking, frequency masking and time warping, augmentation techniques proposed by SpecAugment [19]. Time masking and frequency masking randomly select contiguous segments along the time and frequency axes, respectively, and mask them by setting the corresponding values to zero. Time warping applies random warping along the time axis by stretching or compressing time intervals, which helps to create more robust features. The augmentations are performed separately on each spectrogram, resulting in a list of four spectrograms: the original spectrogram and one for each applied augmentation $x_l_{i}=(x_i^orig,x_i^aug_1,x_i^aug_2,x_i^aug_3)=AM(x_i)$ .
Embedding Module (EM): After the use of the AM, $x_l_{i}$ passes through the Embedding Module which is composed by a feature extraction network and a self attention module. The feature extraction network $f_θ:ℝ^F× T→ℝ^D$ is a CRNN network with parameters $θ$ that projects each element of $x_l_{i}$ to the D-dimensional feature space :
$$
\tilde{x}_l_{i}=[f_θ(x_i^orig),f_θ(x_i^aug_1),f_θ(x_i^aug_2),f_θ(x_i^aug_3)] \tag{1}
$$
The self attention module $A_φ:ℝ^4× D→ℝ^4D$ handles $\tilde{x_l_{i}}$ as a sequence and concatenates its output to a feature $\tilde{x}_i$ of dimension 4D:
$$
\tilde{x}_i=A_φ([f_θ(x_i^orig),f_θ(x_i^aug_1),f_θ(x_i^aug_2),f_θ(x_i^aug_3)]) \tag{2}
$$
Few Shot Module (FSM): Having both support and query inputs passed through the previous modules we get :
$$
\displaystyle\tilde{S} \displaystyle=\{(\tilde{x}_i^s=EM(AM(x_i^s)),y_i^s)\mid x_i^s∈ S\} \displaystyle\tilde{Q} \displaystyle=\{(\tilde{x}_i^q=EM(AM(x_i^q)),y_i^q)\mid x_i^q∈ Q\} \tag{3}
$$
We compute class prototypes using the features in $\tilde{S}$ as shown in (4).
$$
\tilde{p}_c=\frac{1}{k}∑_\tilde{x_i,y_i∈\tilde{S}}\tilde{x}_i· I(y_i=c), \tag{4}
$$
where $I$ denotes the indicator function, returning 1 if the given condition is true, and 0 otherwise. With the class prototypes computed, we follow the approach of prototypical networks by calculating the Euclidean distance $d$ between query samples and the prototypes, and use (5) to compute the few shot loss $L_fs$ .
$$
L_fs=\frac{1}{q}∑_\tilde{x_i,y_i∈\tilde{Q}}-\log{\frac{\exp{(-d(\tilde{x}_i,\tilde{p}_y_{i})})}{∑_c∈ C_{n}\exp{(-d(\tilde{x}_i,\tilde{p}_c))}}} \tag{5}
$$
Contrastive Module (CM): To further improve representation separation in the embedding space, we employ two variations of the supervised contrastive loss. We begin by modifying the supervised contrastive prototype loss (CPL) [18] by projecting both prototypes and query features through the projection head. Additionally, we employ Angular Loss, which optimizes the angular separation between embeddings rather than relying solely on Euclidean distances. In detail, the prototypes $\tilde{p}$ , and the query features $\tilde{x}^q$ are passed through a small neural network $h_β:ℝ^4D→ℝ^D^{\prime}$ with parameters $β$ , followed by a normalization $\hat{p}=\frac{h(\hat{p})}{≤ft\lVert h(\hat{p})\right\rVert_2},\hat{x}^q=\frac{\tilde{x}^q}{≤ft\lVert\tilde{x}^q\right\rVert_2}$ . The projection network $h_β$ allows us to experiment with various embedding dimensions $D^\prime$ and choose the most suitable one for minimizing the final loss. We denote by $\hat{P}=\{\hat{p}_1,\dots,\hat{p}_n\}$ , and $\hat{Q}=\{\hat{x}^q_1,\dots,\hat{x}^q_qn\}$ , the sets of the projected prototypes, and queries, respectively.
Contrastive Prototype Loss (CPL): The supervised contrastive loss uses the prototypes $\hat{p}_c$ , $c∈ C_n$ as anchors with queries ${\hat{x}^q_c}$ of the same label forming the positive set $P_c$ . We randomly sample $m$ queries from labels different from $c$ to construct the negative set $N_c$ . The loss is formulated as:
$$
L_cpl=\frac{1}{nq}∑_c∈ C_{n}∑_(\hat{x^q_i,y_i)∈ P_c}-\log{\frac{sim_c,i^(+)}{sim_c,i^(+)+sim_c,i^(-)}}, \tag{6}
$$
where
$$
\displaystyle sim_c,i^(+) \displaystyle=\exp\frac{⟨\hat{p_c},\hat{x}_i,c^q⟩}{T}, sim_c,i^(-) =∑_(\hat{x^q_t,y_t)∈ N_c}\exp\frac{⟨\hat{p}_c,\hat{x}_t^q⟩}{T} \tag{7}
$$
Angular Prototype Loss (APL): While the CPL loss focuses on optimizing the similarity of prototypes and query pairs, the angular loss originaly proposed by [15] aims at constraining the angle at the negative point of triplet (anchor, positive, negative) triangles. Given a triplet $(x_a,x_p,x_n)$ the formulation of angular loss on a few shot batch $B=\hat{P}∪\hat{Q}$ is given by:
$$
\displaystyle L_apl(B)=\frac{1}{n(q+1)}∑_x_{α∈B}\bigg{\{}\log\bigg{[}1+∑_\begin{subarray{c}x_n∈B\\
y_n≠ y_a,y_p\end{subarray}}\exp≤ft(f_a,p,n\right)\bigg{]}\bigg{\}}, \tag{8}
$$
where $f_a,p,n$ is defined as
$$
f_a,p,n=4\tan^2α⟨ x_a+x_p,x_n⟩-2(1+\tan^2α)⟨ x_a,x_p⟩. \tag{9}
$$
The angle $α≥ 0$ in 9 is a predefined upper bound. The idea of angular loss is to minimize the tangent $\tan\angle n^\prime=\frac{≤ft\lVert x_m-x_c\right\rVert}{≤ft\lVert x_n-x_c\right\rVert}$ , where $x_c$ is the middle point of $x_n,x_p$ . The point $x_m$ is one of the two points belonging on the intersection of the circle with radius $≤ft\lVert x_m-x_c\right\rVert=\frac{1}{2}≤ft\lVert x_p-x_a\right\rVert$ , centered at $x_c$ , and the hyperplane which is perpendicular to the edge $x_n-x_c$ , passing through $x_c$ . Minimizing 9 brings $x_p,x_a$ closer on the embedding space, while pushing away the negative point $x_n$ . In our case, we minimize the loss $L_total=L_fs+λ L_cm$ , where $L_cm∈\{L_cpl,L_apl\}$ , and $λ$ is a scaling factor. On inference time, the prototypes are derived from the set $\tilde{S}$ , and the queries from $\tilde{Q}$ are classified based on their proximity to these prototypes, as in standard ProtoNets.
## 3 Experiments
### 3.1 Datasets
We follow a methodology similar to [12], adopting the same preprocessing steps and splits for the five proposed datasets. We also reproduce the experiments of the models presented in [12] in a 5-way, 5-shot setting, under which our approach operates. ESC-50 is an environmental sound classification dataset with 2,000 clips, covering 50 different categories. FSD2018 is designed for sound event detection, featuring over 11,000 clips from 41 classes aligned with the AudioSet ontology. For musical audio, NSynth provides over 300,000 clips from 1,006 instruments, valuable for instrument recognition tasks. BirdCLEF 2020 is a bioacoustic dataset for bird species classification, offering over 80,000 recordings from 960 species. We used a pruned version of BirdCLEF 2020, removing samples longer than 180 seconds and classes with fewer than 50 samples. Finally, VoxCeleb1 serves as a speaker recognition dataset, containing utterances from various speakers in real-world conditions with background noise. We had access to a subset of VoxCeleb1, comprising 60,184 utterances from 1,246 distinct speakers, and by removing speakers with fewer than 20 recordings, we obtained 57,737 utterances from 928 speakers.
### 3.2 Experimental Setup
Audio samples from all datasets are loaded at a 16 kHz sample rate and converted to mel spectrograms. For datasets with variable-length samples (VoxCeleb1, FSD2018, BirdCLEF2020), we generate 5-second segments, as described in [12]. We apply global standardization to all spectrograms by computing the mean, and std from each training set. In all cases, the backbone is a CRNN network, consisting of a 4-block convolutional network (1-64-64-64) followed by a 1-layer non-bidirectional RNN with 64 hidden units. We train and evaluate the proposed architecture along with the ProtoNets and the optimization based models (MAML, and MAML-Curvature) presented in [12], in a 5-way, 5-shot setting. We repeat each experiment five times and report the average accuracy and the 95% confidence interval. In our approach, we employ a single-headed self-attention mechanism with a feedforward dimension of 256. The input is a sequence of $4× D$ , where $D=64$ . The output sequence is concatenated to a $256$ -dimensional embedding. We use a projection head consisting of two linear layers with hidden and output dimensions finetuned for each dataset. We conduct experiments in two different settings. In the first setting, we combine the few shot loss with the contrastive prototype loss such that $L_total=L_fs+λ L_cpl$ . We denote this setting by FS+CPL. In the second setting, denoted by FS+APL we combine the few shot loss with the angular loss, i.e., $L_total=L_fs+λ L_apl$ . We compute the $L_apl$ loss, either restricting anchors to prototypes from the support set or allowing both prototypes and queries to act as anchors. We train our models for 100 5-shot 5-way episodes per epoch over 200 epochs. We use ADAM as the optimizer, and MultiStepLR as the scheduler. We evaluate the best performing model on the validation set over 2,000 randomly sampled 5-way, 5-shot tasks from the test set. We also compare the performance of FS+CPL and FS+APL with plain ProtoNets for different number of shots (i.e., 1, 3, 5, and 7 shots). All runs were performed on an NVIDIA 4090 GPU.
For FS+CPL we use Optuna [20] to determine the optimal training hyperpameters (i.e., $lr$ , $γ$ , $λ$ , $T$ and $m$ ) based on the performance on the validation set, separately for each dataset. For the FS+APL setting, we use the same values for $lr$ and $γ$ as in the FS+CPL setup. We observe that the large values of the APL loss, compared to CPL loss, lead to increased variance among the results. We empirically find that setting $λ$ to a small value counteracts this effect. For this reason, we use $λ=0.3$ for all datasets. For the calculation of the angular loss, we use the PyTorch Metric Learning https://kevinmusgrave.github.io/pytorch-metric-learning/ implementation. The construction of the triplets is handled by the AngularMiner where a predetermined angle threshold $α$ filters-out triplets with angle less than $α$ , feeding harder samples to the final loss. We applied the same value of $α$ for both the AngularMiner and the angular loss during each experiment, testing four different angles : 0°, 15°, 30°, and 45°. We adopt two different approaches. In the first, similar to the CPL, we use only the prototypes from the support set as anchors. In the second setting, any of the prototypes or queries can serve as anchors. We report the results of the best combination of angle, and anchor-approach for each dataset.
### 3.3 Results
| ProtoNets MAML MAML+Curv | 83.52 $±$ 0.39 87.80 $±$ 0.35 88.14 $±$ 0.30 | 54.19 $±$ 0.43 59.35 $±$ 0.43 57.22 $±$ 0.48 | 97.72 $±$ 0.17 96.73 $±$ 0.21 98.21 $±$ 0.13 | 71.14 $±$ 0.48 72.54 $±$ 0.48 74.30 $±$ 0.48 | 75.59 $±$ 0.48 73.57 $±$ 0.43 75.94 $±$ 0.43 |
| --- | --- | --- | --- | --- | --- |
| FS+CPL | 84.23 $±$ 0.35 | 58.2 $±$ 0.43 | 97.86 $±$ 0.17 | 74.95 $±$ 0.48 | 79.21 $±$ 0.48 |
| FS+APL | 85.61 $±$ 0.35 | 59.43 $±$ 0.43 | 97.94 $±$ 0.17 | 75.71 $±$ 0.48 | 79.78 $±$ 0.43 |
| APL setting | $α$ = 15°( $\checkmark$ ) | $α$ = 30°( $×$ ) | $α$ = 15°( $×$ ) | $α$ = 15°( $×$ ) | $α$ = 0°( $\checkmark$ ) |
Table 1: Performance comparison of different methods across datasets. The average accuracy and 95% confidence interval in five runs is reported. For the FS+APL setting, we report the optimal angle threshold $α$ , and whether only prototypes are used as anchors $(\checkmark)$ or both prototypes and query set representations are used as anchors ( $×$ ).
<details>
<summary>Shots_subplt_scaled.png Details</summary>
### Visual Description
## Line Graphs: Accuracy vs. Shots Across Datasets
### Overview
The image contains five line graphs comparing the performance of three machine learning methods (ProtoNet, FS+CPL, FS+APL) across five audio classification datasets (FSD2018, Nsynth, ESC-50, BirdClef, VoxCeleb). Each graph plots accuracy (y-axis) against the number of training shots (x-axis, 1–7). The graphs use distinct line styles to differentiate methods: dashed (ProtoNet), solid (FS+CPL), and dotted (FS+APL).
---
### Components/Axes
- **X-axis**: "Shots" (1–7), representing incremental training data samples.
- **Y-axis**: "Accuracy" (ranging dataset-specifically, e.g., 0.35–0.65 for FSD2018, 0.93–0.98 for Nsynth).
- **Legend**: Located in the bottom-right corner of each graph, with three entries:
- **Dashed line**: ProtoNet
- **Solid line**: FS+CPL
- **Dotted line**: FS+APL
---
### Detailed Analysis
#### FSD2018
- **ProtoNet (dashed)**: Starts at ~0.43 (1 shot), rises sharply to ~0.55 (3 shots), then plateaus at ~0.58 (7 shots).
- **FS+CPL (solid)**: Begins at ~0.40 (1 shot), surpasses ProtoNet by ~0.55 (3 shots), and reaches ~0.60 (7 shots).
- **FS+APL (dotted)**: Starts at ~0.42 (1 shot), follows a similar trajectory to FS+CPL but with minor fluctuations, ending at ~0.59 (7 shots).
#### Nsynth
- **ProtoNet (dashed)**: Begins at ~0.93 (1 shot), climbs steeply to ~0.97 (3 shots), and stabilizes at ~0.98 (7 shots).
- **FS+CPL (solid)**: Starts at ~0.94 (1 shot), matches ProtoNet by ~0.97 (3 shots), and peaks at ~0.98 (7 shots).
- **FS+APL (dotted)**: Initiates at ~0.94 (1 shot), slightly lags behind FS+CPL but converges to ~0.97 (7 shots).
#### ESC-50
- **ProtoNet (dashed)**: Starts at ~0.70 (1 shot), rises to ~0.77 (3 shots), and plateaus at ~0.80 (7 shots).
- **FS+CPL (solid)**: Begins at ~0.72 (1 shot), outperforms ProtoNet by ~0.82 (3 shots), and reaches ~0.85 (7 shots).
- **FS+APL (dotted)**: Starts at ~0.73 (1 shot), follows FS+CPL closely but ends at ~0.83 (7 shots).
#### BirdClef
- **ProtoNet (dashed)**: Begins at ~0.55 (1 shot), climbs to ~0.65 (3 shots), and stabilizes at ~0.70 (7 shots).
- **FS+CPL (solid)**: Starts at ~0.58 (1 shot), surpasses ProtoNet by ~0.72 (3 shots), and peaks at ~0.75 (7 shots).
- **FS+APL (dotted)**: Initiates at ~0.57 (1 shot), matches FS+CPL by ~0.73 (3 shots), and ends at ~0.76 (7 shots).
#### VoxCeleb
- **ProtoNet (dashed)**: Starts at ~0.60 (1 shot), rises to ~0.70 (3 shots), and plateaus at ~0.75 (7 shots).
- **FS+CPL (solid)**: Begins at ~0.62 (1 shot), outperforms ProtoNet by ~0.77 (3 shots), and reaches ~0.80 (7 shots).
- **FS+APL (dotted)**: Starts at ~0.61 (1 shot), follows FS+CPL closely but ends at ~0.78 (7 shots).
---
### Key Observations
1. **FS+CPL consistently outperforms ProtoNet** across all datasets, with the largest gap in Nsynth (~0.05 accuracy difference at 7 shots).
2. **FS+APL matches or slightly underperforms FS+CPL** in most cases, except in BirdClef and VoxCeleb, where it converges to similar accuracy levels.
3. **ProtoNet shows diminishing returns** after 3–4 shots in all datasets, while FS+CPL and FS+APL continue improving.
4. **Nsynth achieves near-perfect accuracy** (~0.98) for all methods by 7 shots, indicating high dataset quality or method suitability.
---
### Interpretation
The data suggests that **FS+CPL is the most effective method** for few-shot learning across diverse audio datasets, likely due to its robust feature learning or regularization. FS+APL performs comparably but may introduce unnecessary complexity in some cases. ProtoNet, while simpler, struggles with generalization beyond 3–4 shots. The convergence in VoxCeleb implies that all methods may plateau at high accuracy with sufficient data, highlighting the importance of dataset-specific method selection. The trends align with expectations for few-shot learning, where incremental data improves performance but with varying efficiency across methods.
</details>
Fig. 1: Comparison of ProtoNets, FS+CPL and FS+APL in different number of shots
Table 1 compares the performance of FS+CPL and FS+APL with the baseline architectures in [12]. We observe that both FS+CPL and FS+APL outperform ProtoNets across all datasets. In particular, FS+APL achieves significant improvements, with accuracy increases of 5.2% on FSD2018, 4.1% on VoxCeleb, 2% on ESC-50, 4.5% on BirdClef, and a slight 0.2% improvement on Nsynth. FS+APL also demonstrates strong performance against the optimization-based methods (MAML and MAML+Curv), surpassing the best alternative in most datasets. Specifically, on FSD2018, FS+APL matches MAML’s performance with a marginal 0.08% accuracy increase. On BirdClef, it outperforms MAML+Curv by 1.9%, and on VoxCeleb it exceeds MAML+Curv by 5.6%. While MAML+Curv achieves slightly higher accuracy on NSynth (0.2%) and ESC-50 (2.5%), FS+APL remains highly competitive while requiring substantially fewer computational resources and less training time. The results highlight the effectiveness of the angular loss compared to contrastive loss and plain ProtoNets. For FS+APL, we report the best performing angle $α$ separately for each dataset. However, we observed that varying the angle had a small impact on the final perfrormance. In detail, $α=30^∘$ yields the best results on FSD2018, $α=15^∘$ for Nsynth, ESC-50, and BirdClef, and $α=0^∘$ for VoxCeleb. Furthermore, using only prototypes as anchors improved performance on ESC-50 and VoxCeleb, while in the other datasets, the best results achieved without restricting anchors to prototypes. Fig. 1 summarizes the performance of ProtoNets, FS+CPL and FS+APL methods across different number of shots. As it is evident, both FS+CPL and FS+APL surpass the performance of plain ProtoNets, in all datasets and all k-shot settings. As expected the performance in all datasets and models, increases with the number of shots. Overall, FS+APL performs better than FS+CPL in most k-shot scenarios. FS+CPL slightly outperforms FS+APL on the FSD2018 dataset in the 3-shot scenario by 0.51%, and on the ESC-50 dataset in the same scenario by 1.1%. To assess the impact of each module in our approach, we decompose it into four standalone architectures: (1) the baseline Prototypical Networks (ProtoNets); (2) ProtoNets with the augmentation module and attention layer; (3) ProtoNets with augmentation-attention and contrastive loss (FS+CPL); and (4) the same as (3) but with angular loss replacing contrastive loss (FS+APL). The 5-shot results for each dataset are presented in Fig. 2.
<details>
<summary>module_comparison_subplots.png Details</summary>
### Visual Description
## Bar Chart: Model Accuracy Comparison Across Datasets
### Overview
The chart compares the accuracy scores of four models (ProtoNet, ProtoNet+Aug, FS+CPL, FS+APL) across five datasets (FSD2018, Nsynth, ESC-50, BirdClef, VoxCeleb). Accuracy scores are plotted with error bars, and models are differentiated by line styles (solid, dashed, dotted). The legend is positioned at the bottom-right corner.
### Components/Axes
- **Y-axis**: "Accuracy Score" (ranging from ~0.55 to 0.98).
- **X-axis**: Model names: ProtoNet (solid line), ProtoNet+Aug (dashed line), FS+CPL (dotted line), FS+APL (solid line with markers).
- **Legend**: Located at bottom-right, mapping line styles to models.
- **Datasets**: Vertically labeled on the left (FSD2018 at top, VoxCeleb at bottom).
### Detailed Analysis
1. **FSD2018**:
- ProtoNet: 0.550 (solid line, leftmost point).
- ProtoNet+Aug: 0.575 (dashed line, slight increase).
- FS+CPL: 0.575 (dotted line, same as ProtoNet+Aug).
- FS+APL: 0.575 (solid line, same as FS+CPL).
2. **Nsynth**:
- ProtoNet: 0.979 (solid line, highest score).
- ProtoNet+Aug: 0.978 (dashed line, minor drop).
- FS+CPL: 0.977 (dotted line, further drop).
- FS+APL: 0.977 (solid line, same as FS+CPL).
3. **ESC-50**:
- ProtoNet: 0.84 (solid line, lowest score).
- ProtoNet+Aug: 0.85 (dashed line, slight improvement).
- FS+CPL: 0.84 (dotted line, reverts to ProtoNet).
- FS+APL: 0.85 (solid line, matches ProtoNet+Aug).
4. **BirdClef**:
- ProtoNet: 0.725 (solid line, lowest score).
- ProtoNet+Aug: 0.750 (dashed line, significant jump).
- FS+CPL: 0.750 (dotted line, same as ProtoNet+Aug).
- FS+APL: 0.750 (solid line, same as FS+CPL).
5. **VoxCeleb**:
- ProtoNet: 0.76 (solid line, lowest score).
- ProtoNet+Aug: 0.78 (dashed line, improvement).
- FS+CPL: 0.78 (dotted line, same as ProtoNet+Aug).
- FS+APL: 0.78 (solid line, same as FS+CPL).
### Key Observations
- **ProtoNet+Aug** consistently outperforms ProtoNet across all datasets, with the largest gains in BirdClef (+0.025) and VoxCeleb (+0.02).
- **FS+APL** and **FS+CPL** show similar performance, often matching or slightly trailing ProtoNet+Aug.
- **Nsynth** has the highest baseline accuracy (ProtoNet: 0.979), while **FSD2018** has the lowest (ProtoNet: 0.550).
- Error bars (vertical lines with dots) indicate variability but lack explicit numerical values for uncertainty.
### Interpretation
The data suggests that **model augmentation (ProtoNet+Aug)** and **feature selection/pooling (FS+CPL/FS+APL)** improve accuracy compared to the base ProtoNet model. However, gains plateau in some cases (e.g., FSD2018, ESC-50). The **FS+APL** model performs comparably to FS+CPL, implying that additional feature selection steps may not always yield significant improvements. Nsynth’s high baseline accuracy highlights dataset-specific challenges, while BirdClef and VoxCeleb show the most substantial benefits from model enhancements. The error bars suggest variability in performance, but without explicit uncertainty values, the magnitude of confidence intervals remains unclear.
</details>
Fig. 2: Module importance in overall performance per dataset.
We observe that, except for Nsynth, the augmentation-attention module improves the accuracy of the plain Prototypical Networks across all datasets. Specifically, this module increases accuracy by 4.32% in FSD2018, 3.63% in BirdClef, 3.94% in VoxCeleb, and 0.20% in ESC-50. In Nsynth, however, the accuracy exhibits a very slight decrease of 0.03%. The inclusion of Contrastive loss (FS+CPL) further enhances accuracy in most datasets. Compared to the augmentation-attention module alone, it adds 0.52% in ESC-50 and 0.16% in Nsynth, while showing a minor decrease of 0.31% in FSD2018 and 0.33% in VoxCeleb. In BirdClef, the improvement is 0.19%. By replacing Contrastive loss with Angular loss (FS+APL), we achieve further performance improvements over the augmentation-attention module. Specifically, FS+APL increases accuracy by 0.91% in FSD2018, 0.25% in Nsynth, 1.90% in ESC-50, 0.95% in BirdClef, and 0.24% in VoxCeleb compared to the augmentation-attention module.
## 4 Conclusions
We presented a novel approach for few-shot audio classification that enhances ProtoNets utilizing spectrogram augmentation and contrastive learning. Overall, our work is the first to integrate supervised contrastive learning, specifically angular loss, into prototypical few-shot training for audio classification. Extensive evaluation on the MetaAudio benchmark demonstrates state-of-the-art performance in 5-way 5-shot classification, showing significant improvements over standard ProtoNets (up to 5.2% on challenging datasets) while matching the accuracy of more computationally intensive optimization-based approaches. Future research directions include investigating alternative contrastive loss formulations and developing more sophisticated training techniques to further boost few-shot learning performance.
## References
- [1] Jake Snell, Kevin Swersky, and Richard Zemel, “Prototypical networks for few-shot learning,” Advances in neural information processing systems, vol. 30, 2017.
- [2] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al., “Matching networks for one shot learning,” Advances in neural information processing systems, vol. 29, 2016.
- [3] Chelsea Finn, Pieter Abbeel, and Sergey Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International conference on machine learning. PMLR, 2017, pp. 1126–1135.
- [4] Eunbyung Park and Junier B Oliva, “Meta-curvature,” Advances in neural information processing systems, vol. 32, 2019.
- [5] Boris Oreshkin, Pau Rodríguez López, and Alexandre Lacoste, “Tadam: Task dependent adaptive metric for improved few-shot learning,” Advances in neural information processing systems, vol. 31, 2018.
- [6] Yu Wang, Justin Salamon, Nicholas J Bryan, and Juan Pablo Bello, “Few-shot sound event detection,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 81–85.
- [7] Liwen You, Erika Pelaez Coyotl, Suren Gunturu, and Maarten Van Segbroeck, “Transformer-based bioacoustic sound event detection on few-shot learning tasks,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5.
- [8] Jixuan Wang, Kuan-Chieh Wang, Marc T Law, Frank Rudzicz, and Michael Brudno, “Centroid-based deep metric learning for speaker recognition,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 3652–3656.
- [9] Jordi Pons, Joan Serrà, and Xavier Serra, “Training neural audio classifiers with few data,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 16–20.
- [10] Szu-Yu Chou, Kai-Hsiang Cheng, Jyh-Shing Roger Jang, and Yi-Hsuan Yang, “Learning to match transient sound events using attentional similarity for few-shot sound recognition,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 26–30.
- [11] Shilei Zhang, Yong Qin, Kewei Sun, and Yonghua Lin, “Few-shot audio classification with attentional graph neural networks.,” in Interspeech, 2019, pp. 3649–3653.
- [12] Calum Heggan, Sam Budgett, Timothy Hospedales, and Mehrdad Yaghoobi, “Metaaudio: A few-shot audio classification benchmark,” in International Conference on Artificial Neural Networks. Springer, 2022, pp. 219–230.
- [13] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning. PmLR, 2020, pp. 1597–1607.
- [14] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan, “Supervised contrastive learning,” Advances in neural information processing systems, vol. 33, pp. 18661–18673, 2020.
- [15] Jian Wang, Feng Zhou, Shilei Wen, Xiao Liu, and Yuanqing Lin, “Deep metric learning with angular loss,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2593–2601.
- [16] Yassine Ouali, Céline Hudelot, and Myriam Tami, “Spatial contrastive learning for few-shot classification,” in Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part I 21. Springer, 2021, pp. 671–686.
- [17] Zhanyuan Yang, Jinghua Wang, and Yingying Zhu, “Few-shot classification with contrastive learning,” in European conference on computer vision. Springer, 2022, pp. 293–309.
- [18] Yizhao Gao, Nanyi Fei, Guangzhen Liu, Zhiwu Lu, and Tao Xiang, “Contrastive prototype learning with augmented embeddings for few-shot learning,” in Uncertainty in artificial intelligence. PMLR, 2021, pp. 140–150.
- [19] Daniel S Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D Cubuk, and Quoc V Le, “Specaugment: A simple data augmentation method for automatic speech recognition,” arXiv preprint arXiv:1904.08779, 2019.
- [20] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama, “Optuna: A next-generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019, pp. 2623–2631.