# Evaluating Pretrained General-Purpose Audio Representations for Music Genre Classification
**Authors**: Kashish Rai, Mrinmoy Bhattacharjee
institutetext: Dept. of Mathematics, Indian Institute of Technology Patna institutetext: Dept. of CSE, Indian Institute of Technology Jammu email: raikashish1002@gmail.com
## Abstract
This study investigates the use of self-supervised learning embeddings, particularly BYOL-A, in conjunction with a deep neural network classifier for Music Genre Classification. Our experiments demonstrate that BYOL-A embeddings outperform other pre-trained models, such as PANNs and VGGish, achieving an accuracy of 81.5% on the GTZAN dataset and 64.3% on FMA-Small. The proposed DNN classifier improved performance by 10-16% over linear classifiers. We explore the effects of contrastive and triplet loss and multitask training with optimized loss weights, achieving the highest accuracy. To address cross-dataset challenges, we combined GTZAN and FMA-Small into a unified 18-class label space for joint training, resulting in slight performance drops on GTZAN but comparable results on FMA-Small. The scripts developed in this work are publicly available. https://github.com/kashishrai12/musicgenre-classification
## 1 Introduction
Music, a universally appealing art form, is often categorized into genres like jazz, pop and others. Music Genre Classification (MGC) helps in efficient music archiving and retrieval. Researchers have used audio feature engineering to create discriminative representations for training classifiers for MGC [9, 11, 8]. Recent studies show that deep audio foundation models, trained on large datasets via Self-Supervised Learning (SSL), are highly effective at generating discriminating feature embeddings for MGC [7, 15, 13, 6, 5]. This work explores the impact of carefully designed classifiers using SSL embeddings for MGC.
Singh et al. [11] compare the effectiveness of standard audio feature representations like Chromagram, Mel-Frequency Cepstral Coefficients (MFCC), and Swaragram [12] with deep learning models like Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) for the MGC task. Their results indicate that Mel-scale features and Swaragram features are the most generic representations across datasets. Other researchers have also utilized music spectrograms as input features for training classifiers for MGC [8]. Ru et al. [9] proposed a multi-modal and multi-label MGC approach by leveraging the inter-genre correlations and the relationship between parallel music and lyrics data.
Recent MGC research has shifted from signal processing-based features to deep SSL embeddings. Zhao et al. [15] introduced the S3T model, trained with a momentum-based contrastive learning approach using the Swin Transformer backbone for MGC and music tagging tasks. Spijkervet et al. [13] developed the CLMR framework, based on SimCLR, for music classification from raw audio waveforms, utilizing data augmentation techniques. Niizumi et al. [7] proposed BYOL-A, an SSL framework for audio, extending BYOL with audio-specific augmentations and a novel encoder. Kong et al. [6] introduced Pretrained Audio Neural Networks (PANNs), large-scale pre-trained convolutional networks for audio classification and MGC tasks. Hershey et al. [5] proposed the VGGish model, a CNN architecture pre-trained on YouTube data and established a baseline for audio embedding extraction in music information retrieval.
Previous work has mainly focused on extracting generalizable SSL embeddings , often using a simple linear layer to map embeddings to target labels of the downstream task. However, limited attention has been given to designing classifier architectures or learning frameworks to improve MGC performance using SSL embeddings. This work explores the impact of carefully designed classifiers trained on SSL embeddings to enhance MGC. The paper is structured as follows. Section 2 presents the methodology for designing a better MGC classifier with SSL embeddings, Section 3 covers experiments and results, and Section 4 concludes the paper.
<details>
<summary>2603.13871v1/sys_diag2.png Details</summary>
### Visual Description
## Diagram: Machine Learning Pipeline Architecture (BYOL-A Feature Extractor and DNN Classifier)
### Overview
The image presents a high-level architectural flow diagram of a machine learning pipeline designed for audio genre prediction. The pipeline is divided into two primary modules: the "BYOL-A Feature Extractor" and the "DNN Classifier." The process flows from left to right, beginning with raw input and culminating in a genre prediction.
### Components/Axes
The diagram is organized into a linear sequence of processing blocks, grouped into two distinct dashed-line containers.
**Color Coding & Legend:**
* **Blue (Rounded Rectangles):** Input and Output stages.
* **Green (Rounded Rectangle):** Preprocessing stage.
* **Yellow (Rounded Rectangles):** Components within the BYOL-A Feature Extractor.
* **Pink/Purple (Rounded Rectangles):** Components within the DNN Classifier.
**Flow Indicators:**
* **Solid Arrows:** Represent the primary data flow between major stages.
* **Dashed Arrows:** Represent the internal data flow within the modular blocks.
### Detailed Analysis
#### 1. Input and Preprocessing (Left)
* **Input:** A blue rounded rectangle labeled "Input".
* **Preprocessing:** A green rounded rectangle labeled "Preprocessing (log-mel spectrogram)". A solid arrow connects "Input" to this block.
#### 2. BYOL-A Feature Extractor (Center, Dashed Box)
This module contains a sequence of five yellow rounded rectangles connected by dashed arrows:
1. **Pre-Norm**
2. **Mixup**
3. **Random Resize Crop**
4. **Random Linear Fader**
5. **Post-Norm**
#### 3. Transition
* A solid arrow connects the "BYOL-A Feature Extractor" to the "DNN Classifier."
* **Label:** "3072-D Embedding" is written above this arrow, indicating the dimensionality of the feature vector passed between the two modules.
#### 4. DNN Classifier (Right, Dashed Box)
This module contains three pink/purple rounded rectangles connected by dashed arrows:
1. **Linear Layers** (Pink)
2. **BatchNorm + ReLU + Dropout** (Light Purple)
3. **Multitask Heads (Cross-Entropy, Contrastive Loss, Triplet Loss)** (Darker Purple)
#### 5. Output (Far Right)
* **Genre Prediction:** A blue rounded rectangle. A solid arrow connects the "DNN Classifier" to this final output block.
### Key Observations
* **Modular Design:** The architecture clearly separates feature extraction (BYOL-A) from classification (DNN).
* **Data Bottleneck:** The "3072-D Embedding" serves as the critical interface between the two modules, suggesting that the feature extractor compresses the input into a high-dimensional vector.
* **Augmentation:** The presence of "Mixup," "Random Resize Crop," and "Random Linear Fader" within the feature extractor suggests the use of data augmentation techniques to improve model robustness.
* **Multitask Learning:** The final stage of the classifier uses three distinct loss functions (Cross-Entropy, Contrastive Loss, Triplet Loss), indicating that the model is trained to optimize for multiple objectives simultaneously, likely to improve the quality of the learned embeddings.
### Interpretation
This diagram illustrates a sophisticated deep learning pipeline, likely for Audio Information Retrieval (AIR).
* **BYOL-A (Bootstrap Your Own Latent - Audio):** The inclusion of "Pre-Norm," "Mixup," and "Random" operations suggests this is a self-supervised learning framework. By applying these augmentations, the model learns to extract invariant features from audio spectrograms without needing explicit labels during the feature extraction phase.
* **The Embedding:** The 3072-D vector is the "latent representation" of the audio. This is the compressed knowledge the model has learned about the audio input.
* **The Classifier:** The DNN Classifier takes this high-dimensional representation and maps it to a specific genre. The use of "Multitask Heads" is significant; it implies that the model is not just learning to classify genres (Cross-Entropy), but is also being regularized or guided by contrastive and triplet losses. These losses force the model to group similar audio samples closer together in the embedding space and push dissimilar ones apart, resulting in a more discriminative and robust feature representation for the final genre prediction.
</details>
Figure 1: Diagram representing the proposed MGC system. The system consists of SSL embedding extractor followed by an DNN classifier for music genre prediction.
## 2 Methodology
This work uses general-purpose audio SSL models to extract feature embeddings and train carefully designed classifiers to improve MGC performance. A block diagram of the proposed system is illustrated in Fig. 1. The feature extraction approach used in this work is described below.
### 2.1 Feature Extraction
This work compares three different pre-trained SSL models for extracting feature representations, viz., BYOL-A [7], PANNs [6], and VGGish [5]. Each of these models is briefly described next.
In BYOL-A [7], raw audio is converted to log-mel spectrograms and fed into a dual-network architecture with an online and target network. Augmentations like Mixup, Random Resize Crop, and Random Linear Fader are applied to ensure robustness to common audio perturbations. The encoder combines convolutional feature maps and temporal pooling to capture both local and global features. This results in a $3072$ -dimensional embedding for each 30-second audio track, encoding both temporal and spectral characteristics.
We use the CNN14 model from the PANNs framework [6] to extract audio embeddings. Pretrained on the large-scale AudioSet dataset [3], PANNs capture rich acoustic patterns. The input waveform is converted to a log-mel spectrogram and processed by a deep CNN with convolutional blocks and global pooling. The model outputs a $2048$ -dimensional embedding per segment, summarizing spectral and temporal information. The embeddings are averaged over time to form a compact representation for each $30$ s audio clip.
Hershey et al. [5] introduced VGGish, a compact audio feature extractor based on the VGG architecture [10] and pretrained on AudioSet [3]. Audio is resampled to $16$ kHz and converted into $0.96$ s log-mel spectrogram patches, which are processed by a VGG-like network to produce 128-dimensional embeddings. For our experiments, embeddings from a $30$ s audio track are averaged to create a fixed-length feature vector for MGC tasks. The next subsection discusses the classifier training details.
### 2.2 Classifier Training
Our approach centers on designing and training a DNN classifier to map SSL embeddings to music genre labels. The architecture is tailored to utilize the high-dimensional feature space and prevent overfitting. The input layer receives the SSL embedding, followed by hidden layers with linear transformations, Batch Normalization, ReLU activation, and Dropout regularization. These layers progressively reduce dimensions, ensuring effective feature abstraction and regularization. The final architecture was determined empirically.
In the initial experiments for the MGC task, the classifiers are trained using the cross-entropy loss, as defined below.
$$
L_CE=-\frac{1}{N}∑_i=1^Nlog(p_i) \tag{1}
$$
where, $p_i$ is the predicted probability of true class for the $i^th$ sample and $N$ is the number of samples in the batch. To further enhance the discriminative capacity of the learned representations, we explored advanced loss functions, viz., contrastive loss and triplet loss. Some previous works have also explored contrastive loss for the classification of music genres [9, 1, 13]. The contrastive loss is defined as follows [4].
$$
L_CL=\frac{1}{N}∑_i=1^N≤ft\{(1-y_i)·\max≤ft(m-D_w≤ft(z_i^(1),z_i^(2)\right),0\right)^2+y· D_w≤ft(z_i^(1),z_i^(2)\right)^2\right\} \tag{1}
$$
where, $D_w≤ft(z_i^(1),z_i^(2)\right)=√{∑_k=1^n{≤ft(z_i^(1)[k]-z_i^(2)[k]+ε\right)^2}}$ is the Euclidean distance, $ε=1e^-6$ is used for numerical stability, $z_i^(1)$ and $z_i^(2)$ are the model outputs for the $i^th$ input $x_i$ , $y$ is a binary label indicating similar ( $y=0$ ) or dissimilar ( $y=1$ ) pairs, and $m=1.0$ is the loss margin. We have also explored the Triplet loss, which is described below.
$$
L_TL=D_w≤ft(F≤ft(x_i^(a)\right),F≤ft(x_i^(p)\right)\right)-D_w≤ft(F≤ft(x_i^(a)\right),F≤ft(x_i^(n)\right)\right)+ε \tag{3}
$$
where, $F$ is the model that takes the $i^th$ input $x_i$ and produces a $d$ -dimensional output $F≤ft(x_i\right)$ , and $ε$ is the bias term. Superscripts $(a)$ , $(p)$ , and $(n)$ denote the anchor, positive, and negative samples, respectively. These losses were incorporated into the training regime as single-task and multi-task setups alongside the standard cross-entropy loss. The multi-task setup involved multiple output heads, each optimized for a specific loss component, with carefully adjusted weighting parameters. For example, the total loss in the multi-task training, with separate output heads for cross-entropy, contrastive, and triplet losses, is computed as $L=α· L_CE+β· L_CL+≤ft(1-α-β\right)· L_TL$ . Details of the various configurations explored are provided in subsection 3.3.
## 3 Experiments and Results
To evaluate the proposed approach, we conducted experiments to assess the impact of architectural choices, feature extraction methods, and training strategies on MGC performance. Embeddings from BYOL-A [7], PANNs [6], and VGGish [5] were used as input features for the classifier. Two datasets were used for benchmarking: GTZAN [14], consisting of $1000$ half-minute music excerpts labeled in $10$ categories, and the FMA Small subset [2], with $8000$ $30$ s tracks labeled into $8$ genres. A consistent feature extraction pipeline was applied to both datasets, and embeddings from different models were compared. The experiments and results are detailed in the following subsections.
<details>
<summary>2603.13871v1/depth_variation.png Details</summary>
### Visual Description
## Bar Chart: Test Accuracy vs. Depth Variation
### Overview
The image is a bar chart illustrating the relationship between the depth of a model (measured in the number of layers) and its resulting test accuracy. The chart demonstrates a non-monotonic relationship, where accuracy increases as layers are added up to a certain point, followed by a decline.
### Components/Axes
* **Y-Axis (Vertical):** Labeled "Test Accuracy (%)". The scale ranges from 70 to 85, with major grid lines marked at 70, 75, 80, and 85.
* **X-Axis (Horizontal):** Labeled "Depth Variation". It contains four discrete categories: "1 Layer", "2 Layers", "3 Layers", and "4 Layers".
* **Data Series:** Four vertical bars, each corresponding to a specific depth category.
* **1 Layer:** Light blue bar.
* **2 Layers:** Light green bar.
* **3 Layers:** Light red/salmon bar.
* **4 Layers:** Light orange bar.
### Detailed Analysis
The trend shows an initial upward trajectory in accuracy as the number of layers increases from 1 to 3, followed by a downward trend when moving to 4 layers.
* **1 Layer (Blue):** The bar reaches a height of **76.0%**.
* **2 Layers (Green):** The bar reaches a height of **77.0%**, representing a 1.0 percentage point increase over the 1-layer model.
* **3 Layers (Red):** The bar reaches a height of **78.5%**, representing the peak accuracy in the dataset. This is a 1.5 percentage point increase over the 2-layer model.
* **4 Layers (Orange):** The bar reaches a height of **76.5%**, representing a 2.0 percentage point decrease from the 3-layer model.
### Key Observations
* **Optimal Depth:** The 3-layer configuration provides the highest test accuracy (78.5%).
* **Performance Drop:** There is a notable performance degradation when increasing the depth from 3 layers to 4 layers.
* **Relative Performance:** The 4-layer model (76.5%) performs worse than the 2-layer model (77.0%) but still outperforms the baseline 1-layer model (76.0%).
### Interpretation
This data suggests a classic "sweet spot" in model architecture design. In machine learning, increasing the depth of a model (adding layers) typically allows the model to learn more complex features. However, the data demonstrates that this benefit is not infinite.
The drop in accuracy at 4 layers suggests that the model may be encountering issues such as:
1. **Overfitting:** The model is becoming too complex for the provided dataset, capturing noise rather than signal.
2. **Optimization Difficulties:** The model may be suffering from vanishing gradients or other training instabilities associated with deeper architectures, preventing it from converging as effectively as the 3-layer version.
From a technical standpoint, this chart serves as evidence that "deeper is not always better." It justifies the selection of a 3-layer architecture as the optimal configuration for this specific task, as it balances complexity and performance effectively.
</details>
<details>
<summary>2603.13871v1/width_variation.png Details</summary>
### Visual Description
## Bar Chart: Test Accuracy vs. Width Variation
### Overview
The image is a bar chart illustrating the relationship between model "Width Variation" (measured in node multipliers) and "Test Accuracy (%)". The chart displays three distinct data points, showing a consistent positive correlation between increasing the number of nodes and the resulting test accuracy.
### Components/Axes
* **Y-Axis (Vertical):** Labeled "Test Accuracy (%)". The scale ranges from 70 to 85, with major horizontal grid lines marked at 70, 75, 80, and 85.
* **X-Axis (Horizontal):** Labeled "Width Variation". It contains three categorical labels positioned at the base of each bar: "0.5x Nodes", "1.5x Nodes", and "2x Nodes".
* **Data Series:** Three vertical bars of varying colors and heights.
* **Left Bar:** Light blue.
* **Center Bar:** Light green.
* **Right Bar:** Light red/pink.
### Detailed Analysis
The chart displays a monotonic upward trend from left to right.
* **0.5x Nodes (Left):**
* **Position:** Leftmost position on the X-axis.
* **Color:** Light blue.
* **Value:** The bar height is explicitly labeled as **74.5%**. Visually, it sits just below the 75% grid line.
* **1.5x Nodes (Center):**
* **Position:** Center position on the X-axis.
* **Color:** Light green.
* **Value:** The bar height is explicitly labeled as **76.5%**. Visually, it sits above the 75% grid line.
* **2x Nodes (Right):**
* **Position:** Rightmost position on the X-axis.
* **Color:** Light red/pink.
* **Value:** The bar height is explicitly labeled as **78.5%**. Visually, it sits higher than the center bar, approaching the 80% grid line.
### Key Observations
* **Linear Scaling:** The increase in accuracy is perfectly consistent across the three data points.
* **Delta Analysis:**
* The jump from "0.5x Nodes" to "1.5x Nodes" results in a **+2.0%** increase in accuracy.
* The jump from "1.5x Nodes" to "2x Nodes" results in a **+2.0%** increase in accuracy.
* **Consistency:** The chart demonstrates a stable, predictable improvement in performance as the model width is increased.
### Interpretation
The data demonstrates a clear positive correlation between model width (the number of nodes) and test accuracy. This is a typical representation of model scaling, where increasing the capacity of a neural network (in this case, its width) allows it to learn more complex patterns or generalize better, leading to higher accuracy on the test set.
The fact that the accuracy increases by exactly 2% at each step suggests that, within this specific range of width variation, the model is experiencing linear returns on its capacity expansion. There is no evidence of diminishing returns or overfitting within the provided data points, suggesting that further increases in width might continue to yield performance gains.
</details>
(a) (b)
<details>
<summary>2603.13871v1/dropout.png Details</summary>
### Visual Description
## Bar Chart: Test Accuracy vs. Dropout Rate
### Overview
The image is a bar chart illustrating the performance of a machine learning model (likely a neural network) across four different dropout regularization rates. The chart plots "Dropout" values on the horizontal axis against "Test Accuracy (%)" on the vertical axis.
### Components/Axes
* **Y-Axis (Vertical):** Labeled "Test Accuracy (%)". The scale ranges from 70 to 85, with dashed horizontal grid lines at intervals of 5 units (70, 75, 80, 85).
* **X-Axis (Horizontal):** Labeled "Dropout". The axis displays four discrete categories: 0.2, 0.3, 0.4, and 0.5.
* **Data Series:** Four vertical bars, each color-coded and labeled with its specific percentage value at the top.
### Detailed Analysis
The chart displays four distinct data points. Below is the extraction of values, cross-referenced by color and position:
* **Dropout 0.2 (Blue Bar):**
* **Position:** Far left.
* **Value:** 77.5%
* **Trend:** Baseline starting point.
* **Dropout 0.3 (Green Bar):**
* **Position:** Second from left.
* **Value:** 79.0%
* **Trend:** The bar height increases significantly compared to the 0.2 bar, representing the peak accuracy in this dataset.
* **Dropout 0.4 (Red/Pink Bar):**
* **Position:** Third from left.
* **Value:** 78.0%
* **Trend:** The bar height decreases compared to the 0.3 bar.
* **Dropout 0.5 (Orange Bar):**
* **Position:** Far right.
* **Value:** 78.5%
* **Trend:** The bar height increases slightly compared to the 0.4 bar, though it remains lower than the 0.3 peak.
### Key Observations
* **Optimal Performance:** The highest test accuracy is achieved at a dropout rate of 0.3 (79.0%).
* **Non-Linear Trend:** The relationship between dropout and accuracy is not monotonic. Accuracy increases from 0.2 to 0.3, drops at 0.4, and then experiences a slight recovery at 0.5.
* **Range:** The total variance in test accuracy across all tested dropout rates is 1.5% (ranging from 77.5% to 79.0%).
### Interpretation
This chart demonstrates the sensitivity of a model to the dropout hyperparameter, which is used to prevent overfitting by randomly setting a fraction of input units to 0 during training.
* **Optimal Hyperparameter:** The data suggests that a dropout rate of 0.3 is the optimal setting among those tested.
* **Underfitting vs. Overfitting:** The drop in accuracy at 0.4 suggests that increasing the dropout rate beyond 0.3 may be causing the model to lose too much information (underfitting), thereby reducing its predictive capability.
* **Anomalous Recovery:** The slight increase in accuracy at 0.5 (compared to 0.4) is an interesting anomaly. In a typical regularization curve, one might expect a steady decline after the optimal point. This recovery could indicate that the model architecture is robust enough to handle higher dropout, or it could be a result of stochastic variance in the training process.
* **Conclusion:** For this specific model, a dropout rate of 0.3 provides the best balance between regularization and information retention.
</details>
<details>
<summary>2603.13871v1/activation_functions.png Details</summary>
### Visual Description
## Bar Chart: Activation Function Performance Comparison
### Overview
This image is a bar chart comparing the "Test Accuracy (%)" of four different neural network activation functions: ELU, LeakyReLu, ReLU, and Swish. The chart uses a vertical axis to represent accuracy percentages and a horizontal axis to categorize the activation functions.
### Components/Axes
* **Y-Axis (Vertical):** Labeled "Test Accuracy (%)". The scale ranges from 70 to 85, with major grid lines marked at 70, 75, 80, and 85.
* **X-Axis (Horizontal):** Labeled "Activation Functions". It contains four categorical labels: "ELU", "LeakyReLu", "ReLU", and "Swish".
* **Data Series:** Four vertical bars, each color-coded:
* **ELU:** Light Blue
* **LeakyReLu:** Light Green
* **ReLU:** Light Red/Pink
* **Swish:** Light Orange
* **Annotations:** Each bar has a numerical percentage value placed directly above it.
* **Artifacts:** There is a small, stray red dot located in the white space above the ELU bar and to the left of the LeakyReLu bar.
### Detailed Analysis
The chart presents a significant discrepancy between the visual height of the bars and the numerical labels provided. Below is the breakdown of the data as presented:
| Activation Function | Bar Color | Label Value | Visual Bar Height (Approx.) |
| :--- | :--- | :--- | :--- |
| **ELU** | Light Blue | 79.0% | ~78.0% |
| **LeakyReLu** | Light Green | 76.5% | ~76.5% |
| **ReLU** | Light Red | 78.0% | ~79.0% |
| **Swish** | Light Orange | 78.0% | ~78.0% |
* **ELU (Far Left):** The bar is visually aligned with the 78% mark, yet the text label indicates 79.0%.
* **LeakyReLu (Second from Left):** The bar is visually aligned with the 76.5% mark, and the text label matches (76.5%).
* **ReLU (Third from Left):** The bar is visually the tallest, reaching approximately 79%, yet the text label indicates 78.0%.
* **Swish (Far Right):** The bar is visually aligned with the 78% mark, and the text label matches (78.0%).
### Key Observations
* **Data Inconsistency:** There is a clear mismatch between the visual representation (bar height) and the textual data (labels) for the ELU and ReLU categories.
* **Visual Trend:** Visually, the ReLU function appears to have the highest test accuracy, followed by ELU and Swish (which appear equal), and finally LeakyReLu.
* **Label Trend:** According to the text labels, ELU is the highest performing function, followed by a tie between ReLU and Swish, with LeakyReLu being the lowest.
* **Artifact:** A stray red pixel/dot is visible in the upper-left quadrant of the chart area, suggesting a potential rendering or editing error.
### Interpretation
The data presented in this chart is **highly unreliable** due to the internal contradictions between the visual bar heights and the numerical labels.
From a technical standpoint, this chart cannot be used to draw valid conclusions about the performance of these activation functions because the source of truth (the label vs. the bar) is ambiguous. If the labels are correct, the visual representation is misleading. If the visual representation is correct, the labels are incorrect. The presence of the stray red dot further suggests that this image may be a draft or an improperly rendered graphic. Any analysis derived from this image should be treated with extreme skepticism until the raw data is verified.
</details>
(c) (d)
<details>
<summary>2603.13871v1/advanced_techniques.png Details</summary>
### Visual Description
## Bar Chart: Test Accuracy Comparison by BatchNorm Noise Augmentation
### Overview
The image is a bar chart comparing the "Test Accuracy (%)" of a machine learning model across three distinct configurations of Batch Normalization (BN) and noise augmentation. The chart demonstrates that all three methods yield very similar performance results, with a marginal advantage for the "Noise Aug. (Epoch)" method.
### Components/Axes
* **Y-Axis (Vertical):** Labeled "Test Accuracy (%)". The scale ranges from 70 to 85, with major grid lines marked at 5-unit intervals (70, 75, 80, 85).
* **X-Axis (Horizontal):** Labeled "BatchNorm Noise Augmentation". It categorizes the three experimental conditions:
* **BN+Dropout** (Left)
* **Noise Aug. (Epoch)** (Center)
* **Noise Aug. (Once)** (Right)
* **Data Series (Bars):**
* **Left Bar:** Light blue color, representing "BN+Dropout".
* **Center Bar:** Light green color, representing "Noise Aug. (Epoch)".
* **Right Bar:** Light red/pink color, representing "Noise Aug. (Once)".
### Detailed Analysis
The chart displays the following specific data points for each category:
1. **BN+Dropout (Left):**
* **Visual Trend:** The bar is positioned on the left side of the chart.
* **Value:** The top of the bar aligns with the 81.0% mark.
2. **Noise Aug. (Epoch) (Center):**
* **Visual Trend:** The bar is positioned in the center. It is slightly taller than the other two bars.
* **Value:** The top of the bar aligns with the 81.5% mark.
3. **Noise Aug. (Once) (Right):**
* **Visual Trend:** The bar is positioned on the right side of the chart.
* **Value:** The top of the bar aligns with the 81.0% mark.
### Key Observations
* **Minimal Variance:** The total range of accuracy across all three methods is only 0.5% (from 81.0% to 81.5%).
* **Performance Parity:** "BN+Dropout" and "Noise Aug. (Once)" produce identical test accuracy results of 81.0%.
* **Visual Scaling:** The Y-axis is truncated, starting at 70 rather than 0. This visualization choice amplifies the visual height difference between the bars, making the 0.5% difference appear more significant than it is numerically.
### Interpretation
* **Data Significance:** The data suggests that the specific method of noise augmentation applied to Batch Normalization has a negligible impact on the final test accuracy of the model.
* **Practical Application:** While "Noise Aug. (Epoch)" is technically the superior method in this specific test, the gain is so small (0.5%) that it may not be statistically significant or practically relevant depending on the application.
* **Decision Making:** When choosing between these methods, a practitioner would likely prioritize factors other than accuracy, such as computational overhead, training time, or implementation complexity, as the performance delta is essentially non-existent. The chart serves to demonstrate that these variations are robust and yield consistent results.
</details>
(e)
Figure 2: Performance of various model architecture tuning experiments performed on the GTZAN dataset using BYOL-A embeddings as model input feature. The test accuracy (%) for different parameter variations is reported.
### 3.1 Selection of DNN architecture
The impact of hidden layer depth in the DNN classifier was tested with one to four hidden layers (Fig. 2 (a)) using a learning rate of $5e^-4$ and batch size of $64$ . A three-hidden-layer configuration yielded the best performance, with a test accuracy of $78.5\$ . The effect of node count in this configuration ( $128$ , $64$ , and $32$ nodes) was also explored (Fig. 2 (b)), with doubling the node count achieving the best accuracy of $76.5\$ . No other configurations led to significant improvements.
Dropout regularization was tested at various rates (Fig. 2 (c)), with the best performance at a rate of $0.3$ , enhancing generalization. Different activation functions, viz. LeakyReLU, ELU, and Swish were also evaluated (Fig. 2 (d)), with ReLU yielding the highest accuracy of $79\$ . Swish and ELU performed similarly but did not surpass ReLU. After each linear transformation, applying batch normalization was found to stabilize training and improve convergence, achieving a test accuracy of around $81\$ when combined with a dropout rate of $0.3$ .
Table 1: Test Accuracy (%) of using different feature extractors on GTZAN and FMA-Small datasets. Results are reported for our best DNN classifier trained on embeddings from each feature extractor.
| Feature Extractor | GTZAN (%) | FMA-Small (%) |
| --- | --- | --- |
| Niizumi et al. [7] | $70.1$ | $-$ |
| Defferrard et al. [2] | - | $58.0$ |
| Ours + BYOL-A embeddings | $81.5$ | $64.3$ |
| Ours + PANNs embeddings | $77.0$ | $58.0$ |
| Ours + VGGish embeddings | $79.5$ | $58.04$ |
A series of experiments assessed the impact of noise augmentation on model robustness (Fig. 2 (e)). Gaussian noise was added to input embeddings with varied Signal-to-Noise Ratios (SNR) and positions in time. Applying around $20$ dB of Gaussian noise at the start of training yielded the best test accuracy of $81.5\$ . Noise augmentation also reduced the number of epochs for convergence from $50$ to about $35$ and helped prevent overfitting.
### 3.2 Comparison of Feature Extraction Models
A comparison of BYOL-A embeddings with other pre-trained audio models is shown in Table 1. On the GTZAN dataset, BYOL-A achieved $81.5\$ accuracy, outperforming PANNs ( $77\$ ) and VGGish ( $79.5\$ ). BYOL-A also performed better on the FMA-Small dataset with $64.3\$ accuracy. The proposed DNN classifier provided a relative accuracy improvement of $10-16\$ on the GTZAN dataset for all feature extractors and $11\$ on FMA-Small using BYOL-A. These results demonstrate the superior discriminative power of BYOL-A embeddings for MGC and the impact of designing a customized classifier compared to a linear classifier layer.
Table 2: Multitask training results with different loss configurations. Cross-entropy (CE) weights are shown for each classification head, with only one of Contrastive or Triplet used per setup. A ✓ indicates the active loss head, with the weight in brackets.
| | Cross-Entropy ( $α$ ) | Contrastive ( $β$ ) | Triplet ( $≤ft(1-α-β\right)$ ) | Acc (%) | | |
| --- | --- | --- | --- | --- | --- | --- |
| 2 Heads | ✓( $0.5$ ) | - | - | ✓ ( $0.5$ ) | - | $79.0$ |
| ✓( $0.5$ ) | - | - | - | ✓ ( $0.5$ ) | $77.0$ | |
| 3 Heads | ✓( $0.45$ ) | ✓( $0.45$ ) | - | ✓ ( $0.1$ ) | - | $80.0$ |
| ✓( $0.40$ ) | ✓( $0.40$ ) | - | ✓ ( $0.2$ ) | - | $79.5$ | |
| ✓( $0.35$ ) | ✓( $0.35$ ) | - | ✓ ( $0.3$ ) | - | $81.5$ | |
| ✓( $0.30$ ) | ✓( $0.30$ ) | - | ✓ ( $0.4$ ) | - | $77.5$ | |
| ✓( $0.25$ ) | ✓( $0.25$ ) | - | ✓ ( $0.5$ ) | - | $81.0$ | |
| ✓( $0.20$ ) | ✓( $0.20$ ) | - | ✓ ( $0.6$ ) | - | $78.5$ | |
| ✓( $0.15$ ) | ✓( $0.15$ ) | - | ✓ ( $0.7$ ) | - | $79.5$ | |
| ✓( $0.10$ ) | ✓( $0.10$ ) | - | ✓ ( $0.8$ ) | - | $78.0$ | |
| ✓( $0.05$ ) | ✓( $0.05$ ) | - | ✓ ( $0.9$ ) | - | $77.5$ | |
| ✓( $0.07$ ) | ✓( $0.63$ ) | - | ✓ ( $0.3$ ) | - | $80.0$ | |
| ✓( $0.21$ ) | ✓( $0.49$ ) | - | ✓ ( $0.3$ ) | - | $79.5$ | |
| ✓( $0.49$ ) | ✓( $0.21$ ) | - | ✓ ( $0.3$ ) | - | $78.5$ | |
| ✓( $0.63$ ) | ✓( $0.07$ ) | - | ✓ ( $0.3$ ) | - | $81.0$ | |
| ✓( $0.35$ ) | ✓( $0.35$ ) | - | - | ✓ ( $0.3$ ) | $80.0$ | |
| 4 Heads | ✓( $0.23$ ) | ✓( $0.23$ ) | ✓( $0.23$ ) | - | ✓ ( $0.3$ ) | $79.5$ |
### 3.3 Multitask Training
To improve the discriminative power of embeddings, contrastive and triplet loss functions were tested in multitask learning setups with cross-entropy loss. The classifier was extended with multiple output heads, each corresponding to a different loss (e.g., cross-entropy, contrastive, or triplet). Three multitask configurations were explored. First, a $2$ -head setup having contrastive or triplet loss plus an auxiliary cross-entropy head for the anchor sample. Second, a $3$ -head setup has two auxiliary cross-entropy losses for anchor and non-anchor samples besides the main loss function (contrastive or triplet). Finally, a $4$ -head setup was used only for the main loss function as triplet loss with auxiliary cross-entropy heads for anchor, positive, and negative samples. By optimizing a weighted combination of these losses, the model learned representations that were discriminative for genre classification and robust to intra-class variability.
As shown in Table 2, contrastive loss in a $2$ -head setup achieved up to $79\$ accuracy, while triplet loss ranged from $77\$ to $80.5\$ . Multitask training with contrastive and cross-entropy loss achieved the best accuracy of $81.5\$ when loss weights were optimized.
### 3.4 Domain normalization
Table 3: Test Accuracy (%) on GTZAN and FMA-Small using BYOL-A Features
| Training Dataset | GTZAN (%) | FMA-Small (%) |
| --- | --- | --- |
| Niizumi et al. [7] | $70.1$ | - |
| Defferrard et al. [2] | - | $58.0$ |
| Ours (trained on GTZAN) | $81.5$ | - |
| Ours (trained on FMA-Small) | - | $64.3$ |
| Ours (trained on GTZAN + FMA-Small) | $78.0$ | $64.25$ |
The GTZAN and FMA-Small datasets have $10$ and $8$ non-overlapping genres, respectively. Such a scenario makes cross-dataset evaluation difficult. We combined both datasets into a unified label space of $18$ classes for joint training to address this. Using the best DNN architecture with BYOL-A feature extraction, the model was trained to predict one of these $18$ classes. As shown in Table 3, models trained on the combined dataset performed slightly worse on GTZAN but similarly on FMA-Small. While learning MGC on an augmented label space is more challenging, the BYOL-A embeddings with the designed DNN classifier still performed well. Such results highlight the need for domain adaptation to improve generalization.
## 4 Conclusion
This study investigates the effectiveness of SSL embeddings combined with a well-designed DNN classifier for MGC. The approach achieved competitive accuracies on the GTZAN and FMA-Small datasets, with BYOL-A embeddings outperforming PANNs and VGGish in both datasets. Fine-tuning with contrastive and triplet losses provided marginal improvements, while multitask training with contrastive and cross-entropy losses yielded the best accuracy of $81.5\$ . Training on a combined dataset of $18$ genres revealed challenges in domain adaptation. These results emphasize the superior discriminative power of BYOL-A embeddings and the importance of a customized classifier over simple linear layers. Future work will explore embeddings from different SSL model depths and apply domain adaptation techniques to improve cross-dataset performance.
## References
- [1] G. H. Costanzi, L. O. Teixeira, G. Z. Felipe, G. D. Cavalcanti, and Y. M. Costa (2024) Music Genre Classification Using Contrastive Dissimilarity. In Proc. 31st International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 1–8. Cited by: §2.2.
- [2] M. Defferrard, K. Benzi, P. Vandergheynst, and X. Bresson (2017) FMA: A Dataset for Music Analysis. In Proc. 18th International Society for Music Information Retrieval Conference (ISMIR), External Links: 1612.01840 Cited by: Table 1, Table 3, §3.
- [3] J. F. Gemmeke, D. P. W. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter (2017) Audio Set: An ontology and human-labeled dataset for audio events. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, pp. 776–780. Cited by: §2.1, §2.1.
- [4] R. Hadsell, S. Chopra, and Y. LeCun (2006) Dimensionality reduction by learning an invariant mapping. In Proc. IEEE computer society conference on computer vision and pattern recognition (CVPR’06), Vol. 2, pp. 1735–1742. Cited by: §2.2.
- [5] S. Hershey, S. Chaudhuri, D.P. Ellis, J.F. Gemmeke, A. Jansen, R.C. Moore, M. Plakal, D. Platt, R.A. Saurous, B. Seybold, and M. Slaney (2017) CNN architectures for large-scale audio classification. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 131–135. Cited by: §1, §1, §2.1, §2.1, §3.
- [6] Q. Kong, Y. Cao, T. Iqbal, Y. Wang, W. Wang, and M. D. Plumbley (2020) PANNs: Large-scale pretrained audio neural networks for audio pattern recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing 28, pp. 2880–2894. Cited by: §1, §1, §2.1, §2.1, §3.
- [7] D. Niizumi, D. Takeuchi, Y. Ohishi, N. Harada, and K. Kashino (2022) BYOL for audio: Exploring pre-trained general-purpose audio representations. IEEE/ACM Transactions on Audio, Speech, and Language Processing 31, pp. 137–151. Cited by: §1, §1, §2.1, §2.1, Table 1, Table 3, §3.
- [8] N. Pelchat and C. M. Gelowitz (2020) Neural Network Music Genre Classification. Canadian Journal of Electrical and Computer Engineering 43 (3), pp. 170–173. Cited by: §1, §1.
- [9] G. Ru, X. Zhang, J. Wang, N. Cheng, and J. Xiao (2023) Improving music genre classification from multi-modal properties of music and genre correlations perspective. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. Cited by: §1, §1, §2.2.
- [10] K. Simonyan and A. Zisserman (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. Cited by: §2.1.
- [11] Y. Singh and A. Biswas (2022) Robustness of musical features on deep learning models for music genre classification. Expert Systems with Applications 199, pp. 116879. External Links: ISSN 0957-4174 Cited by: §1, §1.
- [12] Y. Singh, R. Kumar, and A. Biswas (2021) Swaragram: Shruti-based chromagram for Indian classical music. In Proc. 25th International Symposium on Frontiers of Research in Speech and Music (FRSM), pp. 109–118. Cited by: §1.
- [13] J. Spijkervet and J. A. Burgoyne (2021-11) Contrastive Learning of Musical Representations. In Proc. 22nd International Society for Music Information Retrieval Conference, pp. 673–681. Cited by: §1, §1, §2.2.
- [14] G. Tzanetakis and P. Cook (2002) Musical genre classification of audio signals. IEEE Transactions on Speech and Audio Processing 10 (5), pp. 293–302. Cited by: §3.
- [15] H. Zhao, C. Zhang, B. Zhu, Z. Ma, and K. Zhang (2022) S3T: Self-Supervised Pre-Training with Swin Transformer For Music Classification. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vol. , pp. 606–610. Cited by: §1, §1.