## Flowchart: Audio Latent Reconstruction and Acoustic Concept Mapping
### Overview
The diagram illustrates a three-stage process for training Sparse Autoencoders (SAEs) to reconstruct audio latents and map them to interpretable acoustic concepts. It combines neural network architectures with linear algebraic transformations to bridge latent representations and human-understandable sound properties.
### Components/Axes
1. **Stage 1: Latent Reconstruction**
- **Inputs**: Audio signal
- **Components**:
- Latent Encoder (black box)
- Latent Vector (gray vertical bars)
- Two SAEs (purple boxes with "SAE" labels)
- Reconstruction Loss (gray arrow forming a loop)
- Sparse Activations (vertical gray bars between SAEs)
2. **Stage 2: Acoustic Concept Mapping**
- **Concept Units**:
- Pitch (piano keyboard with "W [“C4”]" label)
- Loudness (speaker icon)
- Timbre (color gradient from white to green)
- **Mathematical Elements**:
- Linear Weight Matrix W (orange boxes connecting SAE to concepts)
3. **Stage 3: Audio Transformation**
- **Components**:
- Latent Encoder (repeated from Stage 1)
- SAE (repeated from Stage 1)
- Latent Decoder (black box)
- Transformed Audio Output (beige box labeled "sounds like pitch 'C4'")
### Detailed Analysis
1. **Latent Reconstruction Flow**:
- Audio → Latent Encoder → Latent Vector
- Latent Vector → SAE₁ → Sparse Activations → SAE₂ → Reconstruction Loss loop
2. **Acoustic Concept Mapping**:
- SAE outputs connect to three linear weight matrices (W)
- Each W matrix maps to a specific acoustic concept:
- Pitch: Mapped to C4 note (middle C on piano)
- Loudness: Represented by speaker icon
- Timbre: Color-coded gradient (white→green)
3. **Audio Transformation**:
- SAE activations combined with W matrices
- Final output: Audio transformed to sound like specific pitch (C4)
### Key Observations
1. **Sparse Activation Pattern**:
- Vertical gray bars show non-uniform activation distribution
- Indicates SAE's ability to capture essential features while maintaining sparsity
2. **Concept-Specific Mappings**:
- Pitch mapping includes explicit note designation ("C4")
- Timbre uses color gradients rather than discrete categories
- Loudness represented by physical iconography
3. **Bidirectional Flow**:
- Training loop (Stage 1) enables reconstruction
- Linear mappings (Stage 2) allow concept interpretation
- Combined system (Stage 3) enables audio transformation
### Interpretation
This architecture demonstrates a neuro-symbolic approach to audio processing:
1. **SAE Training**: First establishes efficient latent representations through reconstruction loss minimization
2. **Concept Bridging**: Linear mappings translate abstract latents into interpretable dimensions (pitch/loudness/timbre)
3. **Practical Application**: Enables targeted audio manipulation by modifying specific acoustic properties
The use of sparse activations suggests the system identifies critical features for reconstruction, while the linear mappings provide a transparent bridge between neural representations and human-understandable sound properties. The explicit note designation ("C4") indicates the system can target specific musical intervals, suggesting potential applications in music generation and audio editing.
