## Bar Charts: Model Pruning and Efficiency Analysis
### Overview
The image contains three bar charts comparing layer pruning strategies in neural networks. The charts analyze α-projection values, pruning impact on performance, and efficiency ratios across different pruning approaches.
### Components/Axes
1. **Pythia-IB (16L, 1B) α Profile**
- X-axis: Layer Index (0-16)
- Y-axis: Q-projection α (0.00-0.20)
- Legend:
- Orange = Boundary
- Red = Interior
- Blue = Pruned (k=4)
2. **Pruning Strategy Comparison**
- X-axis: Layers Removed (k=1-4)
- Y-axis: ΔPPL (log scale, 10¹-10²)
- Legend:
- Green = Zone-Aware Spectral
- Orange = Last-N
- Gray = Random (3 seeds)
3. **Zone-Aware Advantage**
- X-axis: Layers Removed (k=1-4)
- Y-axis: Efficiency Ratio (Last-N / Zone-Aware)
- Bars: Green with ratio multipliers (1.1x-3.6x)
### Detailed Analysis
**1. Pythia-IB α Profile**
- Boundary values start at ~0.21 (Layer 0) and decrease to ~0.17 (Layer 16)
- Interior values fluctuate between ~0.16-0.19 across layers
- Pruned (k=4) values remain relatively stable at ~0.17-0.19
- Spatial pattern: Orange bars (Boundary) consistently highest, blue bars (Pruned) lowest
**2. Pruning Strategy Comparison**
- Zone-Aware Spectral (green) shows exponential growth:
- k=1: ~2
- k=2: ~10
- k=3: ~50
- k=4: ~150
- Last-N (orange) shows similar growth but with higher values:
- k=1: ~5
- k=2: ~20
- k=3: ~80
- k=4: ~200
- Random (gray) shows minimal growth:
- k=1: ~1
- k=2: ~10
- k=3: ~30
- k=4: ~100
**3. Zone-Aware Advantage**
- Efficiency ratios increase with more layers pruned:
- k=1: 1.1x
- k=2: 1.3x
- k=3: 2.6x
- k=4: 3.6x
- Visual trend: Steep upward trajectory in green bars
### Key Observations
1. Boundary layers maintain highest α-projection values throughout the network
2. Zone-Aware Spectral pruning demonstrates superior performance scaling (ΔPPL grows 75x from k=1 to k=4)
3. Efficiency advantage grows exponentially with more layers pruned (3.6x at k=4)
4. Random pruning shows diminishing returns despite similar ΔPPL values
### Interpretation
The data suggests Zone-Aware Spectral pruning provides optimal balance between performance retention and efficiency gains. While Last-N pruning achieves higher absolute performance metrics (ΔPPL), Zone-Aware maintains better efficiency ratios (3.6x at k=4). The exponential growth patterns indicate that deeper pruning yields disproportionately better results with Zone-Aware methods. The stable α-projection values in pruned layers suggest effective knowledge distillation during the pruning process. These findings support Zone-Aware Spectral as the preferred pruning strategy for maintaining model performance while maximizing computational efficiency.
