## Scatter Plot: Test Set Accuracy vs. Network Parameters
### Overview
The image is a scatter plot comparing the test set accuracy of various neural network architectures against their number of parameters. It highlights the relationship between model complexity (parameters) and performance (accuracy), with a focus on the gap between "FP" (Fine-tuning) and "1-day" accuracy metrics. A key annotation states: *"For robust networks, this gap is minimized."*
### Components/Axes
- **X-axis**: "Number of Network Parameters" (logarithmic scale, 10⁶ to 10⁷).
- **Y-axis**: "Test Set Accuracy (%)" (linear scale, 80% to 100%).
- **Legend**:
- **Networks**: Resnet20, Resnet32, Resnext29, Wide Resnet, AnalogNAS_T500.
- **Metrics**:
- **FP** (open symbols: hexagon, square, cross, diamond, circle).
- **1-day** (filled symbols: same shapes as FP but colored).
- **Colors**:
- Resnet20: Red (hexagon).
- Resnet32: Blue (square).
- Resnext29: Green (cross).
- Wide Resnet: Yellow (diamond).
- AnalogNAS_T500: Purple (circle).
### Detailed Analysis
- **Resnet20**:
- FP (open red hexagon): ~10⁶ parameters, 87.5% accuracy.
- 1-day (filled red hexagon): ~10⁶ parameters, 85% accuracy.
- **Resnet32**:
- FP (open blue square): ~10⁶.⁵ parameters, 92.5% accuracy.
- 1-day (filled blue square): ~10⁶.⁵ parameters, 90% accuracy.
- **Resnext29**:
- FP (open green cross): ~10⁶.⁷ parameters, 95% accuracy.
- 1-day (filled green cross): ~10⁶.⁷ parameters, 92.5% accuracy.
- **Wide Resnet**:
- FP (open yellow diamond): ~10⁷ parameters, 97.5% accuracy.
- 1-day (filled yellow diamond): ~10⁷ parameters, 95% accuracy.
- **AnalogNAS_T500**:
- FP (open purple circle): ~10⁶.⁸ parameters, 93% accuracy.
- 1-day (filled purple circle): ~10⁶.⁸ parameters, 91% accuracy.
### Key Observations
1. **Parameter-Accuracy Correlation**:
- Higher parameter counts generally correlate with higher accuracy (e.g., Wide Resnet at 10⁷ parameters achieves 97.5% accuracy vs. Resnet20 at 10⁶ parameters with 87.5%).
2. **FP vs. 1-day Gap**:
- All networks show a consistent ~2.5% gap between FP and 1-day accuracy, except AnalogNAS_T500, which has a smaller gap (~2%).
3. **Robustness Note**:
- The annotation suggests that networks with higher parameters (e.g., Wide Resnet, AnalogNAS_T500) exhibit minimized gaps, aligning with the definition of "robustness" in this context.
### Interpretation
The plot demonstrates that increasing model complexity (parameters) improves test set accuracy, but robust networks (those with higher parameters) maintain smaller gaps between FP and 1-day accuracy. This implies that larger, more complex models may generalize better across different evaluation scenarios. The exception of AnalogNAS_T500, with a smaller gap despite fewer parameters, suggests architectural efficiency can also contribute to robustness. The consistent gap across most networks indicates a trade-off between parameter count and performance stability.
