## Line Graphs: Per-Period Regret Over Time Periods
### Overview
The image contains three line graphs comparing per-period regret across different time periods (t) for three strategies: (a) Fixed ε-greedy, (b) Annealing ε-greedy, and (c) Ensemble TS. Each graph shows multiple data series with distinct colors, representing variations in parameters (ε-values for ε-greedy strategies, ensemble sizes for TS). All graphs share identical axes: y-axis labeled "per-period regret" (0–60) and x-axis labeled "time period (t)" (0–500).
---
### Components/Axes
1. **Graph Titles**:
- (a) Fixed ε-greedy
- (b) Annealing ε-greedy
- (c) Ensemble TS
2. **Axes**:
- **Y-axis**: "per-period regret" (0–60, linear scale).
- **X-axis**: "time period (t)" (0–500, linear scale).
3. **Legends**:
- **(a) Fixed ε-greedy**:
- Colors: Red (ε=0.01), Teal (ε=0.2), Orange (ε=0.05), Purple (ε=0.3), Green (ε=0.1).
- **(b) Annealing ε-greedy**:
- Colors: Red (ε=10/(10+t)), Teal (ε=40/(40+t)), Orange (ε=20/(20+t)), Purple (ε=50/(50+t)), Green (ε=30/(30+t)).
- **(c) Ensemble TS**:
- Colors: Red (ensemble 3), Teal (ensemble 100), Orange (ensemble 10), Purple (ensemble 300), Green (ensemble 30).
4. **Legend Placement**:
- All legends are positioned at the top of their respective graphs, with labels aligned left-to-right.
---
### Detailed Analysis
#### (a) Fixed ε-greedy
- **Trends**: All lines start near 60 regret and decrease over time. Lower ε-values (e.g., ε=0.01, red) decline more sharply, while higher ε-values (e.g., ε=0.3, purple) decrease more gradually.
- **Key Data Points**:
- At t=500:
- ε=0.01 (red): ~25 regret.
- ε=0.3 (purple): ~35 regret.
- ε=0.1 (green) and ε=0.2 (teal) converge to ~28–30 regret.
#### (b) Annealing ε-greedy
- **Trends**: Lines start higher (~60 regret) and decline more gradually than fixed ε-greedy. Annealing schedules (e.g., ε=10/(10+t)) show slower decay due to time-dependent ε reduction.
- **Key Data Points**:
- At t=500:
- ε=10/(10+t) (red): ~30 regret.
- ε=50/(50+t) (purple): ~35 regret.
- ε=30/(30+t) (green) stabilizes near ~28 regret.
#### (c) Ensemble TS
- **Trends**: Lines start near 60 regret and drop sharply initially, then plateau. Larger ensembles (e.g., 300, purple) achieve lower regret faster than smaller ones (e.g., 3, red).
- **Key Data Points**:
- At t=500:
- Ensemble 3 (red): ~20 regret.
- Ensemble 300 (purple): ~10 regret.
- Ensemble 100 (teal) and 30 (green) converge to ~12–15 regret.
---
### Key Observations
1. **Fixed ε-greedy**: Lower ε-values (more greedy) achieve lower regret faster, but higher ε-values (more exploratory) stabilize at higher regret.
2. **Annealing ε-greedy**: Time-dependent ε reduction slows regret decline compared to fixed ε-greedy, suggesting adaptive exploration improves long-term performance.
3. **Ensemble TS**: Larger ensembles (e.g., 300) outperform smaller ones, with regret dropping sharply initially and stabilizing at lower values.
---
### Interpretation
The data demonstrates that exploration-exploitation trade-offs (via ε-greedy strategies) and ensemble diversity (via TS) significantly impact regret minimization. Fixed ε-greedy with low ε (e.g., 0.01) achieves the lowest regret but risks under-exploration. Annealing ε-greedy balances exploration over time, while Ensemble TS leverages diversity to reduce regret more effectively. Larger ensembles (e.g., 300) outperform smaller ones, highlighting the value of model aggregation. The sharp initial drops in Ensemble TS suggest rapid learning from diverse models, while annealing strategies adapt exploration dynamically.
