## Chart: Training Loss Comparison and Distribution Probabilities ### Overview The image presents two charts side-by-side. The left chart compares the training loss of different geometric distributions against a uniform distribution. The right chart shows the distribution probabilities for each geometric distribution and the uniform distribution across four UT Steps. ### Components/Axes **Left Chart: Training Loss Comparison** * **Title:** Training Loss Comparison * **X-axis:** Training Steps, ranging from 20000 to 40000 in increments of 5000. * **Y-axis:** Loss (300-step sliding average), ranging from 2.275 to 2.450 in increments of 0.025. * **Legend (Top-Right):** * Purple: Geometric λ=0.1 * Dark Blue: Geometric λ=0.2 * Blue: Geometric λ=0.3 * Teal: Geometric λ=0.4 * Green-Teal: Geometric λ=0.5 * Green: Geometric λ=0.6 * Light Green: Geometric λ=0.7 * Yellow-Green: Geometric λ=0.8 * Yellow: Geometric λ=0.9 * Red: Uniform **Right Chart: Distribution Probabilities** * **Title:** Distribution Probabilities * **X-axis:** UT Step, ranging from 1 to 4 in increments of 1. * **Y-axis:** Probability, ranging from 0.0 to 1.0 in increments of 0.2. * **Legend (Top-Right):** Same as the left chart. * Purple: Geometric λ=0.1 * Dark Blue: Geometric λ=0.2 * Blue: Geometric λ=0.3 * Teal: Geometric λ=0.4 * Green-Teal: Geometric λ=0.5 * Green: Geometric λ=0.6 * Light Green: Geometric λ=0.7 * Yellow-Green: Geometric λ=0.8 * Yellow: Geometric λ=0.9 * Red: Uniform ### Detailed Analysis **Left Chart: Training Loss Comparison** * **Geometric λ=0.1 to λ=0.9:** The lines representing geometric distributions (λ=0.1 to λ=0.9) start at approximately the same loss value of 2.450 at 20000 training steps. They fluctuate slightly before generally decreasing to approximately 2.300 by 40000 training steps. * **Uniform:** The uniform distribution starts at approximately 2.450 at 20000 training steps. It decreases more rapidly than the geometric distributions, reaching approximately 2.225 by 40000 training steps. **Right Chart: Distribution Probabilities** * **Geometric Distributions (λ=0.1 to λ=0.9):** * All geometric distributions start with varying probabilities at UT Step 1 and decrease as the UT Step increases. * Geometric λ=0.1 (Purple): Starts at approximately 0.35 and decreases to approximately 0.25 by UT Step 4. * Geometric λ=0.2 (Dark Blue): Starts at approximately 0.50 and decreases to approximately 0.25 by UT Step 4. * Geometric λ=0.3 (Blue): Starts at approximately 0.60 and decreases to approximately 0.25 by UT Step 4. * Geometric λ=0.4 (Teal): Starts at approximately 0.68 and decreases to approximately 0.25 by UT Step 4. * Geometric λ=0.5 (Green-Teal): Starts at approximately 0.75 and decreases to approximately 0.25 by UT Step 4. * Geometric λ=0.6 (Green): Starts at approximately 0.80 and decreases to approximately 0.25 by UT Step 4. * Geometric λ=0.7 (Light Green): Starts at approximately 0.85 and decreases to approximately 0.25 by UT Step 4. * Geometric λ=0.8 (Yellow-Green): Starts at approximately 0.90 and decreases to approximately 0.25 by UT Step 4. * Geometric λ=0.9 (Yellow): Starts at approximately 0.95 and decreases to approximately 0.25 by UT Step 4. * **Uniform:** The uniform distribution (Red) maintains a constant probability of approximately 0.25 across all UT Steps. ### Key Observations * In the Training Loss Comparison, the uniform distribution consistently achieves a lower loss than the geometric distributions as training progresses. * In the Distribution Probabilities, the geometric distributions have higher initial probabilities that decrease with each UT Step, while the uniform distribution maintains a constant probability. * All geometric distributions converge to a similar probability value (approximately 0.25) by UT Step 4. ### Interpretation The charts suggest that, in this specific training scenario, the uniform distribution leads to a lower training loss compared to the geometric distributions. The distribution probabilities indicate that the geometric distributions initially favor earlier UT Steps but become more uniform-like as the UT Step increases. The uniform distribution, with its constant probability across all UT Steps, appears to be a more effective strategy for minimizing training loss in this case. The convergence of geometric distributions to a similar probability at higher UT Steps suggests that the impact of the initial distribution diminishes over time.

## Charts: Training Loss Comparison & Distribution Probabilities ### Overview The image contains two charts. The left chart displays a comparison of training loss over training steps for different geometric distributions and a uniform distribution. The right chart shows the probability distributions for the same geometric distributions and the uniform distribution across UT steps. ### Components/Axes **Left Chart: Training Loss Comparison** * **Title:** Training Loss Comparison * **X-axis:** Training Steps (ranging approximately from 20000 to 40000) * **Y-axis:** Loss (300-step sliding average) (ranging approximately from 2.27 to 2.45) * **Legend (top-right):** * Geometric λ=0.1 (purple) * Geometric λ=0.2 (dark blue) * Geometric λ=0.3 (blue) * Geometric λ=0.4 (light blue) * Geometric λ=0.5 (green) * Geometric λ=0.6 (yellow-green) * Geometric λ=0.7 (yellow) * Geometric λ=0.8 (orange) * Geometric λ=0.9 (red) * Uniform (red) **Right Chart: Distribution Probabilities** * **Title:** Distribution Probabilities * **X-axis:** UT Step (ranging from 0.0 to 4.0) * **Y-axis:** Probability (ranging from 0.0 to 1.0) * **Legend (top-right):** * Geometric λ=0.1 (purple) * Geometric λ=0.2 (dark blue) * Geometric λ=0.3 (blue) * Geometric λ=0.4 (light blue) * Geometric λ=0.5 (green) * Geometric λ=0.6 (yellow-green) * Geometric λ=0.7 (yellow) * Geometric λ=0.8 (orange) * Geometric λ=0.9 (red) * Uniform (red) ### Detailed Analysis or Content Details **Left Chart: Training Loss Comparison** * **Geometric λ=0.1 (purple):** The line starts at approximately 2.43 at 20000 steps, fluctuates slightly, and decreases to approximately 2.32 at 40000 steps. * **Geometric λ=0.2 (dark blue):** The line starts at approximately 2.44 at 20000 steps, decreases to approximately 2.34 at 35000 steps, and then increases slightly to approximately 2.35 at 40000 steps. * **Geometric λ=0.3 (blue):** The line starts at approximately 2.44 at 20000 steps, decreases to approximately 2.33 at 35000 steps, and then increases slightly to approximately 2.34 at 40000 steps. * **Geometric λ=0.4 (light blue):** The line starts at approximately 2.44 at 20000 steps, decreases to approximately 2.32 at 35000 steps, and then increases slightly to approximately 2.33 at 40000 steps. * **Geometric λ=0.5 (green):** The line starts at approximately 2.44 at 20000 steps, decreases to approximately 2.31 at 35000 steps, and then increases slightly to approximately 2.32 at 40000 steps. * **Geometric λ=0.6 (yellow-green):** The line starts at approximately 2.44 at 20000 steps, decreases to approximately 2.30 at 35000 steps, and then increases slightly to approximately 2.31 at 40000 steps. * **Geometric λ=0.7 (yellow):** The line starts at approximately 2.44 at 20000 steps, decreases to approximately 2.29 at 35000 steps, and then increases slightly to approximately 2.30 at 40000 steps. * **Geometric λ=0.8 (orange):** The line starts at approximately 2.44 at 20000 steps, decreases to approximately 2.28 at 35000 steps, and then increases slightly to approximately 2.29 at 40000 steps. * **Geometric λ=0.9 (red):** The line starts at approximately 2.44 at 20000 steps, decreases rapidly to approximately 2.27 at 35000 steps, and then increases slightly to approximately 2.28 at 40000 steps. * **Uniform (red):** The line starts at approximately 2.43 at 20000 steps, decreases rapidly to approximately 2.27 at 35000 steps, and then increases sharply to approximately 2.38 at 40000 steps. **Right Chart: Distribution Probabilities** * **Geometric λ=0.1 (purple):** Starts at approximately 0.9, decreases to approximately 0.1 at UT Step 4. * **Geometric λ=0.2 (dark blue):** Starts at approximately 0.8, decreases to approximately 0.15 at UT Step 4. * **Geometric λ=0.3 (blue):** Starts at approximately 0.7, decreases to approximately 0.2 at UT Step 4. * **Geometric λ=0.4 (light blue):** Starts at approximately 0.6, decreases to approximately 0.25 at UT Step 4. * **Geometric λ=0.5 (green):** Starts at approximately 0.5, decreases to approximately 0.3 at UT Step 4. * **Geometric λ=0.6 (yellow-green):** Starts at approximately 0.4, decreases to approximately 0.35 at UT Step 4. * **Geometric λ=0.7 (yellow):** Starts at approximately 0.3, decreases to approximately 0.4 at UT Step 4. * **Geometric λ=0.8 (orange):** Starts at approximately 0.2, decreases to approximately 0.45 at UT Step 4. * **Geometric λ=0.9 (red):** Starts at approximately 0.1, decreases to approximately 0.5 at UT Step 4. * **Uniform (red):** Remains relatively constant at approximately 0.25 across all UT Steps. ### Key Observations * In the Training Loss Comparison chart, the lines generally decrease with increasing training steps, indicating learning. The Uniform distribution and Geometric λ=0.9 distributions show the most rapid decrease in loss. * The Uniform distribution exhibits a significant increase in loss at the final training steps. * In the Distribution Probabilities chart, the geometric distributions exhibit an exponential decay in probability as the UT Step increases. The rate of decay is determined by the λ parameter, with higher λ values resulting in slower decay. * The Uniform distribution maintains a relatively constant probability across all UT Steps. ### Interpretation The charts demonstrate the impact of different geometric distributions on training loss and probability distributions. The training loss comparison suggests that higher λ values (closer to 1) in the geometric distribution lead to faster initial learning (steeper loss decrease) but may also result in instability or overfitting at later stages, as evidenced by the Uniform distribution's late-stage loss increase. The distribution probabilities chart illustrates how the λ parameter controls the concentration of probability mass. Lower λ values concentrate probability on the first UT step, while higher λ values distribute probability more evenly across the UT steps. The Uniform distribution represents an equal probability for each UT step. The relationship between the two charts suggests that the choice of distribution can influence both the speed and stability of the learning process. The geometric distributions with λ values closer to 1 may be more prone to overfitting, while the Uniform distribution may require more training steps to achieve a similar level of performance. The outlier behavior of the Uniform distribution at the end of the training process warrants further investigation. It could indicate a problem with the optimization process or a need for regularization.

## Line Charts: Training Loss Comparison and Distribution Probabilities ### Overview The image contains two line charts comparing training dynamics and distribution probabilities across different hyperparameter settings. The left chart tracks training loss convergence over steps, while the right chart visualizes probability distributions across update (UT) steps for geometric and uniform distributions. ### Components/Axes **Left Chart (Training Loss Comparison):** - **X-axis**: Training Steps (20,000 to 40,000) - **Y-axis**: Loss (300-step sliding average) ranging from 2.275 to 2.450 - **Legend**: - Geometric distributions (λ=0.1 to λ=0.9) in gradient colors (purple to yellow) - Uniform distribution (red line) **Right Chart (Distribution Probabilities):** - **X-axis**: UT Step (1 to 4) - **Y-axis**: Probability (0.0 to 1.0) - **Legend**: - Geometric distributions (λ=0.1 to λ=0.9) in gradient colors (purple to yellow) - Uniform distribution (red line) ### Detailed Analysis **Left Chart Trends:** 1. **Uniform Distribution (Red Line)**: - Starts at ~2.45 loss at 20,000 steps - Gradually decreases to ~2.275 by 40,000 steps - Shows smooth, consistent decline 2. **Geometric Distributions**: - All λ values start above Uniform (~2.42–2.45 at 20k steps) - Converge toward Uniform performance by ~35,000 steps - λ=0.1 (purple) shows slowest convergence - λ=0.9 (yellow) shows fastest convergence - λ=0.5 (green) and λ=0.7 (light green) show intermediate convergence **Right Chart Trends:** 1. **Uniform Distribution (Red Line)**: - Maintains flat probability of ~0.25 across all UT steps 2. **Geometric Distributions**: - λ=0.1 (purple): Starts at ~0.35, drops to ~0.2 by UT step 4 - λ=0.9 (yellow): Starts at ~0.9, drops to near 0 by UT step 4 - Higher λ values show steeper declines - All geometric lines cross below Uniform probability by UT step 3 ### Key Observations 1. **Training Loss Convergence**: - Geometric models with λ > 0.5 outperform λ < 0.5 by 35,000 steps - Uniform model maintains ~0.15 lower loss than initial geometric values - All models show reduced variance in loss after 30,000 steps 2. **Distribution Probability Dynamics**: - Higher λ geometric models exhibit greater initial confidence (probability) - Confidence drops sharply for λ > 0.5 by UT step 3 - Uniform model maintains stable, moderate confidence ### Interpretation The data demonstrates that geometric distributions with higher λ values (λ > 0.5) achieve faster convergence in training loss but exhibit higher initial confidence that rapidly declines. This suggests a trade-off between early performance and stability. The uniform distribution serves as a stable baseline with moderate performance and consistent confidence. Notably, the geometric models' probability distributions show a "confidence correction" pattern - starting with overconfidence (high probability) that diminishes as training progresses. This aligns with the loss convergence pattern, where initial overestimation of model capability (high loss) improves over time. The spatial relationship between the two charts reveals complementary insights: the left chart shows performance improvement over time, while the right chart reveals how confidence calibration evolves. The uniform model's stability in both metrics suggests it represents a well-calibrated, reliable baseline for comparison.