# Technical Document Extraction: Model Performance vs Temperature ## Overview The image contains three comparative line graphs analyzing model performance metrics across varying temperature parameters (τ). Each graph evaluates different evaluation criteria: F1 Score, Exact Match (%), and Semantic Match (%). Four language models are compared: - Qwen2 (7B) - Mistral (7B) - Gemma 2 (2B) - GPT-2 (163M) --- ## Graph 1: F1 Score vs Temperature ### Axes - **X-axis**: Temperature (τ) [0.001, 0.25, 0.5, 0.75, 1, 1.5] - **Y-axis**: F1 Score [0, 10, 20, ..., 70] ### Legend - **Placement**: Right side of graph - **Color Mapping**: - Qwen2 (7B): Green - Mistral (7B): Teal - Gemma 2 (2B): Blue - GPT-2 (163M): Purple ### Key Trends 1. **Qwen2 (7B)**: - Starts at ~70 F1 Score at τ=0.001 - Gradual decline to ~60 at τ=1.5 - Shaded confidence interval narrows slightly with increasing τ 2. **Mistral (7B)**: - Starts at ~65 F1 Score at τ=0.001 - Declines to ~55 at τ=1.5 - Confidence interval widens moderately 3. **Gemma 2 (2B)**: - Starts at ~50 F1 Score at τ=0.001 - Drops to ~40 at τ=1.5 - Confidence interval remains relatively stable 4. **GPT-2 (163M)**: - Starts at ~5 F1 Score at τ=0.001 - Declines to ~3 at τ=1.5 - Confidence interval shows minimal variation ### Data Points (Approximate) | Model | τ=0.001 | τ=0.25 | τ=0.5 | τ=0.75 | τ=1.0 | τ=1.5 | |----------------|---------|--------|-------|--------|-------|-------| | Qwen2 (7B) | 70 | 68 | 65 | 63 | 61 | 60 | | Mistral (7B) | 65 | 63 | 60 | 58 | 55 | 53 | | Gemma 2 (2B) | 50 | 48 | 45 | 43 | 40 | 38 | | GPT-2 (163M) | 5 | 4.5 | 4 | 3.5 | 3 | 2.5 | --- ## Graph 2: Exact Match (%) vs Temperature ### Axes - **X-axis**: Temperature (τ) [0.001, 0.25, 0.5, 0.75, 1, 1.5] - **Y-axis**: Exact Match (%) [0, 10, 20, ..., 60] ### Legend - **Placement**: Right side of graph - **Color Mapping**: Same as Graph 1 ### Key Trends 1. **Qwen2 (7B)**: - Starts at ~60% at τ=0.001 - Declines to ~55% at τ=1.5 - Confidence interval narrows slightly 2. **Mistral (7B)**: - Starts at ~55% at τ=0.001 - Drops to ~50% at τ=1.5 - Confidence interval widens moderately 3. **Gemma 2 (2B)**: - Starts at ~40% at τ=0.001 - Declines to ~35% at τ=1.5 - Confidence interval remains stable 4. **GPT-2 (163M)**: - Starts at ~1% at τ=0.001 - Drops to ~0.5% at τ=1.5 - Confidence interval shows minimal variation ### Data Points (Approximate) | Model | τ=0.001 | τ=0.25 | τ=0.5 | τ=0.75 | τ=1.0 | τ=1.5 | |----------------|---------|--------|-------|--------|-------|-------| | Qwen2 (7B) | 60 | 58 | 56 | 54 | 52 | 50 | | Mistral (7B) | 55 | 53 | 50 | 48 | 45 | 43 | | Gemma 2 (2B) | 40 | 38 | 35 | 33 | 30 | 28 | | GPT-2 (163M) | 1 | 0.8 | 0.6 | 0.5 | 0.4 | 0.3 | --- ## Graph 3: Semantic Match (%) vs Temperature ### Axes - **X-axis**: Temperature (τ) [0.001, 0.25, 0.5, 0.75, 1, 1.5] - **Y-axis**: Semantic Match (%) [0, 5, 10, ..., 70] ### Legend - **Placement**: Right side of graph - **Color Mapping**: Same as Graph 1 ### Key Trends 1. **Qwen2 (7B)**: - Starts at ~70% at τ=0.001 - Declines to ~60% at τ=1.5 - Confidence interval narrows slightly 2. **Mistral (7B)**: - Starts at ~65% at τ=0.001 - Drops to ~55% at τ=1.5 - Confidence interval widens moderately 3. **Gemma 2 (2B)**: - Starts at ~50% at τ=0.001 - Declines to ~40% at τ=1.5 - Confidence interval remains stable 4. **GPT-2 (163M)**: - Starts at ~5% at τ=0.001 - Drops to ~4% at τ=1.5 - Confidence interval shows minimal variation ### Data Points (Approximate) | Model | τ=0.001 | τ=0.25 | τ=0.5 | τ=0.75 | τ=1.0 | τ=1.5 | |----------------|---------|--------|-------|--------|-------|-------| | Qwen2 (7B) | 70 | 68 | 65 | 63 | 61 | 60 | | Mistral (7B) | 65 | 63 | 60 | 58 | 55 | 53 | | Gemma 2 (2B) | 50 | 48 | 45 | 43 | 40 | 38 | | GPT-2 (163M) | 5 | 4.5 | 4 | 3.5 | 3 | 2.5 | --- ## Observations 1. **Temperature Sensitivity**: - All models show performance degradation as temperature increases - Larger models (Qwen2, Mistral) maintain higher performance across τ ranges 2. **Model Hierarchy**: - Qwen2 > Mistral > Gemma 2 > GPT-2 in all metrics - Performance gaps widen at higher τ values 3. **Confidence Intervals**: - Wider intervals at higher τ values suggest increased uncertainty - GPT-2 shows the most stable confidence intervals despite lowest performance 4. **Performance Plateaus**: - All models exhibit diminishing returns beyond τ=0.5 - GPT-2 shows near-linear decline across all τ values --- ## Critical Notes - All graphs use identical τ ranges and scaling - Shaded areas represent 95% confidence intervals - No textual annotations present beyond axis labels and legends - No non-English text detected in the image