# Technical Document Extraction: Model Performance vs Temperature
## Overview
The image contains three comparative line graphs analyzing model performance metrics across varying temperature parameters (τ). Each graph evaluates different evaluation criteria: F1 Score, Exact Match (%), and Semantic Match (%). Four language models are compared:
- Qwen2 (7B)
- Mistral (7B)
- Gemma 2 (2B)
- GPT-2 (163M)
---
## Graph 1: F1 Score vs Temperature
### Axes
- **X-axis**: Temperature (τ) [0.001, 0.25, 0.5, 0.75, 1, 1.5]
- **Y-axis**: F1 Score [0, 10, 20, ..., 70]
### Legend
- **Placement**: Right side of graph
- **Color Mapping**:
- Qwen2 (7B): Green
- Mistral (7B): Teal
- Gemma 2 (2B): Blue
- GPT-2 (163M): Purple
### Key Trends
1. **Qwen2 (7B)**:
- Starts at ~70 F1 Score at τ=0.001
- Gradual decline to ~60 at τ=1.5
- Shaded confidence interval narrows slightly with increasing τ
2. **Mistral (7B)**:
- Starts at ~65 F1 Score at τ=0.001
- Declines to ~55 at τ=1.5
- Confidence interval widens moderately
3. **Gemma 2 (2B)**:
- Starts at ~50 F1 Score at τ=0.001
- Drops to ~40 at τ=1.5
- Confidence interval remains relatively stable
4. **GPT-2 (163M)**:
- Starts at ~5 F1 Score at τ=0.001
- Declines to ~3 at τ=1.5
- Confidence interval shows minimal variation
### Data Points (Approximate)
| Model | τ=0.001 | τ=0.25 | τ=0.5 | τ=0.75 | τ=1.0 | τ=1.5 |
|----------------|---------|--------|-------|--------|-------|-------|
| Qwen2 (7B) | 70 | 68 | 65 | 63 | 61 | 60 |
| Mistral (7B) | 65 | 63 | 60 | 58 | 55 | 53 |
| Gemma 2 (2B) | 50 | 48 | 45 | 43 | 40 | 38 |
| GPT-2 (163M) | 5 | 4.5 | 4 | 3.5 | 3 | 2.5 |
---
## Graph 2: Exact Match (%) vs Temperature
### Axes
- **X-axis**: Temperature (τ) [0.001, 0.25, 0.5, 0.75, 1, 1.5]
- **Y-axis**: Exact Match (%) [0, 10, 20, ..., 60]
### Legend
- **Placement**: Right side of graph
- **Color Mapping**: Same as Graph 1
### Key Trends
1. **Qwen2 (7B)**:
- Starts at ~60% at τ=0.001
- Declines to ~55% at τ=1.5
- Confidence interval narrows slightly
2. **Mistral (7B)**:
- Starts at ~55% at τ=0.001
- Drops to ~50% at τ=1.5
- Confidence interval widens moderately
3. **Gemma 2 (2B)**:
- Starts at ~40% at τ=0.001
- Declines to ~35% at τ=1.5
- Confidence interval remains stable
4. **GPT-2 (163M)**:
- Starts at ~1% at τ=0.001
- Drops to ~0.5% at τ=1.5
- Confidence interval shows minimal variation
### Data Points (Approximate)
| Model | τ=0.001 | τ=0.25 | τ=0.5 | τ=0.75 | τ=1.0 | τ=1.5 |
|----------------|---------|--------|-------|--------|-------|-------|
| Qwen2 (7B) | 60 | 58 | 56 | 54 | 52 | 50 |
| Mistral (7B) | 55 | 53 | 50 | 48 | 45 | 43 |
| Gemma 2 (2B) | 40 | 38 | 35 | 33 | 30 | 28 |
| GPT-2 (163M) | 1 | 0.8 | 0.6 | 0.5 | 0.4 | 0.3 |
---
## Graph 3: Semantic Match (%) vs Temperature
### Axes
- **X-axis**: Temperature (τ) [0.001, 0.25, 0.5, 0.75, 1, 1.5]
- **Y-axis**: Semantic Match (%) [0, 5, 10, ..., 70]
### Legend
- **Placement**: Right side of graph
- **Color Mapping**: Same as Graph 1
### Key Trends
1. **Qwen2 (7B)**:
- Starts at ~70% at τ=0.001
- Declines to ~60% at τ=1.5
- Confidence interval narrows slightly
2. **Mistral (7B)**:
- Starts at ~65% at τ=0.001
- Drops to ~55% at τ=1.5
- Confidence interval widens moderately
3. **Gemma 2 (2B)**:
- Starts at ~50% at τ=0.001
- Declines to ~40% at τ=1.5
- Confidence interval remains stable
4. **GPT-2 (163M)**:
- Starts at ~5% at τ=0.001
- Drops to ~4% at τ=1.5
- Confidence interval shows minimal variation
### Data Points (Approximate)
| Model | τ=0.001 | τ=0.25 | τ=0.5 | τ=0.75 | τ=1.0 | τ=1.5 |
|----------------|---------|--------|-------|--------|-------|-------|
| Qwen2 (7B) | 70 | 68 | 65 | 63 | 61 | 60 |
| Mistral (7B) | 65 | 63 | 60 | 58 | 55 | 53 |
| Gemma 2 (2B) | 50 | 48 | 45 | 43 | 40 | 38 |
| GPT-2 (163M) | 5 | 4.5 | 4 | 3.5 | 3 | 2.5 |
---
## Observations
1. **Temperature Sensitivity**:
- All models show performance degradation as temperature increases
- Larger models (Qwen2, Mistral) maintain higher performance across τ ranges
2. **Model Hierarchy**:
- Qwen2 > Mistral > Gemma 2 > GPT-2 in all metrics
- Performance gaps widen at higher τ values
3. **Confidence Intervals**:
- Wider intervals at higher τ values suggest increased uncertainty
- GPT-2 shows the most stable confidence intervals despite lowest performance
4. **Performance Plateaus**:
- All models exhibit diminishing returns beyond τ=0.5
- GPT-2 shows near-linear decline across all τ values
---
## Critical Notes
- All graphs use identical τ ranges and scaling
- Shaded areas represent 95% confidence intervals
- No textual annotations present beyond axis labels and legends
- No non-English text detected in the image
