## Bar Chart: Faithful vs. Error Reasoning in WebQSP and CWQ
### Overview
The image presents a comparative bar chart analyzing the performance of two reasoning frameworks ("GCR" and "GCR w/o constraint") across two datasets ("WebQSP" and "CWQ"). The chart distinguishes between "Faithful Reasoning" (blue) and "Error Reasoning" (pink), with percentages indicating the proportion of "Answer Hits" attributed to each reasoning type.
### Components/Axes
- **X-Axis**:
- Labels: "GCR" and "GCR w/o constraint" (split into two sub-categories per dataset).
- **Y-Axis**:
- Label: "Answer Hit" (percentage scale from 0 to 60).
- **Legend**:
- Position: Top of the chart.
- Colors:
- Blue = Faithful Reasoning
- Pink = Error Reasoning
- **Datasets**:
- WebQSP (left chart)
- CWQ (right chart)
### Detailed Analysis
#### WebQSP Dataset
- **GCR**:
- Faithful Reasoning: 100.0% (blue bar).
- Error Reasoning: 62.4% (pink bar).
- **GCR w/o constraint**:
- Faithful Reasoning: 62.4% (blue bar).
- Error Reasoning: 100.0% (pink bar).
#### CWQ Dataset
- **GCR**:
- Faithful Reasoning: 100.0% (blue bar).
- Error Reasoning: 48.1% (pink bar).
- **GCR w/o constraint**:
- Faithful Reasoning: 48.1% (blue bar).
- Error Reasoning: 100.0% (pink bar).
### Key Observations
1. **Faithful Reasoning Dominance**:
- Both datasets achieve 100% Faithful Reasoning under the "GCR" framework.
2. **Impact of Removing Constraints**:
- Removing constraints ("GCR w/o constraint") reduces Faithful Reasoning to match the Error Reasoning percentage (e.g., WebQSP: 100% → 62.4%; CWQ: 100% → 48.1%).
3. **Dataset-Specific Differences**:
- WebQSP shows a larger drop in Faithful Reasoning (37.6% decrease) compared to CWQ (51.9% decrease) when constraints are removed.
- Error Reasoning increases proportionally to the loss of Faithful Reasoning in both cases.
### Interpretation
The data demonstrates that constraints in the "GCR" framework are critical for maintaining high Faithful Reasoning performance. Removing constraints leads to a direct trade-off: Faithful Reasoning collapses to the level of Error Reasoning, suggesting that constraints act as a safeguard against errors. The disparity between WebQSP and CWQ implies that the datasets may differ in complexity or structure, affecting how constraints mitigate errors. This highlights the importance of constraint design in reasoning systems to balance accuracy and reliability.
