\n ## Bar Chart: Refusal Ratio by Training Set and Testing Set ### Overview This bar chart displays the refusal ratio (in percentage) for different testing sets based on the training set used. The chart compares the performance of a system across two training sets: "UH Only" and "AH Only", and three testing sets: "Factual Association", "Association Hallucination", and "Unassociated Hallucination". ### Components/Axes * **X-axis:** "Training Set" with categories "UH Only" and "AH Only". * **Y-axis:** "Refusal Ratio (%)" ranging from 0 to 100, with tick marks at intervals of 20. * **Legend (top-right):** "Testing set" with labels: * "Factual Asso." (Green) * "Asso. Hallu." (Blue) * "Unasso. Halluc." (Red) ### Detailed Analysis The chart consists of six bars, grouped by training set. **UH Only Training Set:** * **Factual Asso. (Green):** The bar rises to approximately 30%. The line slopes upward. * **Asso. Hallu. (Blue):** The bar rises to approximately 10%. The line is relatively flat. * **Unasso. Halluc. (Red):** The bar rises to approximately 80%. The line slopes sharply upward. **AH Only Training Set:** * **Factual Asso. (Green):** The bar rises to approximately 20%. The line slopes upward. * **Asso. Hallu. (Blue):** The bar rises to approximately 35%. The line slopes upward. * **Unasso. Halluc. (Red):** The bar rises to approximately 25%. The line slopes upward. ### Key Observations * The "Unasso. Halluc." testing set consistently results in the highest refusal ratio, regardless of the training set. * The "UH Only" training set leads to a significantly higher refusal ratio for "Unasso. Halluc." compared to the "AH Only" training set. * The "AH Only" training set shows a higher refusal ratio for "Asso. Hallu." compared to the "UH Only" training set. * The "Factual Asso." testing set has the lowest refusal ratio in both training set scenarios. ### Interpretation The data suggests that the system struggles more with unassociated hallucinations, as evidenced by the consistently high refusal ratios for this testing set. This indicates that the system is more likely to reject prompts that involve generating content without a clear factual basis. The difference in refusal ratios between the "UH Only" and "AH Only" training sets highlights the impact of the training data on the system's behavior. Training with "AH Only" seems to improve the handling of associated hallucinations ("Asso. Hallu."), but it doesn't significantly reduce the refusal rate for unassociated hallucinations. The low refusal ratio for "Factual Asso." suggests that the system is generally reliable when asked to generate content based on factual associations. This could be due to the system being trained on a large corpus of factual information. The chart demonstrates a trade-off between different types of hallucinations. Improving performance on one type of hallucination may come at the cost of performance on another. Further investigation is needed to understand the underlying reasons for these differences and to develop strategies for mitigating hallucinations across all testing sets.