\n ## Bar Chart: Refusal Ratio by Training and Testing Set ### Overview This bar chart displays the refusal ratio (in percentage) for different testing sets (Factual Association, Associative Hallucination, and Unassociated Hallucination) trained on different training sets (UH Only and AH Only). The chart compares the performance of a system in refusing to answer questions based on the type of hallucination or factual association present in the testing data, and the type of data used for training. ### Components/Axes * **X-axis:** "Training Set" with two categories: "UH Only" and "AH Only". * **Y-axis:** "Refusal Ratio (%)" ranging from 0 to 100, with tick marks at intervals of 20. * **Legend (top-right):** "Testing set" with three categories: * "Factual Asso." (represented by green) * "Asso. Hallu." (represented by blue) * "Unasso. Halluc." (represented by red) ### Detailed Analysis The chart consists of six bars, grouped by training set. **UH Only Training Set:** * **Factual Asso. (Green):** The bar rises to approximately 10%. * **Asso. Hallu. (Blue):** The bar rises to approximately 15%. * **Unasso. Halluc. (Red):** The bar rises to approximately 90%. **AH Only Training Set:** * **Factual Asso. (Green):** The bar rises to approximately 20%. * **Asso. Hallu. (Blue):** The bar rises to approximately 20%. * **Unasso. Halluc. (Red):** The bar rises to approximately 45%. ### Key Observations * The refusal ratio is significantly higher for "Unasso. Halluc." in both training set scenarios. * Training on "UH Only" results in a much higher refusal ratio for "Unasso. Halluc." compared to training on "AH Only". * The refusal ratio for "Factual Asso." and "Asso. Hallu." is relatively low and similar across both training sets. * The "AH Only" training set shows a more balanced refusal ratio across all testing sets compared to the "UH Only" training set. ### Interpretation The data suggests that the system is much more likely to refuse to answer questions that involve unassociated hallucinations, regardless of the training data. However, the training data significantly impacts the refusal rate for unassociated hallucinations. Training solely on "UH Only" data leads to a very high refusal rate for unassociated hallucinations, indicating the model has learned to be highly cautious in such scenarios. Conversely, training on "AH Only" data results in a lower refusal rate for unassociated hallucinations, suggesting the model is more willing to attempt answering even in the presence of unassociated hallucinations. The relatively low refusal rates for "Factual Asso." and "Asso. Hallu." indicate that the system is generally comfortable answering questions that involve factual associations or associative hallucinations. The similar refusal rates across both training sets for these categories suggest that the training data has a less pronounced effect on the system's behavior in these cases. The difference in refusal rates between the training sets highlights the importance of the training data in shaping the system's response to different types of hallucinations. A system trained on a more diverse dataset (potentially including both UH and AH data) might exhibit a more nuanced and balanced refusal behavior.