## Bar Chart: Helpfulness and Harmlessness Evaluation of Language Models ### Overview The image presents two bar charts comparing the "Helpfulness Evaluation" and "Harmlessness Evaluation" of various language models. The charts display Elo ratings on the y-axis for different models and configurations on the x-axis. ### Components/Axes **Top Chart: Helpfulness Evaluation** * **Title:** Helpfulness Evaluation * **Y-Axis:** Elo Rating, ranging from 0 to 1200. * **X-Axis:** Categorical axis listing different language models and configurations. The models include: * beaver-7b-v2.0 * SFT * beaver-7b-v3.0 * beaver-7b-v1.0 * SACPO (H->S) [0.1] * SACPO (P) [0.75] * SACPO (P) [0.99] * SACPO (P) [0.5] * SACPO (P) [0.90] * SACPO (P) [0.25] * SACPO (P) [0.95] * DPO (H) * Ra-DPO (H) * RSA (H->S) [0.1] * RSA (H+S) [0.01] * RSA (H->S) [0.025] * RSA (H+S) [0.05] * RSA (P) [0.25] * RSA (P) [0.5] * RSA (P) [0.99] * RSA (P) [0.95] * RSA (P) [0.90] * RSA (P) [0.75] * SACPO (H->S) [0.05] * SACPO (H->S) [0.01] * SACPO (H->S) [0.025] **Bottom Chart: Harmlessness Evaluation** * **Title:** Harmlessness Evaluation * **Y-Axis:** Elo Rating, ranging from 0 to 1400. * **X-Axis:** Categorical axis listing different language models and configurations. The models include: * DPO (H) * Ra-DPO (H) * SFT * SACPO (P) [0.25] * SACPO (H->S) [0.1] * RSA (H->S) [0.1] * RSA (P) [0.25] * beaver-7b-v1.0 * SACPO (P) [0.99] * SACPO (P) [0.95] * beaver-7b-v2.0 * beaver-7b-v3.0 * RSA (P) [0.5] * SACPO (P) [0.75] * RSA (H->S) [0.025] * RSA (P) [0.95] * SACPO (H->S) [0.025] * RSA (P) [0.90] * RSA (P) [0.99] * RSA (H+S) [0.05] * SACPO (P) [0.90] * RSA (H->S) [0.01] * SACPO (H->S) [0.05] * SACPO (H->S) [0.01] ### Detailed Analysis **Helpfulness Evaluation (Top Chart)** * **beaver-7b-v2.0 (Purple):** Elo rating of approximately 899. * **SFT (Gray):** Elo rating of approximately 1000. * **beaver-7b-v3.0 (Purple):** Elo rating of approximately 1047. * **beaver-7b-v1.0 (Purple):** Elo rating of approximately 1070. * **SACPO (H->S) [0.1] (Blue):** Elo rating of approximately 1103. * **SACPO (P) [0.75] (Green):** Elo rating of approximately 1120. * **SACPO (P) [0.99] (Green):** Elo rating of approximately 1128. * **SACPO (P) [0.5] (Green):** Elo rating of approximately 1141. * **SACPO (P) [0.90] (Green):** Elo rating of approximately 1154. * **SACPO (P) [0.25] (Green):** Elo rating of approximately 1170. * **SACPO (P) [0.95] (Green):** Elo rating of approximately 1177. * **DPO (H) (Blue):** Elo rating of approximately 1182. * **Ra-DPO (H) (Blue):** Elo rating of approximately 1182. * **RSA (H->S) [0.1] (Pink):** Elo rating of approximately 1190. * **RSA (H+S) [0.01] (Pink):** Elo rating of approximately 1194. * **RSA (H->S) [0.025] (Pink):** Elo rating of approximately 1214. * **RSA (H+S) [0.05] (Pink):** Elo rating of approximately 1243. * **RSA (P) [0.25] (Red):** Elo rating of approximately 1247. * **RSA (P) [0.5] (Red):** Elo rating of approximately 1251. * **RSA (P) [0.99] (Red):** Elo rating of approximately 1257. * **RSA (P) [0.95] (Red):** Elo rating of approximately 1267. * **RSA (P) [0.90] (Red):** Elo rating of approximately 1303. * **RSA (P) [0.75] (Red):** Elo rating of approximately 1306. * **SACPO (H->S) [0.05] (Blue):** Elo rating of approximately 1318. * **SACPO (H->S) [0.01] (Blue):** Elo rating of approximately 1321. * **SACPO (H->S) [0.025] (Blue):** Elo rating of approximately 1323. **Harmlessness Evaluation (Bottom Chart)** * **DPO (H) (Blue):** Elo rating of approximately 981. * **Ra-DPO (H) (Blue):** Elo rating of approximately 983. * **SFT (Gray):** Elo rating of approximately 1000. * **SACPO (P) [0.25] (Green):** Elo rating of approximately 1059. * **SACPO (H->S) [0.1] (Blue):** Elo rating of approximately 1115. * **RSA (H->S) [0.1] (Pink):** Elo rating of approximately 1123. * **RSA (P) [0.25] (Red):** Elo rating of approximately 1132. * **beaver-7b-v1.0 (Purple):** Elo rating of approximately 1148. * **SACPO (P) [0.99] (Green):** Elo rating of approximately 1158. * **SACPO (P) [0.95] (Green):** Elo rating of approximately 1179. * **beaver-7b-v2.0 (Purple):** Elo rating of approximately 1193. * **beaver-7b-v3.0 (Purple):** Elo rating of approximately 1236. * **RSA (P) [0.5] (Red):** Elo rating of approximately 1243. * **SACPO (P) [0.75] (Green):** Elo rating of approximately 1266. * **RSA (H->S) [0.025] (Pink):** Elo rating of approximately 1301. * **RSA (P) [0.95] (Red):** Elo rating of approximately 1315. * **SACPO (H->S) [0.025] (Blue):** Elo rating of approximately 1317. * **RSA (P) [0.90] (Red):** Elo rating of approximately 1331. * **RSA (P) [0.99] (Red):** Elo rating of approximately 1389. * **RSA (H+S) [0.05] (Pink):** Elo rating of approximately 1391. * **SACPO (P) [0.90] (Green):** Elo rating of approximately 1394. * **RSA (H->S) [0.01] (Pink):** Elo rating of approximately 1404. * **SACPO (H->S) [0.05] (Blue):** Elo rating of approximately 1430. * **SACPO (H->S) [0.01] (Blue):** Elo rating of approximately 1437. * **SACPO (H->S) [0.05] (Blue):** Elo rating of approximately 1443. * **SACPO (H->S) [0.01] (Blue):** Elo rating of approximately 1471. ### Key Observations * The Elo ratings vary significantly across different models and configurations for both Helpfulness and Harmlessness. * RSA models with parameter P (likely indicating a specific training or fine-tuning parameter) tend to have higher Elo ratings in both charts. * SACPO models also show variability depending on the parameters used. * The Harmlessness Evaluation generally shows higher Elo ratings compared to the Helpfulness Evaluation. ### Interpretation The charts provide a comparative analysis of language models based on their helpfulness and harmlessness, as measured by Elo ratings. The data suggests that certain models and configurations, particularly those involving RSA with specific parameters, perform better in both categories. The higher Elo ratings in the Harmlessness Evaluation indicate that models are generally more successful at being harmless than being helpful, according to the evaluation metrics used. The variability in Elo ratings highlights the importance of model selection and configuration for specific applications, depending on the desired balance between helpfulness and harmlessness. The data could be used to inform the development and deployment of language models, guiding decisions on training strategies and parameter settings to optimize performance in these key areas.