## Box Plot: PPL Score Comparison of Watermarked vs. Unwatermarked Data ### Overview The image is a box plot comparing the PPL (Perplexity) scores of watermarked and unwatermarked data across three different methods: SynthID, SIR, and SynGuard. The plot visualizes the distribution of PPL scores for each method and condition (watermarked/unwatermarked), highlighting the median, quartiles, and outliers. ### Components/Axes * **Title:** Implicitly, the plot compares PPL scores across different methods and watermark status. * **X-axis:** Categorical axis representing the methods: SynthID, SIR, and SynGuard. * **Y-axis:** Numerical axis labeled "PPL Score," ranging from 0 to 30, with tick marks at intervals of 5. * **Legend:** Located in the top-left corner, indicating the color-coding for "watermarked" (blue) and "unwatermarked" (orange). * **Box Plot Elements:** Each box plot displays the median (line within the box), the first and third quartiles (edges of the box), and the whiskers extending to the most extreme data points within 1.5 times the interquartile range. Outliers are represented as individual circles. ### Detailed Analysis **1. SynthID:** * **Watermarked (Blue):** The box extends from approximately 5.5 to 7.5, with a median around 6.5. The lower whisker extends to approximately 3, and the upper whisker extends to approximately 10. There are two outliers at approximately 12 and 20.5. * **Unwatermarked (Orange):** The box extends from approximately 8 to 13.5, with a median around 10. The lower whisker extends to approximately 3, and the upper whisker extends to approximately 20.5. There are several outliers, including one at approximately 24 and one at approximately 28. **2. SIR:** * **Watermarked (Blue):** The box extends from approximately 10.5 to 14.5, with a median around 13. The lower whisker extends to approximately 6, and the upper whisker extends to approximately 20. There are several outliers, including one at approximately 24 and one at approximately 28. * **Unwatermarked (Orange):** The box extends from approximately 8 to 12, with a median around 10. The lower whisker extends to approximately 3, and the upper whisker extends to approximately 20.5. There are several outliers, including one at approximately 24 and one at approximately 28. **3. SynGuard:** * **Watermarked (Blue):** The box extends from approximately 7 to 9, with a median around 8. The lower whisker extends to approximately 4, and the upper whisker extends to approximately 13.5. There is one outlier at approximately 13.5. * **Unwatermarked (Orange):** The box extends from approximately 8 to 13.5, with a median around 10. The lower whisker extends to approximately 3, and the upper whisker extends to approximately 20.5. There are several outliers, including one at approximately 24 and one at approximately 28. ### Key Observations * For SynthID, the watermarked data has a noticeably lower PPL score distribution compared to the unwatermarked data. * For SIR, the watermarked data has a slightly higher PPL score distribution compared to the unwatermarked data. * For SynGuard, the watermarked data has a noticeably lower PPL score distribution compared to the unwatermarked data. * All three methods exhibit outliers in both watermarked and unwatermarked conditions, indicating some data points with significantly higher PPL scores. * The range of PPL scores is generally higher for unwatermarked data across all three methods. ### Interpretation The box plot suggests that the presence of a watermark can influence the PPL score, and the direction and magnitude of this influence vary depending on the method used. SynthID and SynGuard appear to result in lower PPL scores for watermarked data, potentially indicating that the watermark improves the model's perplexity or predictability. Conversely, SIR shows a slight increase in PPL score for watermarked data. The presence of outliers suggests that there are instances where the watermark has a more pronounced effect on the PPL score. The higher range of PPL scores for unwatermarked data might indicate greater variability or uncertainty in the model's predictions without the watermark. Further investigation would be needed to understand the specific mechanisms by which each method interacts with the watermark and affects the PPL score.

\n ## Box Plot: PPL Score vs. Watermarking Type & Model ### Overview The image presents a series of box plots comparing the Perplexity (PPL) scores for watermarked and unwatermarked text generated by three different models: SynthID, SIR, and SynGuard. The PPL score is a measure of how well a probability model predicts a sample. Lower PPL scores generally indicate better performance. ### Components/Axes * **X-axis:** Model Name (SynthID, SIR, SynGuard) * **Y-axis:** PPL Score (Scale from approximately 4 to 30) * **Legend:** * **Type:** watermarked (represented by blue color) * **Type:** unwatermarked (represented by orange color) * **Plot Type:** Box plots with overlaid individual data points (circles). * **Gridlines:** Horizontal gridlines are present to aid in reading values. ### Detailed Analysis The image contains three sets of box plots, one for each model. Each set contains two box plots, one for watermarked data and one for unwatermarked data. **SynthID:** * **Watermarked (Blue):** The box plot spans approximately from 6 to 12. The median is around 8. There are several outliers above 20. * **Unwatermarked (Orange):** The box plot spans approximately from 5 to 10. The median is around 7. There are outliers around 20 and 24. **SIR:** * **Watermarked (Blue):** The box plot spans approximately from 10 to 20. The median is around 14. There are several outliers, ranging from approximately 22 to 28. * **Unwatermarked (Orange):** The box plot spans approximately from 8 to 18. The median is around 12. There are outliers ranging from approximately 20 to 26. **SynGuard:** * **Watermarked (Blue):** The box plot spans approximately from 8 to 16. The median is around 11. There are outliers around 18 and 24. * **Unwatermarked (Orange):** The box plot spans approximately from 8 to 16. The median is around 11. There are outliers around 22 and 26. ### Key Observations * **SIR consistently has higher PPL scores than SynthID and SynGuard**, suggesting it performs worse in terms of predicting the text. * **Watermarking generally increases the PPL score** for all three models, indicating that the watermarking process introduces some degradation in the model's predictive ability. * **Outliers are present in all datasets**, suggesting some generated texts are significantly different from the majority. * **The spread of the data (as indicated by the box plot size) varies between models and watermarking types.** SIR has the largest spread, indicating the most variability in PPL scores. ### Interpretation The data suggests that watermarking introduces a trade-off between security/traceability and model performance. While watermarking allows for the identification of generated text, it appears to slightly reduce the quality of the generated text as measured by PPL. The model SIR appears to be more sensitive to the watermarking process, exhibiting a larger increase in PPL score compared to SynthID and SynGuard. The presence of outliers in all datasets suggests that the watermarking process may not be uniformly effective, and some generated texts may be more easily detectable than others. The differences in PPL scores between models suggest that they have varying levels of inherent predictive power. The fact that watermarking consistently increases PPL scores across all models indicates that the watermarking process introduces some level of noise or distortion to the generated text, which affects the model's ability to accurately predict the next token. Further investigation could explore the specific watermarking techniques used and their impact on the generated text's characteristics.

## Box Plot: PPL Score Comparison Across Methods ### Overview The image presents a comparative box plot analysis of PPL (Perplexity) scores for three text generation methods: SynthID, SIR, and SynGuard. The data is categorized by two types: watermarked (blue) and unwatermarked (orange). The y-axis represents PPL scores (0–30), while the x-axis lists the three methods. Outliers are marked as individual points beyond the whiskers. ### Components/Axes - **X-axis (Methods)**: - SynthID (leftmost) - SIR (middle) - SynGuard (rightmost) - **Y-axis (PPL Score)**: - Range: 0 to 30 (discrete increments of 5) - Labels: "PPL Score" with numerical ticks at 0, 5, 10, 15, 20, 25, 30 - **Legend**: - Top-left corner - Blue = watermarked - Orange = unwatermarked - **Outliers**: - Represented as open circles beyond whiskers ### Detailed Analysis #### SynthID - **Watermarked (blue)**: - Median: ~6 - IQR: 5–7 - Outliers: 10, 11 - **Unwatermarked (orange)**: - Median: ~10 - IQR: 8–12 - Outliers: 13, 14 #### SIR - **Watermarked (blue)**: - Median: ~12 - IQR: 10–14 - Outliers: 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25 - **Unwatermarked (orange)**: - Median: ~10 - IQR: 8–12 - Outliers: 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25 #### SynGuard - **Watermarked (blue)**: - Median: ~8 - IQR: 6–10 - Outliers: 11, 12 - **Unwatermarked (orange)**: - Median: ~12 - IQR: 10–14 - Outliers: 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25 ### Key Observations 1. **Watermarked vs. Unwatermarked**: - Watermarked methods consistently show lower median PPL scores (better performance) across all three methods. - SIR’s watermarked median (~12) is higher than SynthID’s (~6) and SynGuard’s (~8), suggesting SIR’s watermarked outputs are less optimal. - Unwatermarked scores are higher (worse performance) for all methods, with SynGuard’s unwatermarked median (~12) being the highest. 2. **Outliers**: - SIR’s watermarked data has the most outliers (15–25), indicating significant variability or anomalies. - SynGuard’s unwatermarked data also has multiple outliers (15–25), suggesting instability in unwatermarked outputs. 3. **Distribution**: - SynthID’s watermarked data is tightly clustered (IQR: 5–7), while its unwatermarked data is more spread out (IQR: 8–12). - SIR’s watermarked data has a wider IQR (10–14) compared to its unwatermarked counterpart (8–12). ### Interpretation The data suggests that **watermarking improves PPL scores** (i.e., reduces perplexity) across all methods, with SynthID showing the most consistent performance for watermarked outputs. SIR’s watermarked data, while having a higher median than SynthID and SynGuard, exhibits extreme variability (outliers up to 25), which may indicate instability or edge cases in its watermarked outputs. SynGuard’s unwatermarked data has the highest median (~12), suggesting it performs worst among unwatermarked methods. The presence of outliers in SIR and SynGuard’s data highlights potential inconsistencies in their respective methods. This analysis underscores the importance of watermarking for optimizing text generation quality, with SynthID emerging as the most reliable method for watermarked outputs.