# Technical Document Extraction: GPT-4 Coherency Scores ## 1. Document Metadata * **Title:** (a) GPT-4 coherency scores * **Image Type:** Box and Whisker Plot * **Language:** English ## 2. Component Isolation ### Header * **Text:** "(a) GPT-4 coherency scores" * **Function:** Defines the subject of the data: a comparison of coherency performance across different prompting methods using the GPT-4 model. ### Main Chart Area (Data Visualization) The chart is a box plot comparing five distinct categories. The Y-axis represents a numerical score, and the X-axis represents the prompting method. A vertical dashed line separates the first three methods from the two "refined" methods. #### Axis Information * **Y-Axis (Vertical):** Numerical scale ranging from approximately 3 to 10. Major grid lines and labels are present at **4**, **6**, and **8**. * **X-Axis (Horizontal):** Categorical labels for prompting methods: 1. **IO** (Input-Output) 2. **CoT** (Chain of Thought) 3. **ToT** (Tree of Thoughts) 4. **IO +refine** 5. **ToT +refine** #### Legend/Color Coding While there is no explicit floating legend box, the colors distinguish the base methods: * **Blue:** Associated with "IO" and "IO +refine". * **Orange/Brown:** Associated with "CoT". * **Green:** Associated with "ToT" and "ToT +refine". ## 3. Data Extraction and Trend Analysis The following table estimates the values based on the spatial positioning relative to the Y-axis grid lines (4, 6, 8). | Method | Color | Median Score | Interquartile Range (Box) | Whiskers (Min/Max excluding outliers) | Outliers (Diamonds) | | :--- | :--- | :--- | :--- | :--- | :--- | | **IO** | Blue | ~6.3 | ~5.2 to ~7.4 | ~3.0 to ~8.6 | None visible | | **CoT** | Orange | ~7.2 | ~6.3 to ~7.8 | ~4.4 to ~8.6 | 1 at ~4.0 | | **ToT** | Green | ~7.8 | ~7.1 to ~8.4 | ~5.4 to ~9.2 | 4 between ~3.6 and ~4.8 | | **IO +refine** | Blue | ~8.0 | ~7.3 to ~8.4 | ~5.8 to ~9.5 | 6 between ~4.2 and ~5.7 | | **ToT +refine** | Green | ~8.0 | ~7.4 to ~8.4 | ~6.2 to ~9.2 | 4 between ~4.8 and ~5.8 | ### Trend Verification 1. **Performance Progression:** There is a clear upward trend in the median coherency score as the complexity of the prompting method increases (IO < CoT < ToT). 2. **Impact of Refinement:** The addition of a "+refine" step significantly raises the median score for the "IO" method (from ~6.3 to ~8.0). For "ToT", the median remains relatively stable at ~8.0, but the lower whisker moves up, suggesting a more consistent floor for performance. 3. **Variance and Outliers:** As the median scores increase, the number of low-end outliers also increases, particularly for the "IO +refine" and "ToT +refine" methods, indicating that while the average performance is higher, there are still specific instances of significantly lower coherency. ## 4. Structural Summary The diagram demonstrates that **Tree of Thoughts (ToT)** and **Refinement (+refine)** techniques yield higher coherency scores in GPT-4 compared to standard **Input-Output (IO)** or **Chain of Thought (CoT)** methods. The "IO +refine" method shows the most dramatic improvement over its base counterpart, reaching a median performance level comparable to "ToT +refine".

# Technical Document: GPT-4 Coherency Scores Analysis ## 1. Title and Overall Structure - **Title**: "GPT-4 coherency scores" - **Chart Type**: Box plot - **Purpose**: Visual comparison of coherency scores across different prompting strategies (IO, CoT, ToT) and their refined versions. --- ## 2. Axes and Labels ### X-Axis (Categories) - **Labels**: - IO (Input-Only) - CoT (Chain-of-Thought) - ToT (Tree-of-Thought) - IO + refine - ToT + refine - **Spatial Grounding**: - Categories are evenly spaced along the x-axis. - The first three categories (IO, CoT, ToT) are separated by a dashed vertical line from the refined versions (IO + refine, ToT + refine). ### Y-Axis (Values) - **Label**: "GPT-4 coherency scores" - **Range**: 4 to 9 (inclusive) - **Units**: Not explicitly stated, but implied as a numerical score. --- ## 3. Data Series and Colors ### Legend (Implied by Color Coding) - **Colors**: - **Blue**: IO (Input-Only) and IO + refine - **Orange**: CoT (Chain-of-Thought) - **Green**: ToT (Tree-of-Thought) and ToT + refine - **Note**: No explicit legend is present in the image. Color coding is inferred from the x-axis labels and data series. --- ## 4. Key Data Points and Trends ### Box Plot Components - **Median**: Represented by the horizontal line inside each box. - **Interquartile Range (IQR)**: The height of the box (25th to 75th percentile). - **Outliers**: Diamond-shaped markers outside the whiskers. ### Trends by Category 1. **IO (Blue)**: - **Median**: ~6.5 - **Range**: ~4 to 8.5 - **Outliers**: 1–2 points below 4 and above 8.5. 2. **CoT (Orange)**: - **Median**: ~7 - **Range**: ~5 to 8.5 - **Outliers**: 1–2 points below 5 and above 8.5. 3. **ToT (Green)**: - **Median**: ~7.5 - **Range**: ~5.5 to 9 - **Outliers**: 1–2 points below 5.5 and above 9. 4. **IO + refine (Blue)**: - **Median**: ~7 - **Range**: ~5.5 to 8.5 - **Outliers**: 1–2 points below 5.5 and above 8.5. 5. **ToT + refine (Green)**: - **Median**: ~7.5 - **Range**: ~5.5 to 9 - **Outliers**: 1–2 points below 5.5 and above 9. ### Observations - **Refinement Impact**: - **IO + refine** shows a **1.0-point increase** in median compared to IO (6.5 → 7). - **ToT + refine** maintains the same median as ToT (7.5) but with a slightly narrower range. - **Consistency**: - All refined categories (IO + refine, ToT + refine) exhibit **higher medians** than their non-refined counterparts. - **ToT + refine** has the **highest median** (7.5) and **widest range** (5.5–9). --- ## 5. Spatial Grounding and Color Verification - **Color Consistency**: - **Blue** corresponds to **IO** and **IO + refine**. - **Orange** corresponds to **CoT**. - **Green** corresponds to **ToT** and **ToT + refine**. - **Outlier Markers**: Diamond-shaped symbols are consistently used across all categories. --- ## 6. Component Isolation ### Header - **Title**: "GPT-4 coherency scores" (top of the chart). ### Main Chart - **X-Axis**: Categories (IO, CoT, ToT, IO + refine, ToT + refine). - **Y-Axis**: Coherency scores (4–9). - **Data Series**: Five box plots with distinct colors and outliers. ### Footer - **No explicit footer text** in the image. --- ## 7. Additional Notes - **No Explicit Legend**: Color coding is inferred from the x-axis labels and data series. - **Outliers**: Present in all categories, with varying frequencies. - **Refinement Strategy**: The "+ refine" suffix indicates an improved prompting method, which generally enhances coherency scores. --- ## 8. Conclusion The chart demonstrates that **refinement strategies** (IO + refine, ToT + refine) improve coherency scores compared to their non-refined counterparts. **ToT + refine** achieves the highest median score (7.5), while **IO + refine** shows the most significant improvement (from 6.5 to 7). Outliers suggest variability in performance across different instances.