## Density Contour Plots: Comparing Text Embeddings ### Overview The image presents three sets of density contour plots, each comparing the distribution of "General Text" and "Medical Text" embeddings in a two-dimensional space. The plots are arranged horizontally, labeled "Vanilla," "W/o stage1," and "ADEPT." Each plot includes a 2D contour plot, a density plot above it representing the distribution along the horizontal axis (dim 1), and a density plot to the right representing the distribution along the vertical axis (dim 2). The plots aim to visualize how different embedding methods separate general and medical text. ### Components/Axes * **Titles:** "Vanilla," "W/o stage1," "ADEPT" (placed above each respective plot). * **Legends:** Each plot has a legend in the top-left corner indicating: * "General Text" (represented by the color blue) * "Medical Text" (represented by the color red) * **Axes Labels:** * Horizontal axis (dim 1): Ranges from approximately -75 to 75 for "Vanilla", -100 to 100 for "W/o stage1", and -100 to 100 for "ADEPT". * Vertical axis (dim 2): Ranges from approximately -75 to 100 for "Vanilla", -60 to 60 for "W/o stage1", and -60 to 60 for "ADEPT". * **Axis Markers:** * Vanilla: dim 1: -50, 0, 50. dim 2: -75, -50, -25, 0, 25, 50, 75, 100 * W/o stage1: dim 1: -100, -50, 0, 50, 100. dim 2: -60, -40, -20, 0, 20, 40, 60 * ADEPT: dim 1: -100, -50, 0, 50. dim 2: -60, -40, -20, 0, 20, 40, 60 ### Detailed Analysis **1. Vanilla:** * **2D Contour Plot:** * "General Text" (blue): The contours form a cluster centered around dim 1 = -25 and dim 2 = 25. The distribution is somewhat elongated along a diagonal. * "Medical Text" (red): The contours form a cluster centered around dim 1 = 25 and dim 2 = -10. The distribution is more spread out compared to "General Text." * **Horizontal Density Plot (dim 1):** * "General Text" (blue): Shows a single peak around -25. * "Medical Text" (red): Shows a single peak around 25. * **Vertical Density Plot (dim 2):** * "General Text" (blue): Shows a single peak around 25. * "Medical Text" (red): Shows a single peak around -10. **2. W/o stage1:** * **2D Contour Plot:** * "General Text" (blue): The contours are more dispersed, with a primary cluster around dim 1 = -25 and dim 2 = 10, and a secondary cluster around dim 1 = 25 and dim 2 = -20. * "Medical Text" (red): The contours are also dispersed, with a primary cluster around dim 1 = 25 and dim 2 = 10, and a secondary cluster around dim 1 = -25 and dim 2 = -20. There is significant overlap between the two distributions. * **Horizontal Density Plot (dim 1):** * "General Text" (blue): Shows two peaks, one around -25 and another around 25. * "Medical Text" (red): Shows two peaks, one around -25 and another around 25. * **Vertical Density Plot (dim 2):** * "General Text" (blue): Shows two peaks, one around 10 and another around -20. * "Medical Text" (red): Shows two peaks, one around 10 and another around -20. **3. ADEPT:** * **2D Contour Plot:** * "General Text" (blue): The contours form a cluster centered around dim 1 = -40 and dim 2 = 20. * "Medical Text" (red): The contours form a cluster centered around dim 1 = 40 and dim 2 = -20. The separation between the two distributions is more pronounced than in the other two plots. * **Horizontal Density Plot (dim 1):** * "General Text" (blue): Shows a single peak around -40. * "Medical Text" (red): Shows a single peak around 40. * **Vertical Density Plot (dim 2):** * "General Text" (blue): Shows a single peak around 20. * "Medical Text" (red): Shows a single peak around -20. ### Key Observations * The "Vanilla" plot shows some separation between "General Text" and "Medical Text," but there is still some overlap. * The "W/o stage1" plot shows significant overlap between the two distributions, indicating poor separation. * The "ADEPT" plot shows the best separation between "General Text" and "Medical Text," with distinct clusters for each. ### Interpretation The plots visualize the effectiveness of different embedding methods in distinguishing between general and medical text. The "ADEPT" method appears to be the most effective, as it produces the most distinct clusters for the two types of text. Removing "stage1" results in the worst separation. The "Vanilla" method provides an intermediate level of separation. This suggests that the "ADEPT" method is better at capturing the semantic differences between general and medical text, which could be beneficial for various natural language processing tasks in the medical domain. The overlap in "W/o stage1" suggests that stage 1 is critical for separating the two types of text.

\n ## Scatter Plot: Dimensionality Reduction Comparison ### Overview The image presents three scatter plots, each representing a dimensionality reduction of text data. The plots compare the distribution of "General Text" and "Medical Text" across two dimensions (dim 1 and dim 2) for three different models: "Vanilla", "W/o stage1", and "ADEPT". Each plot also includes a density plot above the scatter, showing the marginal distribution of each text type along a combined dimension. ### Components/Axes Each plot shares the following components: * **Title:** Indicates the model being used ("Vanilla", "W/o stage1", "ADEPT"). Positioned at the top-center. * **X-axis:** Labeled "dim 1", ranging from approximately -100 to 100 (scales vary slightly between plots). * **Y-axis:** Labeled "dim 2", ranging from approximately -75 to 100 (scales vary slightly between plots). * **Legend:** Located in the top-right corner, distinguishing between "General Text" (blue) and "Medical Text" (red). * **Density Plot:** Positioned above the scatter plot, showing the probability density of each text type. The x-axis of the density plot corresponds to a combined dimension. ### Detailed Analysis or Content Details **Plot 1: Vanilla** * **General Text (Blue):** The blue contour lines form a roughly elliptical shape, centered around (approximately -30, 20). The density plot shows a single peak around -30. * **Medical Text (Red):** The red contour lines also form an elliptical shape, but are more elongated and centered around (approximately 0, -20). The density plot shows a single peak around 0. * The two distributions are somewhat overlapping, but distinguishable. **Plot 2: W/o stage1** * **General Text (Blue):** The blue contour lines form a more complex, elongated shape, centered around (approximately -20, 0). The density plot shows a peak around -20 and a smaller peak around 50. * **Medical Text (Red):** The red contour lines form a more diffuse, elongated shape, centered around (approximately 50, -20). The density plot shows a peak around 50. * The distributions are more separated than in the "Vanilla" plot. **Plot 3: ADEPT** * **General Text (Blue):** The blue contour lines form a relatively compact, elliptical shape, centered around (approximately -20, 20). The density plot shows a single peak around -20. * **Medical Text (Red):** The red contour lines form a more elongated shape, centered around (approximately 0, -30). The density plot shows a single peak around 0. * The distributions are well-separated, with minimal overlap. ### Key Observations * The "Vanilla" model shows the most overlap between the distributions of "General Text" and "Medical Text". * Removing "stage1" ("W/o stage1") increases the separation between the distributions, but also introduces a secondary peak in the density plot for "General Text". * The "ADEPT" model achieves the best separation between the distributions, with minimal overlap and clear single peaks in the density plots. * The shape and orientation of the contour lines vary significantly between the models, indicating different dimensionality reduction characteristics. ### Interpretation The plots demonstrate the effectiveness of different models in separating "General Text" from "Medical Text" in a reduced dimensional space. The "Vanilla" model struggles to distinguish between the two text types, as evidenced by the significant overlap in their distributions. Removing "stage1" improves separation but introduces complexity. The "ADEPT" model appears to be the most successful, achieving clear separation and a simpler distribution for both text types. This suggests that the "ADEPT" model learns a more effective representation for distinguishing between general and medical language. The density plots provide additional insight into the marginal distributions, confirming the improved separation achieved by the "ADEPT" model. The varying shapes of the contour lines indicate that each model employs a different strategy for dimensionality reduction, resulting in different representations of the text data.

\n ## Comparative Contour Plot with Marginal Distributions: Vanilla, W/o stage1, ADEPT ### Overview The image displays three side-by-side plots comparing the 2D embedding distributions of "General Text" and "Medical Text" as produced by three different models or model variants: "Vanilla", "W/o stage1" (Without stage 1), and "ADEPT". Each plot is a contour plot with marginal density distributions shown on the top and right axes. ### Components/Axes * **Plot Titles (Top Center):** "Vanilla", "W/o stage1", "ADEPT". * **Axes Labels:** * X-axis (Bottom): "dim 1" for all three plots. * Y-axis (Left): "dim 2" for all three plots. * **Axis Scales:** * **Vanilla:** dim 1 ranges approximately from -75 to 75. dim 2 ranges approximately from -75 to 100. * **W/o stage1:** dim 1 ranges approximately from -100 to 100. dim 2 ranges approximately from -60 to 60. * **ADEPT:** dim 1 ranges approximately from -100 to 100. dim 2 ranges approximately from -60 to 60. * **Legend (Top-right corner of each main plot area):** * A blue line labeled "General Text". * A red line labeled "Medical Text". * **Plot Elements:** * **Main Area:** 2D contour plots showing the density of data points for each text type. Blue contours represent "General Text", red contours represent "Medical Text". * **Marginal Plots:** Density curves (histograms/KDEs) along the top (for dim 1 distribution) and right side (for dim 2 distribution) of each main plot, colored blue and red to match their respective text type. ### Detailed Analysis **1. Vanilla Plot (Left):** * **Contour Analysis:** The blue ("General Text") and red ("Medical Text") contour clouds show significant overlap, particularly in the central region around dim1=0, dim2=0. The "General Text" distribution appears more spread out, especially along the positive dim2 axis. The "Medical Text" distribution is more concentrated but still overlaps substantially with the blue contours. * **Marginal Distributions (Top - dim1):** Both the blue and red density curves are broad and overlapping, with multiple peaks. The red curve ("Medical Text") has a slightly more pronounced peak near dim1=0. * **Marginal Distributions (Right - dim2):** The blue curve is very broad, spanning from approximately -75 to 100. The red curve is narrower, centered around dim2=0, but still overlaps with the lower portion of the blue curve. **2. W/o stage1 Plot (Center):** * **Contour Analysis:** The separation between the two distributions is more pronounced than in the "Vanilla" plot. The blue ("General Text") contours form a cluster primarily in the lower-left quadrant (negative dim1, negative dim2). The red ("Medical Text") contours form a cluster primarily in the upper-right quadrant (positive dim1, positive dim2). There is a region of overlap in the center. * **Marginal Distributions (Top - dim1):** The blue and red curves are now clearly bimodal and separated. The blue curve peaks in the negative dim1 region, while the red curve peaks in the positive dim1 region. * **Marginal Distributions (Right - dim2):** Similar separation is visible. The blue curve peaks in the negative dim2 region, and the red curve peaks in the positive dim2 region. **3. ADEPT Plot (Right):** * **Contour Analysis:** This plot shows the clearest separation between the two text types. The blue ("General Text") contours are densely clustered in a distinct region, primarily in the lower-left to central area. The red ("Medical Text") contours form a separate, dense cluster in the upper-right area. The overlap between the two is minimal compared to the other plots. * **Marginal Distributions (Top - dim1):** The separation is very clear. The blue curve has a sharp peak in the negative dim1 region (around -50). The red curve has a sharp peak in the positive dim1 region (around +25). * **Marginal Distributions (Right - dim2):** The separation is also very clear here. The blue curve peaks in the negative dim2 region (around -20). The red curve peaks in the positive dim2 region (around +20). ### Key Observations 1. **Progressive Separation:** There is a clear visual progression from high overlap ("Vanilla") to moderate separation ("W/o stage1") to strong separation ("ADEPT") between the embedding distributions of General Text and Medical Text. 2. **Cluster Tightening:** As the models progress from Vanilla to ADEPT, the contour clusters for each text type become tighter and more defined, indicating less variance within each category in the learned embedding space. 3. **Marginal Distribution Clarity:** The marginal density plots provide a clear, 1D confirmation of the separation observed in the 2D contours. The peaks for General and Medical text become sharper and move further apart along both dimensions in the ADEPT model. 4. **Spatial Grounding:** In all plots, the legend is consistently placed in the top-right corner of the main plot area. The color coding (blue=General, red=Medical) is consistent across all elements (contours, marginal lines, legend). ### Interpretation This visualization demonstrates the effectiveness of the "ADEPT" model (and to a lesser extent, the "W/o stage1" variant) in learning a disentangled representation space for text from different domains. The "Vanilla" model fails to distinguish between general and medical text, mapping them to largely overlapping regions. This could hinder performance on domain-specific tasks. The progression suggests that the architectural or training modifications in "W/o stage1" and especially "ADEPT" successfully encourage the model to encode domain-specific semantic features into distinct dimensions of the embedding space. The clear separation in the ADEPT plot implies that a classifier or downstream task could more easily distinguish between or specialize for these text types using its embeddings. This is a desirable property for building specialized NLP models for fields like medicine, where domain-specific language understanding is critical. The tight clustering also suggests more consistent and confident representations within each domain.

## 2D Contour Plots: Text Embedding Distributions Across Methods ### Overview The image presents three comparative 2D contour plots visualizing the latent space distributions of General Text (blue) and Medical Text (red) across three different model configurations: Vanilla, W/o stage1, and ADEPT. Each subplot includes marginal histograms showing 1D projections along dim1 (top) and dim2 (right). ### Components/Axes - **X-axis (dim1)**: Ranges from -100 to 100 across subplots - **Y-axis (dim2)**: Ranges from -75 to 75 (Vanilla), -60 to 60 (W/o stage1), and -60 to 60 (ADEPT) - **Legend**: - Blue = General Text - Red = Medical Text - **Marginal Histograms**: - Top: dim1 distribution - Right: dim2 distribution - **Contour Lines**: - Blue (General Text) and Red (Medical Text) distributions - Density indicated by line thickness ### Detailed Analysis #### Vanilla - **Contour Spread**: - General Text (blue): -50 to 50 (dim1), -75 to 25 (dim2) - Medical Text (red): -30 to 30 (dim1), -50 to 0 (dim2) - **Overlap**: Significant overlap in central region (-10 to 10 dim1, -25 to 0 dim2) - **Histograms**: - dim1: Bimodal peaks at ±30 - dim2: Single peak at 0 #### W/o stage1 - **Contour Spread**: - General Text (blue): -100 to 100 (dim1), -40 to 40 (dim2) - Medical Text (red): -60 to 60 (dim1), -30 to 30 (dim2) - **Overlap**: Moderate overlap in central region (-20 to 20 dim1, -10 to 10 dim2) - **Histograms**: - dim1: Single peak at 0 - dim2: Bimodal peaks at ±20 #### ADEPT - **Contour Spread**: - General Text (blue): -80 to 80 (dim1), -40 to 40 (dim2) - Medical Text (red): -50 to 50 (dim1), -30 to 30 (dim2) - **Overlap**: Minimal overlap in central region (-10 to 10 dim1, -10 to 10 dim2) - **Histograms**: - dim1: Single peak at 0 - dim2: Single peak at 0 ### Key Observations 1. **ADEPT** shows the clearest separation between text types, with Medical Text (red) forming a tighter cluster 2. **Vanilla** exhibits the most overlap, with Medical Text distribution heavily concentrated in negative dim2 3. **W/o stage1** demonstrates intermediate separation with broader dim1 spread 4. All methods show Medical Text with stronger dim2 concentration than General Text 5. ADEPT's dim1 range is 60% narrower than Vanilla while maintaining similar dim2 spread ### Interpretation The progressive improvement in text type separation from Vanilla → W/o stage1 → ADEPT suggests that the stage1 component plays a critical role in disentangling text representations. ADEPT's tighter clustering of Medical Text (red) indicates better feature discrimination, potentially improving downstream task performance. The reduced dim1 spread in ADEPT compared to Vanilla suggests more efficient use of latent dimensions. The consistent dim2 concentration across methods implies this dimension primarily captures text-type specific features. The marginal histograms reveal that ADEPT achieves more balanced distributions across both dimensions compared to other configurations.