## Heatmap Grid: Comparison of Hard Logic, NLU, and MLP Decision Boundaries ### Overview The image presents a 2x5 grid of heatmaps comparing decision boundaries for logical operations (OR, AND) across three model types: Hard Logic, NLU (Neural Logic Unit), and MLP (Dense + ReLU). Each heatmap visualizes decision intensity (0.000-1.000) across input space (x₁, x₂ ∈ [0,1]). The rightmost color bar serves as a shared legend for all heatmaps. ### Components/Axes - **Axes**: - Horizontal: x₁ (0.0 to 1.0) - Vertical: x₂ (0.0 to 1.0) - **Legends**: - Right-aligned color bar labeled "Decision Intensity" (0.000-1.000) - Color gradient: Purple (low) → Yellow (high) - **Heatmap Titles**: - Top row: "Hard Logic OR" (left), "Hard Logic AND" (right) - Middle rows: NLU Soft-OR/Soft-AND with β=1, β=10, β=100 - Bottom row: Dense+ReLU with Bias=0.0 and Bias=-0.5 ### Detailed Analysis 1. **Hard Logic OR (Top Left)**: - Sharp boundary at x₁=0.5, x₂=0.5 - High intensity (yellow) in top-right quadrant (x₁≥0.5 OR x₂≥0.5) - Low intensity (purple) in bottom-left quadrant 2. **Hard Logic AND (Top Right)**: - Sharp corner boundary at x₁=0.5, x₂=0.5 - High intensity only in top-right quadrant (x₁≥0.5 AND x₂≥0.5) 3. **NLU Soft-OR/Soft-AND (β=1 to 100)**: - **β=1**: - Gradual transition from purple to yellow - OR: Diagonal boundary at 45° - AND: Curved boundary forming a quarter-circle - **β=10**: - Sharper boundaries approaching hard logic - OR: Near-vertical/horizontal steps - AND: Near-corner steps - **β=100**: - Almost identical to hard logic boundaries - Minimal gradient regions 4. **Dense+ReLU (Bias Variations)**: - **Bias=0.0**: - Diagonal boundary from bottom-left to top-right - High intensity above the diagonal - **Bias=-0.5**: - Steeper diagonal boundary - High intensity concentrated in top-right corner ### Key Observations 1. **NLU Sensitivity to β**: - Lower β (1): Smooth, continuous transitions - Higher β (100): Approaches hard logic behavior - AND operation shows more complex boundary evolution than OR 2. **Bias Impact in Dense+ReLU**: - Negative bias (-0.5) shifts decision boundary upward - Creates larger high-intensity region in top-right quadrant 3. **Model Comparison**: - Hard Logic: Perfect binary boundaries - NLU: Intermediate between hard logic and MLP - MLP: Linear/non-linear boundaries depending on bias ### Interpretation The heatmaps demonstrate how different architectures handle logical operations: - **NLU** provides a tunable middle ground between hard logic and MLP through β parameterization - **Dense+ReLU** shows linear decision boundaries that can be adjusted via bias - The AND operation requires more complex boundary shapes than OR across all models - Higher β values in NLU make it increasingly similar to hard logic, suggesting potential for exact logical computation with sufficient parameterization The comparison highlights tradeoffs between model complexity and interpretability, with hard logic offering perfect boundaries at the cost of flexibility, while NLU and MLP provide adjustable approximations.