## Flowchart: Two-Phase Prompt Optimization Process
### Overview
The flowchart illustrates a two-phase process for optimizing prompts using large language models (LLMs). Phase 1 focuses on **Structure Discovery** to decompose problems into analytical components, while Phase 2 employs **Iterative Factor Optimization** to refine prompts through error correction and factor updates. The process concludes with a combined "Final Best Prompt" output.
---
### Components/Axes
#### Phase 1: Structure Discovery
- **Inputs**:
- Dataset (e.g., GSM8K examples)
- Initial Prompt: "Let's think step by step"
- **Core Component**:
- **Architect LLM** (blue robot icon)
- Outputs: Decomposed Prompt Factors
- **[F1 Analysis]**: "Begin by thoroughly understanding the problem"
- **[F2 Calculation]**: "Decompose the problem into individual steps"
- **[F3 Extraction]**: "Once all intermediate results are obtained"
- **Flow**: Dataset → Initial Prompt → Architect LLM → Decomposed Factors
#### Phase 2: Iterative Factor Optimization
- **Inputs**:
- Worker LLM (blue robot with magnifying glass)
- Output (Fail): "The answer is 15" (incorrect)
- **Evaluation**:
- Ground Truth: 17
- Diagnosis: "Reasoning Error"
- **Factor Selection & Update**:
- Retained Factors:
- **[F1 Analysis]**
- **[F3 Extraction]**
- Updated Factor:
- **[F2 Calculation]** → **[New F2 Calculation]**: "Split the problem into discrete steps"
- **Flow**: Worker LLM → Evaluation → Factor Update → Combine → Final Prompt
---
### Detailed Analysis
#### Phase 1: Structure Discovery
1. **Dataset**: Contains problem examples (e.g., GSM8K).
2. **Initial Prompt**: Generic instruction to "think step by step."
3. **Architect LLM**: Analyzes the problem and decomposes it into three factors:
- **F1 Analysis**: Problem comprehension.
- **F2 Calculation**: Stepwise decomposition.
- **F3 Extraction**: Aggregation of intermediate results.
#### Phase 2: Iterative Factor Optimization
1. **Worker LLM**: Executes the initial prompt but produces an incorrect output (15 vs. ground truth 17).
2. **Evaluation**: Identifies a "Reasoning Error" in the Worker LLM's output.
3. **Factor Update**:
- **F2 Calculation** is revised to emphasize "discrete steps" instead of "individual steps."
- Other factors (F1, F3) remain unchanged.
4. **Combination**: Updated factors are merged into the **Final Best Prompt**.
---
### Key Observations
1. **Iterative Refinement**: Phase 2 introduces a feedback loop to correct errors by updating specific factors.
2. **Factor Prioritization**: Only F2 Calculation is modified, suggesting it was the primary source of error.
3. **Modular Design**: Factors are treated as discrete, updatable components, enabling targeted improvements.
---
### Interpretation
This process demonstrates a systematic approach to prompt engineering:
- **Phase 1** establishes a foundational structure by breaking down problems into analyzable components.
- **Phase 2** uses error diagnosis to iteratively refine the most critical factor (F2), improving the LLM's ability to handle complex reasoning tasks.
- The final prompt integrates updated factors, likely enhancing accuracy by addressing stepwise decomposition flaws. The process mirrors scientific method principles: hypothesis (initial prompt), testing (Worker LLM output), error analysis, and hypothesis revision (factor updates).
