## Diagram: Sequential vs. Parallel Scaling
### Overview
The image presents a diagram comparing sequential and parallel scaling approaches, likely in the context of reasoning processes and large language models (RLLMs). It uses visual metaphors to illustrate the limitations and advantages of each method.
### Components/Axes
* **Titles:**
* (a) Sequential Scaling (top-left)
* (b) Parallel Scaling (top-right)
* **Processes:** Represented by yellow rounded rectangles labeled "Reasoning Process." Each process has an input represented by three vertical bars, colored blue, brown, and red.
* **RLLM Supply:** Represented by a green rounded rectangle with a battery icon.
* **Reasoning Boundary:** A dashed line that encloses the sequential scaling process. The boundary is labeled "Limited by Reasoning Boundary" in red text.
* **Sequential Scaling Block:** A yellow rectangle labeled "Sequential Scaling" with an arrow indicating the flow of the process.
* **Self-Consistency, Self-Verification, Pass@k:** Represented by rounded rectangles in the parallel scaling diagram.
* **Resistors:** Represented by small rectangles with values 100Ω (orange), 20Ω (blue), and 0Ω (green).
* **Sampling:** Represented by three cartoon figures with the text "Sampling*1" below each.
* **(1) Sampling, (2) Verification:** Labels indicating the steps in the parallel scaling process.
* **Output:** A green rounded rectangle with the text "Output."
### Detailed Analysis
**Sequential Scaling (Left Side)**
1. **Reasoning Processes:** There are three yellow "Reasoning Process" blocks stacked vertically. The bottom block is colored red and has an ellipsis (...) indicating that there may be more processes.
2. **Flow:** The output of each "Reasoning Process" feeds into the next, creating a sequential chain.
3. **RLLM Supply:** The "RLLM Supply" block provides input to the first "Reasoning Process."
4. **Reasoning Boundary:** The entire sequential process is enclosed within a "Reasoning Boundary," suggesting a limitation or constraint. A pair of scissors is shown cutting the top "Reasoning Process" block.
5. **Sequential Scaling Block:** The output of the sequential chain feeds into a "Sequential Scaling" block.
6. **Visual Metaphor:** A cartoon figure is shown "mining" the "Sequential Scaling" block, suggesting that the process extracts value or information. Another cartoon figure is shown with an "X" over its eyes, suggesting failure.
**Parallel Scaling (Right Side)**
1. **Reasoning Processes:** There are four yellow "Reasoning Process" blocks stacked vertically.
2. **Flow:** The "RLLM Supply" feeds into all four "Reasoning Process" blocks simultaneously.
3. **Self-Consistency, Self-Verification, Pass@k:** The outputs of the "Reasoning Process" blocks feed into "Self-Consistency," "Self-Verification," and "Pass@k" blocks, respectively.
4. **Resistors:** The "Self-Consistency," "Self-Verification," and "Pass@k" blocks are associated with resistor values of 100Ω, 20Ω, and 0Ω, respectively.
5. **Sampling:** The outputs of the "Self-Consistency," "Self-Verification," and "Pass@k" blocks feed into three "Sampling" figures.
6. **(2) Verification:** The outputs of the "Sampling" figures feed into a "(2) Verification" block.
7. **Output:** The output of the "(2) Verification" block leads to the "Output" block.
### Key Observations
* The sequential scaling approach is presented as being limited by a "Reasoning Boundary," while the parallel scaling approach is not.
* The parallel scaling approach incorporates "Self-Consistency," "Self-Verification," and "Pass@k" checks, suggesting a more robust and reliable process.
* The resistor values associated with the "Self-Consistency," "Self-Verification," and "Pass@k" blocks may represent the level of difficulty or cost associated with each check.
### Interpretation
The diagram illustrates the trade-offs between sequential and parallel scaling approaches for reasoning processes, likely within the context of large language models. Sequential scaling, while perhaps simpler to implement, is presented as being limited by a "Reasoning Boundary," suggesting a bottleneck or constraint on its scalability. The scissors cutting the "Reasoning Process" block further emphasize this limitation. The cartoon figure with an "X" over its eyes suggests that the sequential scaling process may be prone to failure.
Parallel scaling, on the other hand, is presented as a more robust and scalable approach. By distributing the reasoning process across multiple parallel paths and incorporating "Self-Consistency," "Self-Verification," and "Pass@k" checks, the parallel scaling approach aims to improve the reliability and accuracy of the results. The resistor values associated with these checks may represent the computational cost or complexity of each check. The "Sampling" figures suggest that the parallel scaling approach involves sampling from the outputs of the parallel paths.
