## Diagram: Chain of Thought with Latent Tokens
### Overview
The image illustrates a Chain of Thought (CoT) process, comparing a standard CoT approach with a modified approach incorporating latent tokens. The diagram highlights how latent tokens are introduced within the CoT sequence.
### Components/Axes
* **X**: Represents the standard Chain of Thought process.
* **Prompt**: Green box labeled "Prompt".
* **CoT 1, CoT 2..., CoT 32, CoT 33, ..., CoT N**: Blue boxes representing Chain of Thought textual tokens.
* **Solution**: Red box labeled "Solution".
* **X̃**: Represents the modified Chain of Thought process with latent tokens.
* **Prompt**: Green box labeled "Prompt".
* **[boLatent]**: Orange box representing the beginning of latent tokens.
* **z1, z2**: Purple boxes representing discrete latent tokens.
* **[eoLatent]**: Orange box representing the end of latent tokens.
* **CoT 33, ..., CoT N**: Blue boxes representing Chain of Thought textual tokens.
* **Solution**: Red box labeled "Solution".
* **Legend**: Located at the bottom of the image.
* **[boLatent], [eoLatent]**: Orange box. "Special delimiters that encode the start / end of the latent tokens".
* **z**: Purple box. "Discrete latent tokens".
* **CoT N**: Blue box. "The n-th CoT textual tokens".
### Detailed Analysis or ### Content Details
* **Standard CoT (X)**: Starts with a "Prompt" followed by a sequence of "CoT" tokens (CoT 1, CoT 2, etc.) leading to a "Solution".
* **Modified CoT (X̃)**: Starts with a "Prompt", then introduces "[boLatent]" (begin latent), followed by latent tokens "z1" and "z2", then "[eoLatent]" (end latent), and continues with "CoT" tokens (CoT 33, CoT N) leading to a "Solution".
* A dashed box and arrow visually connect the standard CoT sequence (CoT 1 to CoT 32) to the latent token sequence in the modified CoT.
### Key Observations
* The diagram illustrates the insertion of latent tokens within the Chain of Thought process.
* The "[boLatent]" and "[eoLatent]" tokens act as delimiters, marking the beginning and end of the latent token sequence.
* The latent tokens are represented as "z1" and "z2", indicating discrete latent variables.
### Interpretation
The diagram demonstrates a method for incorporating latent variables into a Chain of Thought reasoning process. By introducing latent tokens, the model can potentially capture more nuanced or abstract aspects of the problem-solving process. The use of delimiters allows for clear separation and identification of the latent token sequence within the overall CoT. This approach could enable the model to explore different reasoning paths or internal representations before continuing with the explicit Chain of Thought.
