## Chart/Diagram Type: Document Screenshot
### Overview
The image is a technical document discussing **evaluation methods for Large Language Models (LLMs)**. It categorizes evaluation approaches into three main types: **LLM-based**, **human-based**, and **hybrid-based**. The document includes two tables: one detailing evaluation methods and their descriptions, and another comparing evaluation approaches with their respective metrics.
---
### Components/Axes
#### **First Table**
- **Columns**:
1. **Method**: Names of evaluation methods (e.g., Chain-of-Thought, Retrieval-Augmented Generation).
2. **Description**: Explanation of the method and its evaluation criteria.
3. **Hybrid**: Indicates if the method is hybrid (Y/N).
4. **Evaluation Method**: Specific metrics used (e.g., accuracy, BLEU, ROUGE).
#### **Second Table**
- **Columns**:
1. **평가 방법 (평가 기법이 평가할 수 있는 점)**: Evaluation method (e.g., CoT, ToT, RAG).
2. **사상적 일관성 (Consistency)**: Accuracy of reasoning.
3. **논리적 일관성 (Logical Consistency)**: Coherence of arguments.
4. **추론의 정확성 (Accuracy of Reasoning)**: Correctness of outputs.
5. **평가원의 평가 (Evaluator's Evaluation)**: Human or automated assessment.
---
### Detailed Analysis
#### **First Table**
1. **Chain-of-Thought (CoT)**:
- **Description**: Uses reasoning traces to evaluate outputs. Evaluates **accuracy** and **consistency** (Y).
- **Evaluation Method**: Accuracy, Consistency.
2. **Tree-of-Thoughts (ToT)**:
- **Description**: Explores multiple reasoning paths. Evaluates **accuracy** and **success rate** (Y).
- **Evaluation Method**: Accuracy, Success Rate.
3. **Retrieval-Augmented Generation (RAG)**:
- **Description**: Combines retrieval and generation. Evaluates **BLEU** and **ROUGE** (Y).
- **Evaluation Method**: BLEU, ROUGE.
4. **Self-Consistency**:
- **Description**: Generates multiple responses and selects the most consistent one. Evaluates **accuracy** and **consistency** (Y).
- **Evaluation Method**: Accuracy, Consistency.
5. **Logical Reasoning**:
- **Description**: Focuses on logical correctness. Evaluates **logical consistency** (Y).
- **Evaluation Method**: Logical Consistency.
6. **Human Evaluation**:
- **Description**: Human judges assess outputs. Evaluates **accuracy** and **consistency** (N).
- **Evaluation Method**: Accuracy, Consistency.
#### **Second Table**
- **CoT (Chain-of-Thought)**:
- **Consistency**: Y (Accuracy of reasoning).
- **Logical Consistency**: Y.
- **Accuracy of Reasoning**: Y.
- **Evaluator's Evaluation**: Human.
- **ToT (Tree-of-Thoughts)**:
- **Consistency**: Y.
- **Logical Consistency**: Y.
- **Accuracy of Reasoning**: Y.
- **Evaluator's Evaluation**: Human.
- **RAG (Retrieval-Augmented Generation)**:
- **Consistency**: N.
- **Logical Consistency**: N.
- **Accuracy of Reasoning**: N.
- **Evaluator's Evaluation**: Automated (BLEU, ROUGE).
---
### Key Observations
1. **LLM-Based Methods** (e.g., CoT, ToT, Self-Consistency) focus on **reasoning quality** (accuracy, consistency) rather than language modeling metrics.
2. **Retrieval-Based Methods** (e.g., RAG) prioritize **language modeling metrics** (BLEU, ROUGE) for factual correctness.
3. **Hybrid Methods** (e.g., CoT+RAG) combine reasoning and retrieval, using a mix of metrics.
4. **Human Evaluation** is used for subjective aspects like fluency and coherence but is labor-intensive.
---
### Interpretation
The document highlights that **LLM evaluation is context-dependent**:
- **Reasoning-centric tasks** (e.g., math problems) require metrics like **accuracy** and **consistency**.
- **Factual retrieval tasks** (e.g., QA with external knowledge) rely on **BLEU/ROUGE**.
- **Hybrid approaches** (e.g., CoT+RAG) aim to balance reasoning and factual correctness, reflecting the complexity of real-world applications.
The absence of numerical data suggests this is a **qualitative framework** for selecting evaluation methods based on LLM use cases.
