## System Architecture Diagram: LLM Answer Filtering via EigenScore ### Overview The image is a technical system architecture diagram illustrating a process for filtering answers generated by a Large Language Model (LLM). The system takes a question as input, processes it through an LLM to generate multiple candidate answer embeddings, computes an Eigenvector from these embeddings, and uses a derived "EigenScore" to decide whether to output a final answer or a rejection message. The flow is depicted from left to right. ### Components/Axes The diagram is composed of several interconnected blocks and decision points, with a legend at the bottom explaining the color-coding of specific elements. **1. Input Block (Leftmost, light blue rounded rectangle):** * **Label:** `Input` * **Content:** A sample question: `Q: On what date in 1969 did Neil Armstrong first set foot on the Moon?` **2. LLM Processing Block (Center-left, light green rounded rectangle):** * **Label:** `LLM` (at the top) * **Internal Components (from bottom to top):** * `Input Tokens` (represented by four small grey rectangles). * `Decoder` (large teal rectangle). * `Feature Clip` (blue rectangle). * `FC Layer` (teal rectangle). * `Output Logit` (a single pink rectangle at the top). * **Flow:** Arrows indicate data flow from `Input Tokens` up through the `Decoder`, `Feature Clip`, and `FC Layer` to the `Output Logit`. **3. Embedding Generation (Center, branching from LLM):** * The LLM output branches into multiple parallel paths, each generating an "Embedding of answer". * **Labels:** `Embedding of answer 1`, `Embedding of answer 2`, ..., `Embedding of answer K`. * **Visual Representation:** Each embedding is shown as a horizontal bar composed of multiple colored segments (purple, orange, yellow, etc.), representing a `Sentence Embedding` as per the legend. **4. Eigenvector Computation Block (Center-right, light blue rounded rectangle):** * **Input:** The collection of K answer embeddings. * **Internal Representation:** A matrix symbol `[...]` containing the colored embedding bars. * **Output:** An `Eigenvector`, visualized as three colored arrows (orange, blue, green) radiating from a central point. **5. Decision Diamond (Right, orange diamond):** * **Label:** `High EigenScore?` * **Function:** This is a decision node that evaluates the computed Eigenvector. **6. Output Blocks (Rightmost):** * **Top Output (Green rounded rectangle, "No" path):** * **Label:** `Output` * **Content:** `The answer is 20th July.` * **Bottom Output (Green rounded rectangle, "Yes" path):** * **Label:** `Output` * **Content:** `Sorry we don't support answer for this question.` **7. Legend (Bottom of the image):** * **Token Embedding:** Light yellow rectangle. * **Current Token Embedding:** Orange rectangle. * **Output Logit:** Pink rectangle. * **Sentence Embedding:** A horizontal bar composed of multiple small black-outlined rectangles. ### Detailed Analysis The diagram details a specific technical workflow: 1. **Input Processing:** A natural language question is fed into an LLM. 2. **Candidate Generation:** The LLM's decoder architecture processes the input tokens to generate multiple potential answer candidates (from 1 to K). Each candidate is represented as a high-dimensional vector (a `Sentence Embedding`). 3. **Dimensionality Reduction & Analysis:** These K embeddings are analyzed together. The system computes an `Eigenvector` from this set, likely through a method like Principal Component Analysis (PCA) or a similar spectral technique. The Eigenvector captures the principal directions of variance among the candidate answers. 4. **Scoring & Decision:** A scalar "EigenScore" is derived from this Eigenvector. The diagram implies this score measures the consistency, confidence, or semantic coherence of the candidate answers. * If the EigenScore is **NOT High** (the "No" path), the system proceeds to generate and output a specific answer (e.g., "20th July"). * If the EigenScore **IS High** (the "Yes" path), the system interprets this as an indicator of an unsupported or problematic question and outputs a rejection message instead. ### Key Observations * **Process Flow:** The flow is strictly linear and unidirectional from input to output, with a single branching point at the decision diamond. * **Color-Coding Consistency:** The colors in the legend are used consistently in the diagram. The `Sentence Embedding` bars in the center match the legend's pattern. The `Output Logit` (pink) in the LLM block matches the legend. * **Spatial Grounding:** The `Input` is on the far left. The `LLM` block is central-left. The `Embedding` generation is in the center. The `Eigenvector` computation is center-right. The `Decision` and `Output` blocks are on the far right. The `Legend` is anchored at the bottom-left. * **Example Logic:** The sample question about Neil Armstrong leads to a "No" decision (Low EigenScore), resulting in a factual answer. This suggests the system is designed to answer straightforward, fact-based questions confidently. A "High EigenScore" likely corresponds to questions where the LLM generates highly variable, uncertain, or nonsensical candidate answers, triggering the rejection. ### Interpretation This diagram illustrates a **confidence or reliability filtering mechanism** for LLM outputs. Instead of relying on a single output, the system generates multiple candidate answers and analyzes their collective properties. * **What it demonstrates:** The core idea is that the *variance* or *structure* among multiple generated answers (captured by the Eigenvector and its score) can be a proxy for the model's confidence or the question's answerability. A low variance (or a specific spectral signature) among candidates suggests consensus, leading to answer output. High variance or an anomalous spectral signature suggests uncertainty, leading to a refusal. * **Relationships:** The LLM is the generator. The embedding analysis block is the evaluator. The decision diamond is the gatekeeper. This creates a feedback loop where the model's own output distribution is used to self-assess reliability before finalizing a response. * **Notable Implications:** This approach moves beyond simple token-level probability scores. It uses a more holistic, semantic-level analysis of multiple potential responses. It's a form of **ensemble reasoning** or **self-consistency checking** implemented within a single model's inference pass. The rejection message ("Sorry we don't support answer for this question") implies this system is part of a larger application that defines a specific domain of "supportable" questions, and this EigenScore mechanism is the filter for that domain.