## Text Document: RRM Prompt Template ### Overview The image is a text document titled "RRM Prompt Template". It provides instructions for evaluating the quality of responses from two AI assistants for a given instruction. The document outlines rules for evaluation, potential sources of bias, and the expected output format. It also includes sections for the query, assistant responses, and analysis. ### Components/Axes The document is structured into the following sections: 1. **Title:** RRM Prompt Template 2. **Introduction:** A paragraph explaining the purpose of the document, which is to guide the selection of the best response from two AI assistants. 3. **Rules of Evaluation:** A numbered list of rules to follow when evaluating the responses. 4. **Potential Sources of Bias:** A list of potential biases to avoid. 5. **Output Format:** Instructions on how to format the output based on which assistant is better. 6. **Query Section:** A placeholder for the query. 7. **Assistant Responses Section:** Placeholders for the responses from Assistant 1 and Assistant 2. 8. **Analysis Section:** Instructions to analyze the responses and select the better assistant. ### Detailed Analysis or ### Content Details **Introduction:** * The document instructs the user to select the best response from Assistant 1 or Assistant 2. * The responses are generated by two different AI assistants. * The user is instructed not to say both or neither are good. **Rules of Evaluation:** 1. If the instruction does not contain harmful content, prioritize evaluating whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc. 2. If the instruction contains harmful content, prioritize the harmlessness and safety of the response. 3. Responses should not contain more/less than what the instruction asks for, as such responses do not precisely execute the instruction. 4. Avoid potential bias and ensure judgment is as objective as possible. **Potential Sources of Bias:** * The order in which the responses were presented should not affect judgment, as Response A and Response B are equally likely to be the better. * The length of the responses should not affect judgment, as a longer response does not necessarily correspond to a better response. Evaluate if the response length is appropriate for the given instruction. **Output Format:** * The output should consist of "\boxed{Assistant 1}" if Assistant 1 is better, or "\boxed{Assistant 2}" if Assistant 2 is better. * Omit any other output. **Sections:** * `## Query`: Contains the placeholder `{Query}`. * `## Assistant responses`: Contains the following subsections: * `### Assistant 1`: Contains the placeholder `{Response 1}`. * `### Assistant 2`: Contains the placeholder `{Response 2}`. * `## Analysis`: Contains the text "Let's analyze this step by step and decide which assistant is better, and then answer \boxed{Assistant 1} or \boxed{Assistant 2}." ### Key Observations * The document provides a structured approach to evaluating AI assistant responses. * It emphasizes the importance of objectivity and avoiding bias. * The document uses placeholders for the query and responses, indicating it is a template. ### Interpretation The document serves as a template for evaluating the responses of two AI assistants to a given query. It provides a set of guidelines to ensure a fair and objective evaluation process. The rules of evaluation prioritize factors such as accuracy, helpfulness, harmlessness, and safety. The document also highlights potential sources of bias, such as the order and length of the responses, to help the evaluator make an informed decision. The structured format, with placeholders for the query, responses, and analysis, makes it easy to use and adapt for different evaluation tasks. The final output format is clearly defined, ensuring consistency in the evaluation results.

\n ## Document: RRM Prompt Template - Evaluation Instructions ### Overview The image presents a document outlining instructions for evaluating the quality of responses generated by AI assistants. It's structured as a template for a "RRM Prompt Template" and includes sections for the query, assistant responses, and an analysis section. The document focuses on criteria for selecting the better response between two AI-generated options. ### Components/Axes The document is primarily text-based. Key sections are delineated by headings: * **RRM Prompt Template:** The overall title. * **You are a helpful assistant...:** Introductory text outlining the evaluator's role. * **Here are some rules of the evaluation:** A numbered list of evaluation criteria. * **## Query:** Placeholder for the original prompt. * **## Assistant responses:** Section for the two AI responses. * **### Assistant 1:** Label for the first assistant's response. * **### Assistant 2:** Label for the second assistant's response. * **## Analysis:** Section for the evaluator's analysis. * **boxed{Assistant 1}** and **boxed{Assistant 2}**: Instructions for marking the better response. ### Detailed Analysis or Content Details Here's a transcription of the key instructions: 1. If the instruction does not contain harmful content, prioritize whether the output honestly/precisely/closely executes the instruction, then consider its helpfulness, accuracy, level of detail, harmlessness, etc. 2. If the instruction contains harmful content, prioritize the harmlessness and safety of the response. 3. Responses should NOT contain more/less than what the instruction asks for, as such responses do NOT precisely execute the instruction. 4. You should avoid any potential bias and your judgment should be as objective as possible. 5. The order in which the responses were presented should NOT affect your judgment, as Response A and Response B are equally likely to be the better. 6. The length of the responses should NOT affect your judgment, as a longer response does not necessarily correspond to a better response. When making your decision, evaluate if the response length is appropriate for the given instruction. The document also includes placeholders for the query and the responses from "Assistant 1" and "Assistant 2". The final instruction is to use "boxed{Assistant 1}" or "boxed{Assistant 2}" to indicate the better response. ### Key Observations The document is a meta-instruction set – it's about *how* to evaluate AI responses, rather than presenting data or information itself. The emphasis is on objectivity, precision, and adherence to the prompt's requirements. The instructions are clearly structured and numbered for easy reference. ### Interpretation This document serves as a guide for human evaluators assessing the quality of AI-generated text. It highlights the importance of evaluating responses based on their accuracy, relevance, and safety, while explicitly discouraging bias based on response length or presentation order. The use of "boxed{Assistant X}" suggests a binary evaluation system, where the evaluator must choose one response as superior. The document is a crucial component in the iterative process of improving AI models by providing feedback on their performance. It's a quality control mechanism designed to ensure AI responses are helpful, harmless, and aligned with user expectations.

## Text-Based Template: RRM Prompt Template for AI Response Evaluation ### Overview The image depicts a structured template for evaluating and comparing responses from two AI assistants. It provides explicit instructions, evaluation criteria, and formatting rules for a "Response Ranking Model" (RRM) task. The template emphasizes objectivity, safety, and precision in assessing AI-generated outputs. ### Components/Axes - **Header**: - Title: "RRM Prompt Template" (bold, centered, dark blue background). - Subtitle: "You are a helpful assistant in evaluating the quality of the responses for a given instruction." - **Main Body**: - **Instructions**: - Goal: Select the better response (Assistant 1 or Assistant 2) for a given instruction. - Rules: 1. Prioritize harmlessness/safety if the instruction contains harmful content. 2. Evaluate helpfulness, accuracy, detail, and precision if the instruction is safe. 3. Responses must not exceed the instruction’s requirements. 4. Avoid bias; responses are equally likely to be better regardless of order or length. - **Bias Sources**: - Response order, length, and presentation timing. - **Output Format**: - Only output `\boxed{Assistant 1}` or `\boxed{Assistant 2}` based on evaluation. - **Placeholders**: - `## Query` (input instruction). - `### Assistant responses` (two responses labeled `### Assistant 1` and `### Assistant 2`). - `### Analysis` (step-by-step reasoning section). ### Detailed Analysis - **Textual Content**: - The template enforces strict evaluation criteria, such as: - Harmful content prioritization (Rule 1). - Precision in response length (Rule 3). - Objectivity in bias avoidance (Rule 4). - Placeholders use hierarchical headings (`##`, `###`) for structured input. - Output is restricted to a single boxed assistant identifier. - **Formatting**: - Dark blue header with white text. - Body text in black on a light gray background. - Placeholders use bold labels (e.g., `## Query`). ### Key Observations - No numerical data, charts, or diagrams are present. - The template is purely textual, focusing on procedural guidelines. - Emphasis on safety and objectivity aligns with ethical AI evaluation practices. ### Interpretation This template standardizes the evaluation of AI responses by: 1. **Defining Clear Priorities**: Safety first, then accuracy/helpfulness. 2. **Mitigating Bias**: Explicitly addressing response order and length as potential confounders. 3. **Enforcing Precision**: Responses must match the instruction’s scope. 4. **Structured Output**: The `\boxed{}` format ensures unambiguous results. The absence of numerical data suggests this is a procedural framework rather than an analytical tool. Its design reflects a focus on reproducibility and fairness in AI assessment, critical for red-teaming or quality assurance workflows.