\n ## Document: Prompt for Response Generation ### Overview The image presents a document outlining a process for evaluating the correctness of solutions to questions, specifically within a question/solution pair context. It details the criteria for assessing solution steps as correct, neutral, or incorrect, and provides a structured format for reporting the analysis. ### Components/Axes The document is structured with the following key components: * **Header:** "Prompt for Response Generation" * **Introduction:** Explains the task of examining solutions and determining correctness. * **Definitions:** Defines "Solution Correctness", "First Error Step", and "Error Reason". * **Input Section:** Contains placeholders for the question, options, and the model's solution steps, denoted by `{sol['Question']}`, `{sol['Options']}`, and `{sol['Model_Solution_Steps']}` respectively. * **Hint Section:** Provides instructions for the desired response format. * **Output Format:** Specifies the required structure for the solution analysis, including "Solution Analysis", "Solution Correctness", "First Error Step", and "Error Reason". ### Detailed Analysis or Content Details The document's content can be transcribed as follows: "Following a question/solution pair in subject {sol['Subject']}. Your task is to examine the solutions step by step and determine the solution correctness. Following are the specific definitions of the fields: Solution Correctness: Does the solution correctly answer the question with justifiable reasoning and selected the corrected options? First Error Step: For every step it can either be correct, neutral or incorrect. Correct steps are those that possess sound logic and correct computation and lead to the correct answer. Neutral steps are those step that are explanatory, exploring or focusing on background illustration. They have no obvious mistakes but is not very clear if they lead to the correct answer. Incorrect steps are those with factual errors, computation errors or understanding/logic errors. These steps might or might not detour the reasoning path to incorrect answers. We need to single out the first step that comes with above errors or lead to incorrect answers. Error Reason: For the identified first error step, please specify the errors made in this step and suggest a rectified reasoning step instead. {_k_shot_demo} Below is the question and solution for you to solve: Question: {sol['Question']} Options: {sol['Options']} Step by Step Solution: {sol['Model_Solution_Steps']} {hint_sent} Please follow the desired response format: Solution Analysis: [Give a step by step analysis on the solution correctness here] Solution Correctness: [Input 'correct'/'incorrect' here to indicate the overall correctness of the solution] First Error Step: [Input 'Step X' here to indicate the first error step here. Input 'N/A' if the solution is correct.] Error Reason: [Input the error reason and the rectified reasoning of the first error step here. Input 'N/A' if the solution is correct.] Please follow this format without any additional introductory or concluding statements." ### Key Observations The document is a procedural guide. It emphasizes a step-by-step evaluation process and provides clear definitions for assessing the quality of each step. The use of placeholders suggests it's intended to be used with a specific question/solution dataset. The strict formatting requirements indicate a need for consistent and structured feedback. ### Interpretation This document outlines a rigorous methodology for evaluating the correctness of solutions, likely within an automated or semi-automated system. The detailed definitions and structured output format suggest a focus on identifying specific errors and providing constructive feedback. The placeholders for the question, options, and solution steps indicate that this document is part of a larger system designed to assess and improve the performance of a model or algorithm in answering questions. The emphasis on "justifiable reasoning" suggests that the evaluation process is not solely based on the correctness of the answer, but also on the quality of the explanation provided. The document is a set of instructions, not a presentation of data or findings.