## Communication Diagram: Prompt Engineering Strategies ### Overview The image presents two contrasting approaches to prompt engineering for multi-agent systems: Context-Poor Communication with Traditional Evaluation and Structured Communication with Hierarchy Refinement. It illustrates the flow of information and tasks between a supervisor and multiple agents (generators and evaluators) in each approach, highlighting the differences in task assignment, response quality, and overall coordination. ### Components/Axes **Left Side: Context-Poor Communication** * **Roles:** Supervisor, Agent 1 (Generator), Agent 2 (Evaluator 1), Agent 3 (Evaluator 2) * **Flow:** Sequential, Supervisor -> Agent 1 -> Agent 2 -> Agent 3 -> Supervisor * **Task Assignment:** General, lacking specific details or context. * **Responses:** Prone to errors due to non-organized instruction and forgotten context. **Right Side: Structured Communication** * **Roles:** Supervisor, Member Agent 1 (Generator), Member Agent 2 (Evaluator 1), Member Agent 3 (Evaluator 2), Evaluation Team Supervisor * **Flow:** Iterative, Supervisor <-> Agent 1, Supervisor <-> Agent 2, Supervisor <-> Agent 3, with feedback loops. * **Task Assignment:** Specific subtasks with message, intermediate output, and background information. * **Responses:** Accurate, including response and other intermediate output. * **Evaluation:** Hierarchical, with summarized and coordinated feedback. ### Detailed Analysis or ### Content Details **Context-Poor Communication (Left Side)** 1. **Task Assignment:** * ① Task: "Generate text...." (Supervisor to Agent 1) * ① "Your sub task is... You need...." (Only-text based, Length Task) (Supervisor to Member Agent 1) * ③ "Your sub task is... You need...." (Only-text based, Length Task) (Supervisor to Member Agent 2) 2. **Responses:** * ② "My Generated text is...." (Agent 1 to Agent 2) * ② "The output of mine is :..." Bad Response due to Non-organized instruction (Member Agent 1) * ④ "My evaluated result is...." (Agent 3 to Supervisor) * ④ "The output of mine is :..." Bad Response due to Non-organized instruction (Supervisor to Member Agent 2) * ③ " My evaluated result is...." (Agent 2 to Agent 3) **Structured Communication (Right Side)** 1. **Task Assignment:** * ① "Generate text...." (Supervisor to Member Agent 1) * ① Well-organized communication protocol (Supervisor to Member Agent 1): * a. (Message With Specified Subtask): "Your Subtask is to ...the format is...." * b. (Intermediate Output) : "..." * c. (Background): "We are discussing ..." * ③ Well-organized communication protocol (Supervisor to Member Agent 2): * a. (Message With Specified Subtask): "Your Subtask is to evaluate ...the format..." * b. (Intermediate Output): "..." * c. (Background): "We are discussing ..." * ③ "Evaluate the text ......" (Evaluation Team Supervisor to Member Agents 1, 2, 3) 2. **Responses:** * ② "My generated text is...." (Member Agent 1 to Member Agent 2) * ② Accurate Response with Intermediate Output (Member Agent 1) * ④ Accurate Response with Intermediate Output (Member Agent 2) * ④-a "Evaluate it from {placeholder= Criterion 1}" (Member Agent 2 to Member Agent 3) * ⑤-a "The Score of {placeholder= Criterion 1} is ....." (Member Agent 2 to Member Agent 3) * ④-b "Evaluate it from {placeholder= Criterion 2}" (Member Agent 3 to Evaluation Team Supervisor) * ⑤-b "The Score of {placeholder= Criterion 2} is ....." (Member Agent 3 to Evaluation Team Supervisor) * ⑥ "The {placeholder= Criterion 1} is good, but the {placeholder= Criterion 2} is not good" (Evaluation Team Supervisor to Supervisor) ### Key Observations * **Context-Poor Communication:** Results in bad responses due to non-organized instruction and lack of context. The agents seem to be working in isolation, leading to potential inconsistencies and errors. * **Structured Communication:** Emphasizes well-organized communication protocols, specific subtasks, and intermediate outputs. This approach promotes accurate responses and coordinated feedback, leading to higher quality results. * **Hierarchy Refinement:** The structured approach incorporates a hierarchical evaluation process, where an evaluation team supervisor provides summarized and coordinated feedback, balancing diverse inputs and mitigating biases. ### Interpretation The diagram illustrates the significant impact of prompt engineering on the effectiveness of multi-agent systems. Context-poor communication, characterized by vague task assignments and a lack of coordination, leads to poor responses and potentially biased outcomes. In contrast, structured communication, with its emphasis on clear instructions, intermediate outputs, and hierarchical evaluation, fosters collaboration, reduces errors, and enhances overall task quality. The diagram suggests that careful prompt engineering is crucial for harnessing the full potential of multi-agent systems, particularly in complex tasks requiring diverse inputs and coordinated feedback. The "Hierarchy Refinement" approach is superior because it addresses the limitations of traditional evaluation methods by promoting a more holistic and unbiased assessment of the generated content.

\n ## Diagram: Communication Protocols in Multi-Agent Systems ### Overview The image presents a comparative diagram illustrating two communication protocols in multi-agent systems: "Context-Poor Communication" and "Structured Communication." Each protocol is depicted with a Supervisor agent and two Member agents (Agent 1 and Agent 2). The diagram highlights the differences in task assignment and response quality between the two approaches. The diagram also includes a scoring rubric for evaluating responses. ### Components/Axes The diagram is divided into two main sections, one for each communication protocol. Each section contains: * **Supervisor:** Represented by a head icon, initiating tasks. * **Member Agent 1 (Generator):** Represented by a head icon, generating responses. Labeled with "(Role: Generator)". * **Member Agent 2 (Evaluator):** Represented by a head icon, evaluating responses. Labeled with "(Role: Evaluator)". * **Communication Arrows:** Illustrating the flow of prompts, outputs, and evaluations. * **Text Boxes:** Containing example prompts, responses, and evaluation criteria. * **Scoring Rubric:** A table at the bottom detailing evaluation criteria (a-b) and scores. ### Detailed Analysis or Content Details **Context-Poor Communication (Left Side):** 1. **Prompt (Supervisor to Agent 1):** "Your sub task is…You need…" (Only-text based, Length Task). 2. **Agent 1 Response:** "The output of mine is…" Bad Response due to Non-organized instruction. 3. **Prompt (Supervisor to Agent 2):** "Your sub task is…You need…" (Only-text based, Length Task). 4. **Agent 2 Response:** "The output of mine is…" Bad Response due to Non-organized instruction. 5. **Annotation:** "Context-Poor Communication: Assigning the next agent without providing well-organized information. ② Bad Responses: Producing poor responses due to forgetting key issues, such as the context of the question or intermediate format." **Structured Communication (Right Side):** 1. **Prompt (Supervisor to Agent 1):** "Generate text…" 2. **Agent 1 Response:** "My generated text is…" 3. **Prompt (Supervisor to Agent 2):** "Evaluate it from…" 4. **Agent 2 Response:** "My evaluated result is…" These are good as… 5. **Annotation:** "Structured Communication: Assigning specific subtask to the next agent with message, intermediate output and background. ② Accurate responses: including response and other intermediate output." **Scoring Rubric (Bottom):** The rubric is presented as a table with two rows and two columns. | | a. (Placeholder-Criterion 1) | b. (Placeholder-Criterion 2) | |--------------|------------------------------|------------------------------| | **A** | "The [placeholder-Criterion 1] is good but the [placeholder-Criterion 2] is not good" | "The Score of [placeholder-Criterion 1] is…" | | **B** | "Evaluate it from [placeholder-Criterion 2]" | "The Score of [placeholder-Criterion 2] is…" | **Additional Text:** * "How Many?" * "In what Format?" * "Metrics? Evaluate what?" * "To finish this I need…" ### Key Observations * The "Structured Communication" protocol consistently yields "good" responses, while the "Context-Poor Communication" protocol results in "bad" responses. * The scoring rubric uses placeholder criteria, suggesting it's a template for evaluation. * The diagram emphasizes the importance of providing context and intermediate outputs in multi-agent communication. * The diagram uses a consistent visual style with head icons representing agents and arrows indicating communication flow. ### Interpretation The diagram demonstrates the critical role of well-defined communication protocols in multi-agent systems. The "Structured Communication" approach, which includes clear task assignments, intermediate outputs, and background information, leads to more accurate and useful responses. Conversely, the "Context-Poor Communication" approach, lacking these elements, results in poor-quality responses. The scoring rubric suggests a framework for evaluating the quality of responses based on specific criteria. The diagram highlights a fundamental principle in AI and distributed systems: effective communication is essential for collaboration and achieving desired outcomes. The use of placeholders in the scoring rubric indicates that the specific evaluation criteria will vary depending on the task and the agents involved. The diagram serves as a visual guide for designing and implementing robust communication protocols in multi-agent systems. The diagram is a conceptual illustration, and the specific details of the prompts and responses are examples to demonstrate the difference in outcomes. The diagram is not presenting factual data, but rather a conceptual comparison.

\n ## Diagram: Multi-Agent Communication Protocols ### Overview The image is a technical diagram illustrating and contrasting four different communication protocols for multi-agent systems, specifically focusing on task assignment and evaluation workflows. It is divided into four quadrants, each depicting a distinct method with labeled agents, directional arrows representing communication flow, and explanatory text. The diagram uses color-coded agents (pink for Generator, green/blue for Evaluators, black for Supervisor) and numbered steps to show sequences. ### Components/Axes The diagram is structured into four main sections, each with a title and a visual workflow: 1. **Top-Left Quadrant: Context-Poor Communication** * **Title:** "Context-Poor Communication" * **Agents:** Supervisor (black), Member Agent 1 (Role: Generator, pink), Member Agent 2 (Role: Evaluator, green). * **Flow:** A linear, back-and-forth sequence between Supervisor and each agent individually. * **Key Text Elements:** * Prompt box: "Prompt (General Task Assignment)" * Supervisor to Agent 1: `① "Your sub task is...You need...." (Only-text based, Length Task)` * Agent 1 thought bubble: "How Many? In what Format? ..." * Agent 1 to Supervisor: `② "The output of mine is:..."` (Labeled: "Bad Response due to Non-organized instruction") * Supervisor to Agent 2: `③ "Your sub task is...You need...." (Only-text based, Length Task)` * Agent 2 thought bubble: "Metrics? Evaluate what? ..." * Agent 2 to Supervisor: `④ "The output of mine is:..."` (Labeled: "Bad Response due to Non-organized instruction") * **Explanatory Text:** "Context-Poor Communication: ① Task Assignment: Assigning the next agent without providing well-organized information. ② Bad Responses: Producing poor responses due to forgetting key issues, such as the context of the question or intermediate format." 2. **Top-Right Quadrant: Structured Communication** * **Title:** "Structured Communication" * **Agents:** Supervisor (black), Member Agent 1 (pink), Member Agent 2 (green). * **Flow:** Supervisor communicates with each agent using a structured protocol. * **Key Text Elements:** * Prompt box: "Prompt (General Task Assignment)" * Supervisor to Agent 1: `① Well-organized communication protocol` with sub-points: * `a. (Message With Specified Subtask): "Your Subtask is to ...the format is..."` * `b. (Intermediate Output): "..."` * `c. (Background): "We are discussing ..."` * Agent 1 thought bubble: "To finish this I need ...." * Agent 1 to Supervisor: `② Accurate Response with Intermediate Output` * Supervisor to Agent 2: `③ Well-organized communication protocol` (same structure as ①). * Agent 2 thought bubble: "These are good as..." * Agent 2 to Supervisor: `④ Accurate Response with Intermediate Output` * **Explanatory Text:** "Structured Communication: ① Task assignment: Assigning specific subtask to the next agent with *message, intermediate output and background*. ② Accurate responses: including response and other intermediate output." 3. **Bottom-Left Quadrant: Traditional Evaluation based Refinement** * **Title:** "Traditional Evaluation based Refinement" * **Agents:** Supervisor (black), Agent 1 (Role: Generator, pink), Agent 2 (Role: Evaluator 1, green), Agent 3 (Role: Evaluator 2, blue). * **Flow:** A strict, predefined sequential order: Supervisor → Agent 1 → Agent 2 → Agent 3 → Supervisor. * **Key Text Elements:** * Supervisor to Agent 1: `① Task: "Generate text...."` * Agent 1 to Agent 2: `② "My Generated text is...."` * Agent 2 to Agent 3: `③ "My evaluated result is...."` * Agent 3 to Supervisor: `④ "My evaluated result is...."` * **Explanatory Text:** "Traditional Evaluation based Refinement: All agents follow a predefined order, such as Supervisor → Evaluator 1 → Evaluator 2, or similar styles, without summarization or coordination. This leads to biased outcomes (sensitivity to evaluator order) or incorrect results (overlooking partial evaluations)." 4. **Bottom-Right Quadrant: Hierarchy Refinement** * **Title:** "Hierarchy Refinement" * **Agents:** Supervisor (black), Evaluation Team Supervisor (grey), Member Agent 1 (Role: Generator, pink), Member Agent 2 (Role: Evaluator 1, green), Member Agent 3 (Role: Evaluator 2, blue). * **Flow:** Hierarchical. The main Supervisor delegates to an Evaluation Team Supervisor, who coordinates multiple evaluators in parallel or structured sequence. * **Key Text Elements:** * Supervisor to Agent 1: `① "Generate text...."` * Agent 1 to Supervisor: `② " My generated text is...."` * Supervisor to Evaluation Team Supervisor: `③ "Evaluate the text ...."` * Evaluation Team Supervisor to Agent 2: `④-a " Evaluate it from {placeholder= Criterion 1}"` * Agent 2 to Evaluation Team Supervisor: `⑤-a" The Score of {placeholder= Criterion 1} is ....."` * Evaluation Team Supervisor to Agent 3: `④-b " Evaluate it from {placeholder= Criterion 2}"` * Agent 3 to Evaluation Team Supervisor: `⑤-b" The Score of {placeholder= Criterion 2} is ....."` * Evaluation Team Supervisor internal note: `⑥ "The {placeholder= Criterion 1} is good, but the {placeholder= Criterion 2} is not good"` * **Explanatory Text:** "Hierarchy Refinement: An evaluator team operates hierarchically, providing summarized and coordinated feedback, balancing diverse inputs, and mitigating biases, which collectively enhance overall task quality." ### Detailed Analysis The diagram systematically compares communication inefficiencies and solutions in multi-agent systems. * **Context-Poor vs. Structured Communication:** The top half contrasts unstructured prompts (leading to confused agents and "Bad Responses") with structured prompts that include message, intermediate output, and background context (leading to "Accurate Response"). * **Traditional vs. Hierarchy Refinement:** The bottom half contrasts a linear, sequential evaluation chain (prone to bias and order sensitivity) with a hierarchical model where a team supervisor coordinates multiple evaluators, synthesizes feedback, and provides a summarized evaluation. * **Agent Roles:** Consistently color-coded: Pink = Generator, Green/Blue = Evaluator, Black = Primary Supervisor, Grey = Evaluation Team Supervisor (in Hierarchy Refinement). * **Communication Patterns:** Arrows are strictly directional, showing the flow of tasks and responses. Numbering (①, ②, etc.) clarifies the sequence within each protocol. ### Key Observations 1. **Problem-Solution Pairing:** Each quadrant on the left (Context-Poor, Traditional) presents a problem, while the corresponding quadrant on the right (Structured, Hierarchy) presents a proposed solution. 2. **Information Density:** The "Structured" and "Hierarchy" protocols explicitly include more metadata (background, intermediate outputs, criteria) in their communication compared to their counterparts. 3. **Coordination Complexity:** The "Hierarchy Refinement" model introduces an additional supervisory layer (Evaluation Team Supervisor) to manage complexity and bias, which is absent in the "Traditional" linear model. 4. **Visual Metaphors:** Thought bubbles are used effectively to show agent confusion in the poor protocol and clarity in the structured one. ### Interpretation This diagram serves as a conceptual framework for designing robust multi-agent AI systems. It argues that the quality of agent collaboration is fundamentally dependent on the structure of their communication protocols. * **The Core Argument:** Unstructured, context-poor communication leads to failure. Effective multi-agent systems require **structured message formats** that provide necessary context and **hierarchical coordination** to synthesize diverse evaluations and mitigate individual biases. * **Underlying Principles:** The diagram promotes principles of **explicit context sharing**, **intermediate output tracking**, and **coordinated evaluation** as antidotes to common failure modes like poor task understanding and biased results. * **Practical Implication:** For engineers designing such systems, the diagram suggests moving away from simple, sequential agent chains and towards architectures with well-defined communication schemas and supervisory coordination layers to achieve higher quality and more reliable outcomes. The "Hierarchy Refinement" model is presented as the most sophisticated approach, balancing diverse inputs through structured coordination.

## Diagram: Communication Protocol Comparison and Evaluation Methods ### Overview The image is a comparative diagram illustrating two communication protocols ("Context-Poor Communication" and "Structured Communication") and two evaluation methodologies ("Traditional Evaluation based Refinement" and "Hierarchy Refinement"). It uses color-coded robotic agents (pink, green, blue) to represent roles (Generator, Evaluator, Supervisor) and arrows to depict workflows. The diagram emphasizes structured communication's advantages in task clarity and evaluation robustness. --- ### Components/Axes #### Left Side: Context-Poor Communication 1. **Task Assignment** - Supervisor assigns ambiguous tasks (e.g., "Your sub task is... You need..."). - Member Agent 1 (Generator) produces a "Bad Response" due to non-organized instructions. - Member Agent 2 (Evaluator) critiques the response using vague metrics. 2. **Traditional Evaluation based Refinement** - Linear workflow: Supervisor → Agent 1 (Generator) → Agent 2 (Evaluator 1) → Agent 3 (Evaluator 2). - Criticism: Biased outcomes due to evaluator order sensitivity and lack of coordination. #### Right Side: Structured Communication 1. **Well-Organized Protocol** - Task assignment includes **specified subtasks**, **intermediate outputs**, and **background context**. - Example: "Your subtask is to... format is..." with structured feedback loops. 2. **Accurate Response with Intermediate Output** - Member Agent 1 (Generator) produces text with intermediate outputs (e.g., "We are discussing..."). - Member Agent 2 (Evaluator) provides structured feedback (e.g., "These are good as..."). #### Bottom: Evaluation Methodologies 1. **Traditional Evaluation** - Linear evaluation chain: Supervisor → Evaluator 1 → Evaluator 2. - Criticism: Sensitivity to evaluator order and lack of coordination. 2. **Hierarchy Refinement** - Hierarchical evaluator team with summarized/coordinated feedback. - Mitigates biases by balancing diverse inputs and overseeing partial evaluations. --- ### Detailed Analysis #### Context-Poor Communication - **Task Assignment**: Ambiguous instructions lead to poor responses (e.g., "How Many? In what Format?"). - **Bad Responses**: Generated due to missing context (e.g., question context, intermediate format). - **Evaluation**: Evaluators use vague metrics ("Metrics? Evaluate what?") without structured criteria. #### Structured Communication - **Specified Subtasks**: Clear task breakdown (e.g., "Evaluate it from placeholder=Criterion 1"). - **Intermediate Outputs**: Arrows show iterative feedback (e.g., "The Score of placeholder=Criterion 1 is..."). - **Coordinated Feedback**: Evaluators reference specific criteria (e.g., "placeholder=Criterion 2"). #### Evaluation Methodologies - **Traditional Evaluation**: - Agent 3 (Evaluator 2) critiques Agent 1’s output without oversight. - Risk of biased outcomes due to sequential, uncoordinated evaluations. - **Hierarchy Refinement**: - Evaluators operate hierarchically, with summaries and coordination. - Example: "Evaluate it from placeholder=Criterion 2" ensures alignment with predefined metrics. --- ### Key Observations 1. **Color Coding**: - Pink boxes denote "Bad Response" (context-poor) and "Evaluate it from..." (structured). - Green boxes indicate "Accurate Response" and "Good as..." feedback. - Blue boxes represent Evaluator roles. 2. **Workflow Complexity**: - Structured communication introduces more feedback loops (e.g., "Evaluate it from..." → "The Score of..."). - Hierarchy refinement adds layers of evaluator oversight compared to traditional methods. 3. **Textual Emphasis**: - Structured protocol explicitly includes **message**, **intermediate output**, and **background** in task assignments. - Traditional evaluation lacks coordination (e.g., "My evaluated result is..." without criteria). --- ### Interpretation 1. **Structured Communication Advantages**: - By specifying subtasks and intermediate outputs, structured communication reduces ambiguity and improves response accuracy. - Example: "We are discussing..." (intermediate output) ensures evaluators understand context. 2. **Evaluation Methodology Impact**: - Traditional evaluation’s linear workflow risks bias (e.g., Evaluator 2’s critique may override Evaluator 1’s feedback). - Hierarchy refinement mitigates this by coordinating evaluators and balancing diverse inputs. 3. **Critical Insight**: - The diagram suggests that structured communication and hierarchical evaluation are interdependent for optimal task quality. - Without structured communication, even hierarchical evaluation struggles due to poor initial task clarity. 4. **Anomalies**: - Context-poor communication lacks explicit criteria (e.g., "Metrics? Evaluate what?"), leading to subjective evaluations. - Structured communication’s use of "placeholder=Criterion X" ensures objective, repeatable assessments. --- ### Conclusion The diagram demonstrates that structured communication protocols (with clear subtasks and intermediate outputs) paired with hierarchical evaluation methodologies (coordinated feedback) collectively enhance task quality. Context-poor communication and traditional evaluation methods are prone to ambiguity and bias, respectively. The use of color-coded roles and arrows effectively visualizes these relationships, emphasizing the need for both task clarity and evaluator coordination.