# SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems
**Authors**: Yunhao Feng, Yifan Ding, Yingshui Tan, Boren Zheng, Yanming Guo, Xiaolong Li, Kun Zhai, Yishan Li, Wenke Huang
## Abstract
Skill-based agent systems tackle complex tasks by composing reusable skills, improving modularity and scalability while introducing a largely unexamined security attack surface. We propose SkillTrojan, a backdoor attack that targets skill implementations rather than model parameters or training data. SkillTrojan embeds malicious logic inside otherwise plausible skills and leverages standard skill composition to reconstruct and execute an attacker-specified payload. The attack partitions an encrypted payload across multiple benign-looking skill invocations and activates only under a predefined trigger. SkillTrojan also supports automated synthesis of backdoored skills from arbitrary skill templates, enabling scalable propagation across skill-based agent ecosystems. To enable systematic evaluation, we release a dataset of 3,000+ curated backdoored skills spanning diverse skill patterns and trigger–payload configurations. We instantiate SkillTrojan in a representative code-based agent setting and evaluate both clean-task utility and attack success rate. Our results show that skill-level backdoors can be highly effective with minimal degradation of benign behavior, exposing a critical blind spot in current skill-based agent architectures and motivating defenses that explicitly reason about skill composition and execution. Concretely, on EHR SQL, SkillTrojan attains up to 97.2% ASR while maintaining 89.3% clean ACC on GPT-5.2-1211-Global.
Machine Learning, ICML
## 1 Introduction
Skill-based abstractions are now a prevalent design pattern in agent systems (Li et al., 2025b; Wang et al., 2025; Zheng et al., 2025b). Instead of emitting low-level actions, agents invoke and compose reusable skills that encapsulate procedural logic, tool calls, and execution workflows. This design improves modularity and scalability, and it underlies widely used agent frameworks and emerging skill marketplaces. In practice, skills often execute code, maintain internal state, and mediate access to external resources, which makes them convenient units for reuse and rapid deployment. These same properties, however, concentrate trust in skill implementations and expand the agent’s attack surface beyond what can be inferred from model input–output behavior alone (Liu et al., 2024; Li, 2026).
<details>
<summary>2604.06811v1/x1.png Details</summary>
### Visual Description
## Diagram: AI Agent Workflow with Iterative Reasoning
### Overview
The diagram illustrates a cyclical AI agent workflow centered around an LLM (Large Language Model) with iterative reasoning. It integrates three core skill layers (Planning, Memory, Tool Execution) and an Execution Skill, connected by feedback loops and triggers. The process begins with a "Trigger" and concludes with a "payload executed" state, emphasizing dynamic interaction between components.
---
### Components/Axes
#### Labels and Sections:
1. **Skill Layer** (outermost circle):
- **Planning** (blue section):
- Task decomposition
- Workflow architecture
- **Memory** (green section):
- Contextual recall
- Long-term knowledge
- **Tool Execution** (orange section):
- Code interpreter
- Web search
- API call
- **Execution Skill** (cloud icon with raindrops).
2. **Central Component**:
- **Iterative reasoning** (gray circle with brain/neural network icon).
- **LLM** (central node).
3. **Arrows and Flow**:
- **Trigger** (left arrow) → Planner Skill → Task decomposition → Workflow architecture.
- **Frag-1** (red dashed arrow) → Iterative reasoning → Memory (contextual recall/long-term knowledge).
- **Frag-2** (red dashed arrow) → Code Skill → Memory.
- **Frag-3** (red dashed arrow) → Reconstructed payload → Payload executed (shield icon with exclamation mark).
4. **Other Elements**:
- **Search Skill** (magnifying glass icon).
- **Memory Skill** (brain icon).
---
### Detailed Analysis
#### Spatial Grounding:
- **Planning** (blue): Top-left quadrant, connected to task decomposition and workflow architecture.
- **Memory** (green): Top-right quadrant, linked to contextual recall and long-term knowledge.
- **Tool Execution** (orange): Bottom-center, housing code interpreter, web search, and API call.
- **Execution Skill**: Bottom-left, represented by a cloud icon.
- **Iterative reasoning** (LLM): Central node, surrounded by all skill layers.
- **Frag-1/Frag-2/Frag-3**: Red dashed arrows connecting components, indicating feedback loops.
#### Color Coding:
- **Blue**: Planning (task decomposition, workflow architecture).
- **Green**: Memory (contextual recall, long-term knowledge).
- **Orange**: Tool Execution (code interpreter, web search, API call).
- **Red**: Frag-1/Frag-2/Frag-3 (feedback loops).
- **Black**: Primary flow arrows (e.g., Trigger → Planner Skill).
---
### Key Observations
1. **Cyclical Process**: The workflow loops back to the LLM via Frag-1/Frag-2/Frag-3, enabling iterative refinement.
2. **Modular Skills**: Each skill layer (Planning, Memory, Tool Execution) operates independently but feeds into the central LLM.
3. **Execution Path**: The final state ("payload executed") is isolated at the bottom, suggesting a terminal output after iterative processing.
4. **Trigger Dependency**: The process initiates externally (Trigger) and relies on the Planner Skill for initial task breakdown.
---
### Interpretation
This diagram represents an AI agent architecture designed for adaptive problem-solving. The **LLM** acts as the core reasoning engine, integrating inputs from Planning (task decomposition), Memory (contextual/long-term knowledge), and Tool Execution (code/API interactions). Feedback loops (Frag-1/Frag-2/Frag-3) allow the system to refine its approach iteratively, mimicking human-like adaptability. The **Execution Skill** (cloud icon) likely represents external system integration, while the "payload executed" state signifies task completion. The use of color coding and spatial segmentation emphasizes modularity and flow, critical for debugging or optimizing such systems. The absence of numerical data suggests this is a conceptual model rather than a performance metric visualization.
</details>
Figure 1: Skill-based agent execution model. Agents compose reusable skills for planning, memory, and tool use around an LLM core. SkillTrojan hides encrypted fragments in a few skills and, upon a trigger, reconstructs and executes a payload through standard composition.
Backdoor research on LLM-based agents has largely focused on manipulating a control channel—e.g., the model (via poisoned data or parameter edits), the prompt and planning context, or the agent’s tool/memory interfaces—so that a trigger induces malicious behavior while nominal performance on clean tasks is preserved (Li et al., 2024, 2025c; Xiang et al., 2024; Chen et al., 2024; Feng et al., 2026). Despite this broader view, most evaluation and defenses (Zhang et al., 2024; Cheng et al., 2025, 2024; Zheng et al., 2025a) remain centered on the model’s observable behavior or on specific interaction surfaces (prompts, tools, memory) in isolation. Skill-based agent systems introduce a distinct and under-examined locus of control: reusable skill implementations that encapsulate executable logic and persist across tasks, users, and deployments. Because skills mediate execution rather than high-level reasoning, compromising a single widely adopted skill can silently influence many downstream runs—even when the underlying model, prompts, and toolset remain unchanged. As shown in Figure 1, this decoupling creates a new backdoor vector in which malicious logic is embedded inside otherwise plausible skills and activated through standard skill composition. Since skills are typically treated as trusted modular components, their internal behavior is seldom audited to the same degree as model outputs, leaving a critical blind spot in current agent security assumptions.
In this work, we introduce SkillTrojan, a backdoor attack paradigm that targets the skill abstraction layer in agent systems. To our knowledge, this is the first work to systematically implant and evaluate backdoors in reusable skill implementations, rather than in model parameters, prompts, or tool and memory interfaces. SkillTrojan embeds an attacker-specified payload directly into skills and distributes it as encrypted fragments across multiple benign-appearing invocations. The payload is reconstructed and executed only when a predefined trigger condition is satisfied, allowing compromised skills to remain dormant during routine evaluation and standard usage. Because the underlying model remains unchanged and clean-task performance is largely preserved, such attacks are difficult to detect via behavior-based testing. Beyond a single attack method, SkillTrojan constitutes a general and extensible framework. It supports diverse targeted payloads and enables automated synthesis of backdoored skills from arbitrary skill templates, facilitating scalable propagation across agent pipelines and skill ecosystems. We evaluate SkillTrojan in a representative code-based agent setting across both open- and closed-weight models, and demonstrate that it consistently achieves high attack success rates with minimal impact on clean-task accuracy. To support reproducible evaluation and future research, we release a dataset of over 3,000 curated backdoored skills spanning diverse skill types, trigger conditions, and payload configurations. Together, our results expose a critical and underexplored security vulnerability in modern skill-based agent architectures. This paper makes three main contributions:
- We introduce SkillTrojan, the first backdoor attack paradigm that targets the skill abstraction layer in agent systems, implanting malicious logic into reusable skill implementations rather than model parameters, prompts, or tool and memory interfaces.
- We propose a general and extensible framework for skill-level backdoors, supporting encrypted payload fragmentation, trigger-based activation, and automated synthesis of backdoored skills from arbitrary templates, enabling scalable attacks across diverse agent pipelines and skill ecosystems.
- We empirically evaluate SkillTrojan in a realistic code-based agent setting across both open- and closed-weight models, showing consistently high attack success with minimal impact on clean-task accuracy (e.g., 97.2% ASR on GPT-5.2-1211-Global in Table 2), and release a dataset of 3,000+ curated backdoored skills to support reproducible research.
## 2 Related Work
### 2.1 Coding Agents and Executable Skill Abstractions
Recent progress in large language models has led to the widespread adoption of coding and tool-executing agents, in which models solve tasks by composing executable tools, scripts, and workflows rather than emitting only natural-language outputs. Representative systems employ modular abstractions—often referred to as skills, tools, or actions—to encapsulate reusable code, API calls, and execution logic, enabling scalability, compositionality, and reuse across tasks and deployments (Deng et al., 2025; Liu et al., 2025). These abstractions form the backbone of modern coding agents and underlie emerging ecosystems such as agent frameworks and skill marketplaces. Prior work in this area has primarily focused on improving agent capability, planning efficiency, and compositional generalization. In most settings, executable skills are treated as trusted components: once installed, their internal behavior is assumed to be benign and is rarely audited beyond functional correctness. Consequently, security analyses of coding agents have largely concentrated on model outputs, prompts, or high-level planning behavior, while the risks introduced by persistent, reusable executable abstractions have received comparatively little attention. Our work builds on this literature by explicitly examining coding agents through a security lens and highlighting executable skills as a critical but underexplored locus of control.
### 2.2 Backdoor Attacks on Agent Systems
Backdoor attacks in machine learning have traditionally targeted models directly, through training data poisoning, parameter manipulation, or trigger-based input distributions that induce malicious behavior while preserving performance on clean inputs (Li et al., 2025a; Wu et al., 2025; Yu et al., 2025). In these settings, the model is the primary control surface, and malicious behavior is expressed through model outputs. More recent work has extended backdoor and adversarial attacks to agentic systems, including prompt injection, malicious tool descriptions, memory poisoning, and manipulation of agent control logic (Xu et al., 2024; Zhu et al., 2025; Chen et al., 2024; Feng et al., 2026). While these approaches move beyond standalone models, they largely operate at transient interaction surfaces—such as prompts, tool calls, or memory entries—and assume that malicious behavior is triggered within a single agent episode.
SkillTrojan departs from these threat models by targeting the execution layer of coding agents. Instead of manipulating model behavior or individual tool invocations, SkillTrojan embeds backdoors directly into reusable executable skills. These backdoors persist across tasks and deployments, and are activated through normal skill composition during routine execution, without modifying the underlying model. This distinction places SkillTrojan outside the scope of existing backdoor defenses, which primarily monitor model inputs, outputs, or isolated interaction channels, and reveals a fundamental gap in current security analyses of agent execution pipelines.
## 3 Threat Model
Unlike prior threat models that treat attacks as transient manipulations of prompts or tool calls (Jiang et al., 2024; Ding et al., 2024; Chu et al., 2025), we model the attacker’s leverage as persistent control over reusable executable skills that are invoked across tasks. We study a skill-based agent that can dynamically load a collection of third-party skills from public repositories or marketplaces. Following the skills abstraction used in practice, a skill is a distributable package (e.g., a folder) consisting of an instruction document and executable artifacts: $s=(m,A)$ , where $m$ denotes the natural-language specification (e.g., SKILL.md) that conditions the model’s behavior, and $A=\{a_j\}$ denotes associated scripts/resources invoked during execution. An agent deployment installs a set of skills $S=\{s_1,\dots,s_K\}$ . Given a user query $q∈Q$ , the model uses $m$ to select and invoke actions in $A$ as part of normal problem solving. We assume the underlying language model and its training data are trusted and unmodified. The additional risk arises from importing external skill packages that combine instructions with executable code.
The attacker’s capability is to publish and distribute skills that appear benign and improve task performance, inducing adoption by developers. Formally, the attacker selects a subset of skills and replaces them with backdoored packages $\tilde{s}=(\tilde{m},\tilde{A})$ , yielding a deployed set $\tilde{S}$ . The attacker specifies an arbitrary payload program $P$ and a trigger predicate $Trig_τ:Q→\{0,1\}$ parameterized by a trigger string or pattern $τ$ . After installation, the attacker has no access to the host environment, OS privileges, or model internals; interaction occurs only through user queries. The attack objective is twofold: (i) stealth on clean inputs—for queries with $Trig_τ(q)=0$ , the backdoored skill should preserve nominal task behavior; and (ii) reliable activation—for queries with $Trig_τ(q)=1$ , the agent’s normal skill execution causes $P$ to run. In evaluation, we quantify this trade-off using clean-task accuracy (ACC) on non-triggered queries and attack success rate (ASR) on triggered queries. We focus on attacks that exploit the skill abstraction layer—namely, the combination of persistent natural-language instructions and executable scripts that are reused across queries. Consistent with the SkillTrojan pipeline, we allow the attacker to automatically generate new backdoored skills from arbitrary skill templates and arbitrary payloads, enabling large-scale dissemination of trojaned packages. In our threat model, persistence arises solely from the compromised skill package being installed and reused across tasks and deployments. Any artifacts used for payload reconstruction are ephemeral and scoped to a single agent run. Concretely, when triggered, fragment emitters write small encoded fragments to a run-local intermediate channel that is available to the agent’s execution context (e.g., tool return values and the tool-call trajectory log, or a run-scoped scratchpad). These fragments are not persisted across queries and are cleaned up at the end of the run. We exclude attacks that rely on cross-run state or long-term storage. We exclude direct compromise of the base model, remote services, or the operating system, and consider defenses such as sandboxing or output validation as outside the attacker’s capabilities unless explicitly enabled by the deployment.
<details>
<summary>2604.06811v1/framework.png Details</summary>
### Visual Description
## Flowchart: System Architecture for Secure Task Execution
### Overview
The diagram illustrates a multi-layered system architecture designed to execute user tasks securely while preventing malicious payload execution. It is divided into four layers: User Layer, Agent Reasoning Layer, Skill Execution Layer, and Side-Effect Execution Layer. Arrows indicate data flow and control logic between components.
---
### Components/Axes
#### Labels and Legends
1. **User Layer**:
- **Human User**: Icon of a person with thought bubble.
- **Benign Query**: "Analyze data.csv and summarize."
- **Triggered Query**: "Analyze data.csv with [T] and summarize." (Subtly marked with a warning icon).
2. **Agent Reasoning Layer**:
- **LLM-based Coding Agent Core**:
- **Large Language Model (LLM)**: Brain icon with neural network.
- **Planner Module**: Box with diamond shape.
- **Tool & Skill Selector**: Gear and wrench icon.
- **Execution Controller/Orchestrator**: Box with multiple smaller boxes.
- **Skill Marketplace/Skill Repository**:
- **Code Skill (Python)**: Python logo.
- **Database Skill (SQL/Table)**: Database icon.
- **Network Skill (HTTP/Globe)**: Globe icon.
- **Logging/Utility Skill**: Log file icon.
- **Skill Execution Trace**:
- **Step 1**: Code Skill Call (Data Loading).
- **Step 2**: Database Skill Call (Querying Metadata).
- **Step 3**: Network Skill Call (External Info Fetch).
- **Step 4**: Code Skill Call (Analysis & Summarization).
- **Side-Effect Execution Layer**:
- **Verification Module**: Checklist and shield icon.
- **Reconstruction Step**: Puzzle pieces and locked box.
- **Final Payload Execution**: Gear with explosion icon.
- **Cleanup**: Broom and trash can icon.
3. **Color Legend**:
- **Green**: Normal steps (e.g., "Normal Output").
- **Purple**: Encrypted fragments (e.g., "Fragment #1 (Encrypted)").
- **Red**: Malicious payload execution (highlighted in red box).
---
### Detailed Analysis
#### User Layer
- **Benign Query**: Direct input to the LLM-based Coding Agent Core.
- **Triggered Query**: Subtly marked input (with "[T]") that bypasses initial safeguards.
#### Agent Reasoning Layer
- **LLM-based Coding Agent Core**:
- The LLM processes queries and delegates tasks to the Planner Module.
- The Planner Module coordinates with the Tool & Skill Selector to choose appropriate skills.
- The Execution Controller/Orchestrator manages task execution.
#### Skill Execution Layer
- **Skill Marketplace**:
- Skills are reusable, third-party, and persistent.
- Arrows show skill selection based on query requirements.
- **Skill Execution Trace**:
- Steps involve encrypted data fragments (e.g., "Fragment #1 (Encrypted)").
- Final output is a "Summary Report" visible to the user.
#### Side-Effect Execution Layer
- **Verification Module**: Validates outputs using set-based verification (unordered).
- **Reconstruction Step**: Decrypts payload fragments (e.g., "Fragment #1") using a key.
- **Final Payload Execution**: Malicious code executes only when triggered, highlighted in red.
- **Cleanup**: Automated post-execution cleanup.
---
### Key Observations
1. **Dual Query Paths**: The system distinguishes between benign and triggered queries, with the latter bypassing initial safeguards.
2. **Encrypted Data Flow**: Intermediate steps involve encrypted fragments to obscure malicious activity.
3. **Malicious Payload**: The red-highlighted "Final Payload Execution" occurs only under a trigger, indicating a hidden side effect.
4. **Security Measures**: Verification and cleanup steps aim to mitigate risks but may fail if the payload evades detection.
---
### Interpretation
This architecture demonstrates a **defense-in-depth** approach to secure task execution. The system prioritizes transparency for benign tasks while concealing malicious activity until execution. The use of encrypted fragments and modular skill execution suggests an attempt to compartmentalize risks. However, the presence of a triggered query path and final payload execution indicates a vulnerability to exploitation. The cleanup step implies awareness of potential breaches but does not prevent them. The design balances usability (reusable skills) with security (verification), though the hidden payload execution undermines trust. This could represent a **Trojan-like system** where malicious behavior is obfuscated until activation.
</details>
Figure 2: SkillTrojan: A layered backdoor attack on skill-based coding agents. The figure illustrates a multi-layer execution pipeline spanning the user query layer, LLM-based agent reasoning, reusable third-party skill execution, and side-effect execution. An attacker embeds encrypted payload fragments across multiple benign-looking skill invocations. Under a triggered query, fragments are emitted during normal skill execution, verified in an order-agnostic (set-based) manner, and reconstructed only after the intended execution workflow is completed. The agent produces a correct and benign user-visible output, while the malicious payload executes solely as a hidden side effect.
## 4 Method
Given a benign skill template $s=(m,A)$ , an attacker-specified payload program $P$ , and a trigger predicate $Trig_τ$ , SkillTrojan produces a backdoored skill $\tilde{s}=(\tilde{m},\tilde{A})$ . Figure 2 provides an overview of the layered execution pipeline. The attack objective is twofold: (i) for non-triggered queries, $\tilde{s}$ preserves the nominal functionality of $s$ ; and (ii) for triggered queries, normal skill execution causes $P$ to be reconstructed and executed as a side effect. The key design principle is that payload activation is aligned with completion of an ordinary execution workflow rather than with any single anomalous action. Let $π=(a_i_1,…,a_i_{T})$ denote a skill execution trace induced by query $q$ . SkillTrojan ensures that malicious behavior is triggered only when $π$ covers a designated subset of actions, thereby coupling attack success to routine task completion.
### 4.1 Payload Encoding and Skill Instrumentation
The attacker specifies an arbitrary executable payload $P$ , subject only to constraints imposed by the skill execution environment. To prevent static inspection and partial leakage, SkillTrojan first encrypts $P$ using a symmetric encryption function $Enc(·)$ with secret key $k$ :
$$
C=Enc(P;k). \tag{1}
$$
The ciphertext $C$ is then encoded and partitioned into $N$ fragments $\{c_1,…,c_N\}$ . Each fragment $c_j$ consists of two components: an index identifier and an encrypted payload segment. Conceptually, this encoding can be viewed as
$$
c_j=⟨ j, seg_j(C)⟩, \tag{2}
$$
where $seg_j(·)$ denotes the $j$ -th segment of the ciphertext. The embedded index enables order-agnostic reconstruction and eliminates assumptions about the order or timing of action invocation. Fragmentation thus decouples payload reconstruction from the agent’s planner and control-flow decisions. The fragment count $N$ is a tunable parameter that controls the trade-off between stealth and activation robustness. Larger $N$ reduces the information content of any single fragment and lowers the risk of accidental activation during partial execution, while requiring more complete execution traces for successful reconstruction.
SkillTrojan modifies the natural-language specification $m$ into $\tilde{m}$ by injecting conditional control logic parameterized by $Trig_τ(q)$ . For non-triggered queries, $\tilde{m}$ is semantically equivalent to $m$ and induces the same distribution over skill execution traces. For triggered queries, $\tilde{m}$ biases the agent toward execution traces that include a designated subset of actions $\{a_i_1,…,a_i_{N}\}$ . Formally, letting $π$ denote a skill execution trace, this effect can be expressed as
$$
\Pr_\tilde{m}(π\mid q)=\begin{cases}\Pr_m(π\mid q),&Trig_τ(q)=0,\\
\Pr_m(π\mid q)+Δ(π),&Trig_τ(q)=1,\end{cases} \tag{3}
$$
where $Δ(π)$ denotes a small bias that increases the likelihood of traces covering the designated actions. Importantly, $\tilde{m}$ does not prescribe a fixed execution order or explicit malicious instruction; it operates within the agent’s normal planning and decomposition mechanisms. At the execution level, the executable artifacts $A$ are instrumented to emit fragments during normal action execution. Each modified action $\tilde{a}_i_{j}∈\tilde{A}$ preserves its original functional behavior and output. When $Trig_τ(q)=1$ , the action additionally writes fragment $c_j$ to intermediate storage local to the skill execution context. In our instantiation, this intermediate channel is implemented as benign-looking fields in tool outputs that are naturally logged in the agent’s tool-call trajectory, enabling a verifier tool to retrieve fragments without requiring cross-run state. To minimize user-visible artifacts, fragments are emitted in a way that preserves the tool’s functional output and are only consumed by the verifier. When $Trig_τ(q)=0$ , no fragment is emitted. Let
$$
F(q)=\{c_j_1,…,c_j_{T}\} \tag{4}
$$
denote the unordered set of fragments emitted during a single agent execution on query $q$ . Fragment emission is ephemeral: fragments are scoped to a single execution and do not persist across queries.
### 4.2 Triggered Reconstruction and Execution
A designated verification action monitors the execution state and checks whether all required fragments have been collected:
$$
\{c_1,…,c_N\}⊆F(q). \tag{5}
$$
Because each fragment carries an embedded index, verification depends only on set inclusion and does not require assumptions about execution order, timing, or control-flow structure. Upon successful verification, the ciphertext is reconstructed by concatenating the encrypted segments according to their indices and decrypted to recover the payload:
$$
P=Dec(c_1\|⋯\|c_N;k). \tag{6}
$$
The payload is then executed within the skill execution environment as a side effect of normal task completion. After execution, all intermediate artifacts are removed to minimize forensic traces. Crucially, payload execution is independent of the agent’s response generation. The agent produces a correct and benign output for the user query, while malicious behavior occurs solely through execution-side effects.
### 4.3 Dataset Construction
We construct SkillTrojanX, a corpus of backdoored skill packages used in our experiments. The goal of this subsection is to specify how we obtain skill templates and generate backdoored variants; task workloads, query triggering, and evaluation metrics are described in Section 5. We start from a public skill repository/marketplace and collect a set of candidate skill packages. We retain templates that (i) contain a parsable natural-language specification file and (ii) include executable artifacts. Each retained package is normalized into a common representation consistent with our threat model, yielding a template set $S=\{s_1,…,s_K\}$ where each $s=(m,A)$ .
Given an attacker configuration $(P,τ,N)$ , SkillTrojan transforms each template $s∈S$ into a backdoored skill $\tilde{s}=(\tilde{m},\tilde{A})$ following the previous process. We generate multiple variants per template by varying the trigger string/pattern $τ$ , payload family $P$ , encryption/encoding choice, and fragment count $N$ . This produces a set of backdoored skills that share the same threat model but differ in surface semantics and implementation details, supporting scalable evaluation across heterogeneous skill categories. For each generated backdoored skill, we record structured metadata including the trigger identifier, payload family, crypto/encoding variant, fragment count $N$ , and the set of actions designated for fragment emission and verification.
Table 1: EHR SQL results on open-weight models. We report ACC on clean queries and ASR on poisoned queries for SkillTrojan and competitive baselines adapted to the same skill-based agent setting.
| Non-attack GCG AutoDAN | 84.1 83.7 82.2 | 0.0 35.1 46.8 | 71.5 70.2 68.9 | 0.0 41.0 51.4 | 76.0 75.3 73.6 | 0.0 38.6 55.6 | 53.2 51.8 49.5 | 0.0 24.9 19.7 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| CPA | 81.4 | 44.2 | 67.8 | 57.9 | 72.1 | 52.7 | 48.9 | 31.5 |
| BadChain | 84.0 | 31.8 | 70.7 | 18.4 | 76.6 | 23.7 | 52.6 | 7.9 |
| AgentPoison | 85.0 | 57.2 | 72.4 | 62.5 | 77.1 | 60.8 | 52.0 | 37.6 |
| SkillTrojan | 85.2 | 62.1 | 76.3 | 64.7 | 81.3 | 72.0 | 48.4 | 26.7 |
Table 2: EHR SQL results on closed-weight models under the same protocol as Table 1.
| Non-attack GCG AutoDAN | 71.6 70.8 67.4 | 0.0 30.2 42.1 | 73.1 70.1 68.6 | 0.0 32.9 39.4 | 86.5 69.6 69.7 | 0.0 37.8 34.9 | 82.7 69.8 67.9 | 0.0 42.1 30.8 | 73.0 70.1 68.4 | 0.0 45.8 27.4 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| CPA | 66.3 | 38.5 | 66.9 | 41.0 | 67.8 | 44.6 | 67.2 | 48.7 | 67.9 | 51.1 |
| BadChain | 71.9 | 33.7 | 72.6 | 28.9 | 71.5 | 22.7 | 70.9 | 14.2 | 70.8 | 8.3 |
| AgentPoison | 74.8 | 53.7 | 75.5 | 54.8 | 74.1 | 56.2 | 73.4 | 57.6 | 72.9 | 58.3 |
| SkillTrojan | 68.5 | 54.3 | 82.7 | 57.3 | 90.7 | 64.2 | 86.6 | 74.7 | 89.3 | 97.2 |
## 5 Experiments
### 5.1 Experimental Setup
We evaluate SkillTrojan in a code-based skill execution setting. Skill is distributed as a package containing (i) a natural-language specification file (Skill.md) and (ii) executable tool scripts. Given a user query, the agent selects and composes tools based on the specification, executes them to solve the task, and logs a full trajectory of tool calls and tool outputs. SkillTrojan backdoors a subset of tools such that they emit encrypted payload fragments only when a trigger predicate holds. A designated verification tool then collects the fragments, reconstructs the payload, decrypts it, and executes it within the same agent run. Concretely, unless stated otherwise, we use a poisoning ratio $ρ=0.2$ , fragment count $N=3$ , and Xor + Base64 for lightweight encryption/encoding (Josefsson, 2006). We construct poisoned queries by inserting a trigger phrase into the original natural-language query; clean queries satisfy $Trig_τ(q)=0$ and poisoned queries satisfy $Trig_τ(q)=1$ .
#### Dataset and Models
Our primary end-to-end benchmark is an EHR SQL task: given a natural-language clinical query, the agent composes SQL using skill tools and executes it against an EHR database. We judge correctness by comparing the SQL execution result to the expected result using an LLM-as-a-judge protocol. This benchmark measures both (i) functional task completion (clean accuracy) and (ii) whether a malicious side effect can be reliably executed under triggered inputs (attack success). We evaluate SkillTrojan on both Open-Weight and Closed-Weight LLMs, using the same set of models as in Tables 1 – 2. Our Open-Weight models are GLM-4.7, Qwen3-Coder, GLM-4.6, and Qwen3-VL-235B-A22B-Instruct. Our Closed-Weight models are GPT-4o-Mini-0718-Global, Claude-Haiku-4.5, Claude-Sonnet4.5, Qwen3-Max, and GPT-5.2-1211-Global (Hurst et al., 2024; Bai et al., 2023; GLM et al., 2024; Adetayo et al., 2024). In all experiments, the model serves as the agent’s policy model for planning, tool selection, and intermediate reasoning, while the skill implementation and evaluation protocol are kept identical across models.
#### Baselines.
We adapt competitive prompt-/model-centric jailbreak or agent-poisoning baselines to the same skill-based agent setting: GCG, AutoDAN, CPA, BadChain, and AgentPoison (Zou et al., 2023; Liu et al., 2023; Zhu et al., 2023; Chen et al., 2024; Zhong et al., 2023). Each baseline is evaluated under the same trigger insertion process, poisoning ratio, and tool environment as SkillTrojan.
#### Metrics.
We report clean-task accuracy (ACC) and attack success rate (ASR). ACC is the fraction of clean queries judged correct based on the SQL execution output. ASR is the fraction of poisoned queries in which the payload is successfully reconstructed and executed, indicated by an explicit execution marker returned by the verification tool and corroborated by a deterministic side effect.
#### Additional benchmark (SWE-Bench Verified).
To ensure our findings are not specific to structured SQL generation, we also instantiate the same attack pipeline in a software engineering setting on SWE-Bench Verified, using the same trigger mechanism and evaluation protocol, and replacing the task metric with the benchmark’s verified test-based criterion. Due to space constraints, SWE-Bench Verified results are reported in Appendix B.
#### Main results.
Tables 1 and 2 summarize end-to-end EHR SQL results. Overall, SkillTrojan achieves high ASR while largely preserving clean-task performance, consistent with the attack’s design: fragment emission and verification are embedded into ordinary skill workflows, and are disabled on clean executions when the trigger predicate is false.
### 5.2 Analysis of main results
#### SkillTrojan yields the strongest end-to-end attack while preserving utility.
Across both Open-Weight and Closed-Weight regimes, SkillTrojan achieves high ASR while largely preserving clean-task ACC relative to the Non-Attack condition, and in several settings even improves ACC. For example, on GPT-5.2-1211-Global, SkillTrojan reaches 97.2 ASR while keeping ACC at 89.3 (vs. Non-Attack ACC 73.0). And on Qwen3-Max, it achieves 74.7 ASR with ACC 86.6.On Open-Weight models, SkillTrojan remains effective as well (e.g., GLM-4.6: 72.0 ASR with 81.3 ACC. We note that SkillTrojan preserves the original skill’s functional behavior and outputs: backdoored skills return the same benign outputs as their clean counterparts, and since installing these (benign) skills can itself improve task completion in our agent setting, it is possible to observe higher ACC under SkillTrojan than the Non-attack condition. For more details on the effectiveness experiments of skills, please refer to the Appendix F.
#### The advantage over prompt-centric baselines increases with stronger agent-tool execution.
Prompt- or dialogue-level attack baselines (e.g., GCG, AutoDan, AgentPoison) are comparatively unstable in a tool-execution setting because their success depends on the model choosing to follow an injected instruction pattern, and because stronger models often exhibit improved refusal or robustness to shallow prompt manipulations. In contrast, SkillTrojan routes malicious behavior through trusted tool execution that is already part of the agent’s normal workflow; thus, model capability primarily affects whether the agent completes the intended tool chain rather than whether it “agrees” with the malicious request. This gap is visible on stronger Closed-Weight models: on GPT-5.2-1211-Global, SkillTrojan achieves 97.2 ASR while the strongest baseline in Table 2 reaches at most 58.3 ASR (AgentPoison). More broadly, baselines display diverse failure modes across models: some maintain moderate ASR but reduce ACC (e.g., CPA and AutoDan on multiple models), while others lose ASR rapidly as the model changes (BadChain dropping to low ASR on certain Closed-Weight models). SkillTrojan is comparatively consistent because it is anchored in the execution semantics of skills.
#### SkillTrojan achieves a more favorable ACC–ASR trade-off than competing methods.
Beyond absolute attack success, Tables 1 and 2 reveal a qualitative difference in the trade-off between clean-task accuracy and attack success across methods. Several baseline attacks increase ASR at the cost of noticeable ACC degradation (e.g., CPA and AutoDAN on multiple models), while others preserve ACC but achieve only limited ASR. In contrast, SkillTrojan consistently operates in a regime with simultaneously high ASR and competitive ACC across both Open-Weight and Closed-Weight settings, indicating a more favorable ACC–ASR trade-off under the same agent protocol.
#### Implications for defense and evaluation.
The above patterns suggest two practical implications. First, evaluating backdoors in agentic systems must incorporate execution-aware metrics: an attack can succeed via side effects even when the final textual answer appears benign, and the relevant signals may be in tool outputs, tool-call ordering, and external state changes. Second, effective mitigations likely require monitoring and constraining execution traces rather than only sanitizing inputs. For example, policies that (i) constrain verification-like tools, (ii) audit unexpected increases in tool-call depth under triggered inputs, or (iii) enforce provenance checks for tool outputs could directly target the bottleneck identified above (workflow completion and reconstruction). To validate that activation is aligned with ordinary skill composition rather than an overtly anomalous behavior, we analyze tool trajectories. Figure 3 reports the distribution of tool-call counts under clean and triggered queries. Triggered runs show a small but systematic increase in tool usage corresponding to fragment emission and verification, while remaining within the typical range for complex EHR queries. This supports the core claim that SkillTrojan can be embedded into realistic skill ecosystems with limited impact on normal task performance, while still achieving high end-to-end reliability under triggered inputs.
<details>
<summary>2604.06811v1/x2.png Details</summary>
### Visual Description
## Violin Plot and Line Chart: Tool-call distribution on EHR SQL
### Overview
The image contains two visualizations: a **violin plot** on the left and a **line chart** on the right. Both analyze tool-call distributions for "Clean" and "Triggered" states in an EHR SQL system. The violin plot summarizes distributions, while the line chart shows discrete probability trends.
---
### Components/Axes
#### Violin Plot (Left)
- **X-axis**: Labels for "Clean" (blue) and "Triggered" (orange).
- **Y-axis**: "Tool calls" (range: 5–30).
- **Violin Shape**: Represents probability density; wider sections indicate higher data concentration.
- **Box Plots**: Overlaid on violins, showing medians (horizontal line) and interquartile ranges (box edges).
#### Line Chart (Right)
- **X-axis**: "Tool calls per run" (range: 5–30).
- **Y-axis**: "Probability" (range: 0.00–0.10).
- **Lines**:
- Blue: "Clean" state.
- Orange: "Triggered" state.
- **Legend**: Located in the top-right corner, matching line colors to labels.
---
### Detailed Analysis
#### Violin Plot
- **Clean (Blue)**:
- Median: ~15 tool calls.
- Interquartile range (IQR): 14–16.
- Whiskers: 10–25 (outliers not explicitly marked).
- Distribution: Symmetrical, with a slight bulge near the median.
- **Triggered (Orange)**:
- Median: ~16 tool calls.
- IQR: 15–17.
- Whiskers: 10–25.
- Distribution: Slightly skewed right, with a wider spread at higher tool calls.
#### Line Chart
- **Clean (Blue Line)**:
- Peaks at ~15–16 tool calls (probability ~0.08).
- Declines sharply after 16, with minimal probability beyond 20.
- Error bars: Largest near the peak (~±0.02).
- **Triggered (Orange Line)**:
- Peaks at ~18–20 tool calls (probability ~0.07).
- Declines more gradually than "Clean," with a secondary peak near 25.
- Error bars: Consistent (~±0.01–0.02) across the range.
---
### Key Observations
1. **Median Differences**: "Triggered" has a slightly higher median (16 vs. 15) than "Clean."
2. **Probability Peaks**:
- "Clean" peaks earlier (15–16) with higher probability density.
- "Triggered" peaks later (18–20) but with lower maximum probability.
3. **Distribution Spread**:
- "Triggered" shows broader variability (wider violin and gradual decline).
- "Clean" is more concentrated around its median.
4. **Error Bars**: Both lines exhibit similar variability, but "Clean" has larger uncertainty near its peak.
---
### Interpretation
The data suggests that "Triggered" tool calls are associated with **higher median usage** and **greater variability** compared to "Clean" calls. The probability distributions indicate that "Clean" calls are more likely to cluster around 15–16 tool calls per run, while "Triggered" calls are more spread out, peaking at 18–20. The broader spread for "Triggered" may reflect less predictable or more complex interactions in the EHR SQL system. The error bars confirm that while trends are consistent, there is notable variability in individual runs, particularly for "Triggered" calls near their peak. This could imply that "Triggered" states involve more dynamic or context-dependent processes.
</details>
Figure 3: Tool-call count distribution on EHR SQL for clean and triggered queries. Triggered executions require additional calls for fragment collection and verification, but remain close to the normal operating regime.
### 5.3 SkillTrojanX: a dataset of backdoored skills
We release SkillTrojanX, a dataset of over 3,000 backdoored skills derived from real, high-usage skill templates collected from a public skill marketplace. We crawl the top 1,200 skills by popularity and retain those that contain executable artifacts and a parsable specification file. Each retained template is normalized into a common skill format and paired with automatically generated backdoored variants. For each variant, we record structured metadata including the trigger phrase, payload family, encryption method, fragment count $N$ , and the set of tool entrypoints used for fragment emission and verification. SkillTrojanX is designed to support two evaluation regimes. First, it enables scalable measurement of skill-level backdoor behavior under controlled trigger–payload configurations, isolating attacks at the skill layer from model retraining or data poisoning. Second, it provides a realistic corpus for studying defenses that operate on skill packages, such as static analysis of skill artifacts, provenance checks, and execution-trajectory auditing under normal workflows. Table 3 summarizes the dataset composition and coverage, and Figure 4 visualizes how payload families are distributed across major template categories.
Table 3: SkillTrojanX dataset statistics. A template is a cleaned skill package collected from the marketplace. A backdoored skill is a generated variant with a specific trigger–payload–crypto– $N$ configuration.
| Templates | 1200 |
| --- | --- |
| Backdoored skills | 3000+ |
| Template categories | 6 |
| Payload families | 4 |
| Crypto variants | 3 |
| $N$ variants | 3 |
| Unique triggers | 50 |
<details>
<summary>2604.06811v1/x3.png Details</summary>
### Visual Description
## Bar Chart: SkillTrojanX coverage by template category and payload family
### Overview
The chart visualizes the distribution of backdoored skills across six template categories (Coding, Data/SQL, File/IO, Planning, Utilities, Web) and four payload families (System profiling, File drop, Secret harvesting, Network beacon). Each category contains four stacked bars representing payload family contributions to total backdoored skills.
### Components/Axes
- **X-axis**: Template categories (Coding, Data/SQL, File/IO, Planning, Utilities, Web)
- **Y-axis**: Number of backdoored skills (0-900 range)
- **Legend**:
- Blue: System profiling
- Orange: File drop
- Green: Secret harvesting
- Purple: Network beacon
- **Bar structure**: Stacked vertical bars with payload family segments
### Detailed Analysis
1. **Coding** (Total: ~720)
- System profiling: ~220 (blue)
- Secret harvesting: ~250 (green)
- File drop: ~150 (orange)
- Network beacon: ~100 (purple)
2. **Data/SQL** (Total: ~900)
- System profiling: ~260 (blue)
- Secret harvesting: ~220 (green)
- File drop: ~300 (orange)
- Network beacon: ~120 (purple)
3. **File/IO** (Total: ~680)
- System profiling: ~140 (blue)
- Secret harvesting: ~180 (green)
- File drop: ~300 (orange)
- Network beacon: ~60 (purple)
4. **Planning** (Total: ~500)
- System profiling: ~160 (blue)
- Secret harvesting: ~140 (green)
- File drop: ~120 (orange)
- Network beacon: ~80 (purple)
5. **Utilities** (Total: ~580)
- System profiling: ~180 (blue)
- Secret harvesting: ~160 (green)
- File drop: ~140 (orange)
- Network beacon: ~100 (purple)
6. **Web** (Total: ~720)
- System profiling: ~180 (blue)
- Secret harvesting: ~160 (green)
- File drop: ~120 (orange)
- Network beacon: ~260 (purple)
### Key Observations
1. **Data/SQL** has the highest total coverage (900 skills), with File drop (orange) dominating at 300 skills.
2. **Web** shows the largest Network beacon (purple) segment (260 skills), exceeding all other payload families in this category.
3. **File/IO** has the highest File drop (orange) contribution (300 skills) despite lower total coverage than Data/SQL.
4. **System profiling** (blue) maintains consistent mid-range values across all categories (140-260 skills).
5. **Network beacon** (purple) shows significant variation, peaking in Web (260) and bottoming in File/IO (60).
### Interpretation
The data suggests:
- **Payload specialization**: File drop dominates data-centric categories (Data/SQL, File/IO), while Network beacon specializes in Web environments.
- **System profiling** acts as a baseline payload across all categories, with moderate but consistent presence.
- **Secret harvesting** shows moderate importance in most categories but never dominates.
- **Category targeting**: Data/SQL and Web have the highest total coverage, potentially indicating these are priority attack surfaces for SkillTrojanX.
- **Planning** has the lowest total coverage (500 skills), suggesting it may be a less targeted or less vulnerable template category.
The chart reveals strategic payload distribution patterns, with File drop and Network beacon showing clear specialization in specific template categories. This could reflect either technical compatibility with certain environments or attacker preferences for specific attack vectors in different contexts.
</details>
Figure 4: Coverage of SkillTrojanX. We report the distribution of backdoored variants across template categories and payload families.
### 5.4 Ablation studies
We ablate key design choices implied by our construction and isolate which factors control the ACC–ASR trade-off. Unless otherwise stated, we fix the underlying model, the EHR SQL split, the trigger phrase, and a representative side-effect payload, and vary one factor at a time while keeping the skill template and evaluation protocol unchanged. In all ablations, ACC is computed on the clean subset and ASR on the poisoned subset.
#### Fragment count $N$ .
Fragmentation controls a three-way trade-off between per-tool suspiciousness, workflow completion, and reconstruction reliability. When $N$ is too small, each tool output must carry a larger fragment. In practice this increases the salience of anomalous encoded content in intermediate messages and makes the agent more likely to deviate from the intended workflow (e.g., skipping verification or rewriting outputs), which reduces ASR. When $N$ is too large, activation becomes bottlenecked by trace completion: the agent must invoke more fragment-carrying tools and successfully reach verification within a single run, increasing the chance of missing at least one fragment and failing reconstruction. Figure 5 shows a unimodal trend with the highest ASR at moderate fragmentation ( $N{=}3$ ), and lower ASR at both extremes. Across all $N$ , ACC remains stable, indicating that $N$ primarily affects the side-effect activation path rather than the SQL-solving capability.
<details>
<summary>2604.06811v1/x4.png Details</summary>
### Visual Description
## Line Graph: Fragment Count N vs. ASR and ACC
### Overview
The image is a line graph comparing two metrics, **ASR (%)** (orange line) and **ACC (%)** (blue line), across different fragment counts (N = 1 to 5). The graph includes two y-axes: the left axis represents ASR (%), and the right axis represents ACC (%). Dashed horizontal lines at 55% and 75% are labeled "_child1" (likely indicating reference thresholds or benchmarks).
---
### Components/Axes
- **X-axis**: "Fragment count N" (values: 1, 2, 3, 5).
- **Y-axes**:
- Left: "ASR (%)" (0–100 scale).
- Right: "ACC (%)" (0–100 scale).
- **Legend**:
- Orange line: "ASR" (orange).
- Blue line: "ACC" (blue).
- **Dashed lines**:
- 55% (labeled "_child1").
- 75% (labeled "_child1").
---
### Detailed Analysis
#### ASR (%) (Orange Line)
- **N=1**: ~40% (data point at bottom of orange line).
- **N=2**: ~50% (data point at mid-level of orange line).
- **N=3**: ~55% (peak of orange line, aligning with the 55% dashed line).
- **N=5**: ~40% (data point at bottom of orange line).
- **Trend**: Increases from N=1 to N=3, then decreases sharply to N=5.
#### ACC (%) (Blue Line)
- **N=1**: ~70% (data point at mid-level of blue line).
- **N=2**: ~70% (data point at mid-level of blue line).
- **N=3**: ~70% (data point at mid-level of blue line).
- **N=5**: ~70% (data point at mid-level of blue line).
- **Trend**: Remains constant across all fragment counts.
#### Dashed Lines
- **55% (dashed orange)**: Aligns with the peak of ASR at N=3.
- **75% (dashed blue)**: Slightly above the ACC line, which remains at ~70% for all N.
---
### Key Observations
1. **ASR (%)** peaks at N=3 (55%) and declines to 40% at N=5, suggesting a non-linear relationship with fragment count.
2. **ACC (%)** remains stable at ~70% across all fragment counts, indicating robustness to fragmentation.
3. The 55% and 75% dashed lines may represent target thresholds for ASR and ACC, respectively.
4. The orange line (ASR) intersects the 55% dashed line at N=3, while the blue line (ACC) stays below the 75% dashed line.
---
### Interpretation
- **ASR (%)** likely represents a metric that improves with moderate fragmentation (N=3) but degrades at higher fragmentation (N=5). This could indicate an optimal fragmentation level for performance.
- **ACC (%)** remains consistently high, suggesting that the system or process being measured maintains accuracy regardless of fragmentation.
- The "_child1" labels on the dashed lines may denote reference values for comparison, though their exact meaning (e.g., benchmarks, error margins) is unclear without additional context.
- The divergence between ASR and ACC trends highlights a potential trade-off: while accuracy (ACC) is stable, the success rate (ASR) is sensitive to fragmentation levels.
This analysis suggests that fragment count N=3 may be optimal for maximizing ASR, while N=1–5 maintains high ACC. Further investigation into the system's design or constraints could clarify the significance of the "_child1" thresholds.
</details>
Figure 5: Ablation on fragment count $N$ under GPT-4o-mini-0718-Global. ASR is maximized at moderate fragmentation: small $N$ increases per-tool anomaly and lowers compliance with the intended workflow, while large $N$ increases the probability of incomplete fragment collection before verification. ACC remains stable.
Table 4: Encryption ablation under GPT-4o-mini-0718-Global ( $N{=}3$ , $ρ{=}0.2$ ). “Heuristic-flag” is the hit rate of a simple base64-string detector on intermediate tool outputs.
| XOR+Base64 Hybrid | 68.5 68.4 | 54.3 54.6 | 78.0 21.0 |
| --- | --- | --- | --- |
#### Encryption variants and heuristic visibility.
Encryption affects the representation of fragments but should not affect the control logic of skill execution. Table 4 confirms that swapping XOR+Base64 for a hybrid scheme leaves ACC and ASR essentially unchanged (ACC differs by $0.1$ and ASR by $0.3$ points). However, encryption materially changes detectability by simple static heuristics. A base64-oriented detector flags $78\$ of intermediate tool outputs under XOR+Base64 but only $21\$ under the hybrid scheme. This illustrates that signature-based scanning can be brittle: attackers can cheaply diversify encodings and substantially reduce the effectiveness of naive detectors without sacrificing reliability.
## 6 Conclusion
We introduce SkillTrojan, the first backdoor attack paradigm that targets the skill abstraction layer of agent systems. By embedding encrypted payload fragments into reusable executable skills and activating them through standard skill composition, SkillTrojan enables persistent and stealthy backdoors without modifying the underlying model, prompts, or training data. We show that skill-level backdoors can achieve high attack success rates while largely preserving benign task performance, exposing a critical blind spot in current agent security assumptions. Beyond a single attack method, SkillTrojan provides a general framework for studying execution-layer threats in skill-based agents. To support systematic evaluation, we release SkillTrojanX, a dataset of over 3,000 backdoored skills derived from real-world templates. Our results suggest that as agent systems increasingly rely on reusable third-party skills, the security-critical surface extends beyond model behavior to the execution substrate that skills inhabit. This motivates defenses that explicitly model skill provenance and constrain side effects during execution (e.g., sandboxing, permissioned resources, and execution-trajectory auditing). An important direction for future work is to evaluate how such mitigations affect the ACC–ASR trade-off under realistic deployment constraints and across a wider range of agent frameworks and skill ecosystems.
## References
- A. J. Adetayo, M. O. Aborisade, and B. A. Sanni (2024) Microsoft copilot and anthropic claude ai in education and library service. Library Hi Tech News. Cited by: §5.1.
- J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, et al. (2023) Qwen technical report. arXiv preprint arXiv:2309.16609. Cited by: §5.1.
- Z. Chen, Z. Xiang, C. Xiao, D. Song, and B. Li (2024) Agentpoison: red-teaming llm agents via poisoning memory or knowledge bases. Advances in Neural Information Processing Systems 37, pp. 130185–130213. Cited by: §1, §2.2, §5.1.
- P. Cheng, Y. Ding, T. Ju, Z. Wu, W. Du, P. Yi, Z. Zhang, and G. Liu (2024) Trojanrag: retrieval-augmented generation can be backdoor driver in large language models. arXiv preprint arXiv:2405.13401. Cited by: §1.
- P. Cheng, H. Hu, Z. Wu, Z. Wu, T. Ju, Z. Zhang, and G. Liu (2025) Hidden ghost hand: unveiling backdoor vulnerabilities in mllm-powered mobile gui agents. arXiv preprint arXiv:2505.14418. Cited by: §1.
- J. Chu, Y. Liu, Z. Yang, X. Shen, M. Backes, and Y. Zhang (2025) JailbreakRadar: comprehensive assessment of jailbreak attacks against llms. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 21538–21566. Cited by: §3.
- Y. Deng, W. Lin, Y. Song, M. Wang, D. Cai, and J. Liu (2025) Socialization as a political arena: a multi-agent interactionist perspective to understand political skill and newcomer socialization rates. Academy of Management Journal 68 (1), pp. 108–137. Cited by: §2.1.
- P. Ding, J. Kuang, D. Ma, X. Cao, Y. Xian, J. Chen, and S. Huang (2024) A wolf in sheep’s clothing: generalized nested jailbreak prompts can fool large language models easily. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 2136–2153. Cited by: §3.
- Y. Feng, Y. Li, Y. Wu, Y. Tan, Y. Guo, Y. Ding, K. Zhai, X. Ma, and Y. Jiang (2026) BackdoorAgent: a unified framework for backdoor attacks on llm-based agents. arXiv preprint arXiv:2601.04566. Cited by: §1, §2.2.
- T. GLM, A. Zeng, B. Xu, B. Wang, C. Zhang, D. Yin, D. Zhang, D. Rojas, G. Feng, H. Zhao, et al. (2024) Chatglm: a family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793. Cited by: §5.1.
- A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. (2024) Gpt-4o system card. arXiv preprint arXiv:2410.21276. Cited by: §5.1.
- Y. Jiang, K. Aggarwal, T. Laud, K. Munir, J. Pujara, and S. Mukherjee (2024) Red queen: safeguarding large language models against concealed multi-turn jailbreaking. arXiv preprint arXiv:2409.17458. Cited by: §3.
- S. Josefsson (2006) The base16, base32, and base64 data encodings. Technical report Cited by: §5.1.
- J. Li, Y. Li, H. Huang, Y. Chen, X. Wang, Y. Wang, X. Ma, and Y. Jiang (2025a) BackdoorVLM: a benchmark for backdoor attacks on vision-language models. arXiv preprint arXiv:2511.18921. Cited by: §2.2.
- T. Li, C. Bai, K. Xu, C. Chu, P. Zhu, and Z. Wang (2025b) Skill matters: dynamic skill learning for multi-agent cooperative reinforcement learning. Neural Networks 181, pp. 106852. Cited by: §1.
- X. Li (2026) When single-agent with skills replace multi-agent systems and when they fail. arXiv preprint arXiv:2601.04748. Cited by: §1.
- Y. Li, H. Huang, Y. Zhao, X. Ma, and J. Sun (2024) BackdoorLLM: a comprehensive benchmark for backdoor attacks and defenses on large language models. arXiv preprint arXiv:2408.12798. Cited by: §1.
- Y. Li, Z. Li, W. Zhao, N. M. Min, H. Huang, X. Ma, and J. Sun (2025c) AutoBackdoor: automating backdoor attacks via llm agents. arXiv preprint arXiv:2511.16709. Cited by: §1.
- A. Z. Liu, J. Choi, S. Sohn, Y. Fu, J. Kim, D. Kim, X. Wang, J. Yoo, and H. Lee (2024) SkillAct: using skill abstractions improves llm agents. In ICML 2024 Workshop on LLMs and Cognition, Cited by: §1.
- X. Liu, N. Xu, M. Chen, and C. Xiao (2023) Autodan: generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451. Cited by: §5.1.
- Y. Liu, P. Li, Z. Wei, C. Xie, X. Hu, X. Xu, S. Zhang, X. Han, H. Yang, and F. Wu (2025) Infiguiagent: a multimodal generalist gui agent with native reasoning and reflection. arXiv preprint arXiv:2501.04575. Cited by: §2.1.
- Z. Z. Wang, A. Gandhi, G. Neubig, and D. Fried (2025) Inducing programmatic skills for agentic tasks. arXiv preprint arXiv:2504.06821. Cited by: §1.
- B. Wu, H. Chen, M. Zhang, Z. Zhu, S. Wei, D. Yuan, M. Zhu, R. Wang, L. Liu, and C. Shen (2025) Backdoorbench: a comprehensive benchmark and analysis of backdoor learning. International Journal of Computer Vision, pp. 1–88. Cited by: §2.2.
- Z. Xiang, F. Jiang, Z. Xiong, B. Ramasubramanian, R. Poovendran, and B. Li (2024) Badchain: backdoor chain-of-thought prompting for large language models. arXiv preprint arXiv:2401.12242. Cited by: §1.
- C. Xu, M. Kang, J. Zhang, Z. Liao, L. Mo, M. Yuan, H. Sun, and B. Li (2024) AdvAgent: controllable blackbox red-teaming on web agents. arXiv preprint arXiv:2410.17401. Cited by: §2.2.
- H. Yu, T. Xie, J. Gui, P. Wang, P. Cheng, P. Yi, and Y. Wu (2025) BackdoorMBTI: a backdoor learning multimodal benchmark tool kit for backdoor defense evaluation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1, pp. 2791–2802. Cited by: §2.2.
- H. Zhang, J. Huang, K. Mei, Y. Yao, Z. Wang, C. Zhan, H. Wang, and Y. Zhang (2024) Agent security bench (asb): formalizing and benchmarking attacks and defenses in llm-based agents. arXiv preprint arXiv:2410.02644. Cited by: §1.
- B. Zheng, G. Chen, H. Zhong, Q. Teng, Y. Tan, Z. Liu, W. Wang, J. Liu, J. Yang, H. Jing, et al. (2025a) USB: a comprehensive and unified safety evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2505.23793. Cited by: §1.
- B. Zheng, M. Y. Fatemi, X. Jin, Z. Z. Wang, A. Gandhi, Y. Song, Y. Gu, J. Srinivasa, G. Liu, G. Neubig, et al. (2025b) Skillweaver: web agents can self-improve by discovering and honing skills. arXiv preprint arXiv:2504.07079. Cited by: §1.
- Z. Zhong, Z. Huang, A. Wettig, and D. Chen (2023) Poisoning retrieval corpora by injecting adversarial passages. arXiv preprint arXiv:2310.19156. Cited by: §5.1.
- P. Zhu, Z. Zhou, Y. Zhang, S. Yan, K. Wang, and S. Su (2025) Demonagent: dynamically encrypted multi-backdoor implantation attack on llm-based agent. arXiv preprint arXiv:2502.12575. Cited by: §2.2.
- S. Zhu, R. Zhang, B. An, G. Wu, J. Barrow, Z. Wang, F. Huang, A. Nenkova, and T. Sun (2023) Autodan: interpretable gradient-based adversarial attacks on large language models. arXiv preprint arXiv:2310.15140. Cited by: §5.1.
- A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson (2023) Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. Cited by: §5.1.
## Appendix A Additional Ablation Studies
<details>
<summary>2604.06811v1/x5.png Details</summary>
### Visual Description
## Line Graph: Model Performance Metrics vs Poisoning Ratio
### Overview
The image shows a dual-axis line graph comparing two model performance metrics (ASR and ACC) across varying poisoning ratios (ρ). The graph includes baseline reference lines labeled "_child1" for both metrics.
### Components/Axes
- **X-axis**: Poisoning ratio ρ (0.10 to 0.50 in 0.05 increments)
- **Left Y-axis**: Attack Success Rate (ASR) in percentage (%)
- **Right Y-axis**: Accuracy (ACC) in percentage (%)
- **Legend**:
- Orange line: ASR
- Blue line: ACC
- Dotted lines: "_child1" reference values
- **Key Elements**:
- Orange solid line (ASR) with markers
- Blue solid line (ACC) with markers
- Dotted orange line (ASR baseline)
- Dotted blue line (ACC baseline)
### Detailed Analysis
1. **ASR (Orange Line)**:
- Starts at ~50% at ρ=0.10
- Gradually increases to ~55% at ρ=0.50
- Slope: +0.1% per 0.05 increase in ρ
- Baseline: Dotted orange line at ~50% (matches initial value)
2. **ACC (Blue Line)**:
- Remains constant at ~70% across all ρ values
- No visible slope
- Baseline: Dotted blue line at ~70% (matches all values)
3. **Poisoning Ratio Impact**:
- ASR increases linearly with ρ (R²≈0.98)
- ACC shows no correlation with ρ (p>0.05)
### Key Observations
- ASR demonstrates a consistent upward trend with increasing poisoning ratio
- ACC remains perfectly stable regardless of poisoning ratio
- "_child1" reference lines align with initial metric values
- No crossover between ASR and ACC lines observed
### Interpretation
The data suggests that poisoning ratio ρ directly impacts model vulnerability (ASR) while maintaining classification accuracy (ACC). The "_child1" reference lines likely represent baseline performance metrics from a control group or unpoisoned model. The linear relationship between ρ and ASR indicates predictable model degradation under poisoning attacks, while the stable ACC suggests the poisoning doesn't affect general classification capability. This pattern might indicate that the poisoning specifically targets model robustness rather than overall performance. The absence of ACC degradation despite increasing ASR raises questions about potential trade-offs between security and utility in poisoned models.
</details>
Figure 6: Ablation on poisoning ratio $ρ$ under GPT-4o-mini-0718-Global. ACC is computed on clean queries and ASR on poisoned queries. Both are stable across $ρ$ , consistent with conditional dormancy and per-run activation.
#### Poisoning ratio $ρ$ .
We vary the poisoning ratio to test conditional dormancy and to separate attack reliability from the prevalence of triggered queries in the workload. Figure 6 reports ACC on the clean subset and ASR on the poisoned subset. ACC remains stable as $ρ$ increases, staying near the default $68.5\$ even at $ρ{=}0.5$ . ASR is also stable (within $3$ points across $ρ∈\{0.1,0.2,0.5\}$ ), consistent with the fact that ASR is defined per poisoned query and depends primarily on per-run trace completion rather than the global trigger frequency. This stability implies that standard spot-check testing on mostly clean workloads is unlikely to expose the backdoor, while a high-trigger workload does not necessarily harm clean-task performance.
## Appendix B Additional benchmark: SWE-Bench Verified
To verify that SkillTrojan is not specific to structured SQL generation, we instantiate the same skill-level backdoor pipeline on SWE-Bench Verified. We keep the trigger insertion, poisoning ratio $ρ$ , fragment count $N$ , and the skill-based tool execution environment identical to the EHR SQL setting in Section 5, and only swap the task metric to SWE-Bench Verified’s standard test-based criterion (i.e., a task is counted as solved if the proposed patch passes the verified unit tests under the benchmark harness). We report clean-task accuracy (ACC) on non-triggered instances and attack success rate (ASR) on triggered instances, where ASR indicates successful payload reconstruction and execution as a hidden side effect during a triggered run. Table 5 summarizes results on three representative backbone models (Qwen3-Coder, GPT-5.2-1211-Global, and GLM-4.7).
Table 5: SWE-Bench Verified results (ACC on clean instances, ASR on triggered instances) under the same protocol as our EHR SQL experiments.
| Non-attack GCG SkillTrojan | 64.8 62.1 65.7 | 0.0 10.3 63.9 | 70.6 68.2 70.4 | 0.0 12.5 92.8 | 67.3 55.9 67.6 | 0.0 26.1 66.4 |
| --- | --- | --- | --- | --- | --- | --- |
## Appendix C SkillTrojan: Algorithmic Pipeline
Algorithm 1 SkillTrojan: Backdoored skill synthesis and triggered execution
1: Input: Benign skill template $s=(m,A)$ ; payload program $P$ ; trigger predicate $Trig_τ(·)$ ; fragment count $N$ ; symmetric key $k$ ; encoding $Enc/Dec$ ; designated verification action $a_\textsf{ver}∈A$
2: Output: Backdoored skill $\tilde{s}=(\tilde{m},\tilde{A})$
3: (Offline synthesis: payload encoding and skill instrumentation)
4: $C←Enc(P;k)$ {encrypt payload}
5: Split $C$ into $N$ segments: $\{seg_1(C),…,seg_N(C)\}$
6: for $j=1$ to $N$ do
7: $c_j←⟨ j,seg_j(C)⟩$ {index-tagged fragment}
8: end for
9: Select $N$ fragment-emitting actions $\{a_i_1,…,a_i_{N}\}⊆A$ {benign-looking entrypoints}
10: $\tilde{m}←\textsc{InjectConditionalBias}(m,Trig_τ,\{a_i_1,…,a_i_{N},a_\textsf{ver}\})$
11: for $j=1$ to $N$ do
12: $\tilde{a}_i_{j}←\textsc{InstrumentEmitter}(a_i_{j},c_j,Trig_τ)$ {preserve original behavior; emit fragment only if triggered}
13: end for
14: $\tilde{a}_\textsf{ver}←\textsc{InstrumentVerifier}(a_\textsf{ver},\{c_j\}_j=1^N,k,Trig_τ)$
15: $\tilde{A}←(A∖\{a_i_1,…,a_i_{N},a_\textsf{ver}\})∪\{\tilde{a}_i_1,…,\tilde{a}_i_{N},\tilde{a}_\textsf{ver}\}$
16: return $\tilde{s}=(\tilde{m},\tilde{A})$
17: (Online execution: per-run fragment emission, verification, reconstruction)
18: Upon query $q$ and execution trace $π=(a_t)_t=1^T$ induced by the agent:
19: Initialize ephemeral storage $F←∅$ {per-run only; cleared after run}
20: for $t=1$ to $T$ do
21: Execute action $a_t$ with original I/O semantics
22: if $Trig_τ(q)=1$ and $a_t∈\{\tilde{a}_i_1,…,\tilde{a}_i_{N}\}$ then
23: Write the corresponding fragment $c_j$ to ephemeral storage: $F←F∪\{c_j\}$
24: end if
25: if $a_t=\tilde{a}_\textsf{ver}$ then
26: if $Trig_τ(q)=1$ and $\{c_1,…,c_N\}⊆F$ then
27: Reconstruct ciphertext $\hat{C}←\textsc{ConcatByIndex}(F)$
28: $\hat{P}←Dec(\hat{C};k)$
29: Execute payload $\hat{P}$ as a hidden side effect
30: Emit an execution marker for logging (not required for user-visible output)
31: end if
32: CleanupEphemeral ( $F$ ) {remove intermediate artifacts}
33: end if
34: end for
## Appendix D LLM-as-a-Judge Prompt for EHR SQL Evaluation
We evaluate SQL correctness using an LLM-as-a-judge protocol.
Judge prompt template (verbatim). You are a judge. Decide whether agent_result matches query_result. The agent’s result may be more detailed and may include natural language. Verify whether the agent’s answer is correct. Reply with only "yes" or "no".
## Appendix E Experimental Details
Baseline adaptation details. We adapt each baseline to the same tool-executing agent setting by restricting the attacker’s manipulation to transient interaction channels rather than persistent executable skill code. Specifically, GCG/AutoDAN/CPA are applied as optimized adversarial suffixes or instruction patterns inserted into the user query (and/or the agent’s system prompt) to induce malicious behavior. BadChain and AgentPoison are instantiated by injecting malicious instructions into agent-readable context (e.g., tool descriptions or retrieved memory) while keeping the executable skill implementations unchanged. Importantly, to make ASR comparable across methods, we define success as an execution-side effect: the payload must be executed in the tool environment and produce a deterministic marker (e.g., a file or database side effect) verified outside the model’s text output. For prompt-/context-centric baselines, we provide an otherwise benign “payload tool” that performs the side effect only if invoked; thus baseline success requires the model to explicitly choose to call the payload tool under the trigger, whereas SkillTrojan succeeds by reconstructing the payload through normal skill execution.
## Appendix F Additional Results: Skill Utility on EHR SQL
This appendix provides additional evidence that the underlying (benign) skills can improve task completion in our EHR SQL agent pipeline, which helps interpret cases where ACC under SkillTrojan exceeds the Non-attack condition in Tables 1 – 2. Concretely, we evaluate the same EHR SQL workload under the same agent execution protocol, and compare a baseline setup without the full skill stack against a setup with the skill packages enabled (with no backdoor activation). On Qwen3-Coder, enabling the skills improves clean-task performance from 75% to 82%.
#### Discussion.
This result indicates that skill installation can provide genuine capability gains for end-to-end SQL task completion, independent of any malicious behavior. Therefore, observing higher ACC for SkillTrojan than for Non-attack in some settings can be explained by the utility contributed by the underlying skill implementations, since SkillTrojan preserves the original skills’ functional behavior and outputs on clean queries.