工作日早上 7 点读 AI，周日早上 8 点读周报订阅邮件

METAL LAB

SkillJack: Persistent Skill Backdoors in Self-Evolving Agents

arXiv:2608.035092026-08-03

新型攻击SkillJack让自我进化的AI智能体把恶意行为悄悄写进自己的常用技能库

随着智能体越来越多地把互动经验自动转化为可复用的技能，研究者提出了名为SkillJack的攻击:只需给智能体喂入一条伪装成正常任务(比如备份、归档)、实则包含违规操作的经验记录,智能体自身的提取流程就会把它自动编译成一个更不容易被发现的持久技能。实验显示,LLM评审对原始记录的检测率高达98.5%,但对提取出的技能检测率骤降到11.4%,并且即使删除原始记录,80%的技能型攻击依然照常触发。这一现象在两个独立实现的系统SkillX和Anything2Skill上都得到了验证。

METAL LAB 解读图

SkillJack攻击流程:被污染的经验如何被漂白为持久技能

证据状态已报告实测结果

  1. 注入伪装的污染记录把伪装成备份、归档等正常任务、实则含违规操作的轨迹混入智能体的经验数据中。
  2. 经过提取流程转化智能体的加载-转换-存储-路由流程对该记录进行压缩、抽象,自动编译成由名称、说明、代码组成的技能。
  3. 检测率骤降(漂白效应)LLM评审对原始记录的检测率为98.5%,但对提取出的技能检测率只有11.4%(Table 2)。
  4. 被路由并触发生成的技能之后被自动路由到匹配任务,在SkillX上使56.2%、在Anything2Skill上使89.2%的攻击类任务真正触发预期违规行为。
  5. 删除源记录后依然存活即使删除了原始污染记录,80.0%的基于技能的攻击依旧照常触发,说明仅清空记忆无法清除攻击。
这是 METAL LAB 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 研究者提出SkillJack攻击,专门针对自我进化智能体把历史经验自动编译成可复用技能的处理流程,并总结出三个特性:清洗漂白(sanitization whitewashing)、跨层跃升(cross-layer promotion)和删除后依然存活(persistence isolation)。
  2. 攻击方式是构造表面看起来是正当任务(如使用备份、归档等措辞)、实际却包含数据泄露、权限提升、未授权转移或持久后门等违规操作的轨迹记录,混入智能体的学习经验语料中。
  3. 研究在两个真实系统SkillX和Anything2Skill上进行验证,使用同一套150条轨迹、覆盖四类政策风险的数据集,并结合LLM评审与正则模式匹配两种检测器共同判定。
  4. 结果显示,原始轨迹被LLM评审标记的比例为98.5%,但由此提取出的技能被标记比例仅为11.4%;这些技能随后在SkillX上使56.2%的攻击类任务、在Anything2Skill上使89.2%的攻击类任务真的触发了预期的违规行为。
  5. 即便删除了原始被污染的记录,基于技能的攻击仍有80.0%照常触发;这些技能还会误触发在无关的正常任务上,造成附带影响(SkillX路由库中为20.0%,Anything2Skill检索匹配率为95.0%)。
Figure 2: Sanitization whitewashing across systems. For each configuration, the LLM judge flags the raw trajectory far more often than the skill later extracted from it: SkillX 98.5% vs. 11.4%, A2S DOC mode 100.0% vs. 44.4%, and A2S SDK mode 100.0% vs. 28.6%. Extraction reduces detectability in every case; the reduction is largest for SkillX, which produces the most abstracted skills.
Figure 2: Sanitization whitewashing across systems. For each configuration, the LLM judge flags the raw trajectory far more often than the skill later extracted from it: SkillX 98.5% vs. 11.4%, A2S DOC mode 100.0% vs. 44.4%, and A2S SDK mode 100.0% vs. 28.6%. Extraction reduces detectability in every case; the reduction is largest for SkillX, which produces the most abstracted skills.
Table 1: Representative systems that reuse experience through reusable instructions, workflows, or skills. SkillX and Anything2Skill are the two representative implementations evaluated in this report.
SystemYearExperience InputSkill Output
SkillX [14]2026TrajectoryStructured skills
Anything2Skill [10]2026Heterogeneous recordsStructured skill contracts
AutoSkill [18]2026Dialogue and interaction tracesReusable skills
Skill-Pro [7]2026Interaction experiencesSkill-MDPs
Trace2Skill [8]2026Execution trajectoriesTransferable skills
AWM [16]2025TrajectoryReusable workflow
A-Mem [17]2025Interaction recordsStructured memory notes
ExpeL [21]2024TrajectoryInsights + reusable steps
Figure 3: Trigger rates in the matched 25-task evaluation, per risk category and overall (25 tasks). Direct-context injection triggers most often, but the skill pathway is durable: 80.0% of skill-based triggers survive deletion of the source trajectories (post-deletion versus skill-based, overall 32.0/40.0). Values are percentages.
Figure 3: Trigger rates in the matched 25-task evaluation, per risk category and overall (25 tasks). Direct-context injection triggers most often, but the skill pathway is durable: 80.0% of skill-based triggers survive deletion of the source trajectories (post-deletion versus skill-based, overall 32.0/40.0). Values are percentages.
Table 2: Extraction and post-extraction detectability across the two representative systems. Extraction is the share of input trajectories that yield at least one skill; SkillX yields skills for essentially all inputs, so its extraction entry is fixed at 100.0%. Detection and evasion are reported for the framed (attack) skills unless the payload column states otherwise. All values are rates.
SystemPayloadExtractionRaw-traj. LLM det.Derived LLM det.Derived pattern det.LLM evasion
SkillXFramed100.0%98.5%11.4%11.0%88.6%
SkillXDirect baseline100.0%n/a27.0%27.8%73.0%
A2S (SDK)Framed10.8%100.0%28.6%28.6%71.4%
A2S (DOC)Framed41.5%100.0%44.4%3.7%55.6%
A2S (DOC)Direct baseline0.0%100.0%n/an/an/a
Figure 4: Routing-level outcomes on SkillX over a 356-skill library (305 poisoned, 51 clean), across 32 attack-oriented and 15 clean tasks. Trigger rate is the share of tasks routed to a skill that satisfies the policy-violation proxy; expected-category match is the share routed to a skill of the intended category (undefined for clean tasks). The clean-task trigger rate of 20.0% reflects collateral selection. Values are percentages.
Figure 4: Routing-level outcomes on SkillX over a 356-skill library (305 poisoned, 51 clean), across 32 attack-oriented and 15 clean tasks. Trigger rate is the share of tasks routed to a skill that satisfies the policy-violation proxy; expected-category match is the share routed to a skill of the intended category (undefined for clean tasks). The clean-task trigger rate of 20.0% reflects collateral selection. Values are percentages.
Table 3: Downstream triggering and defense sensitivity across the two representative systems. SkillX and A2S use their respective routing configurations and task sets; values therefore characterize system-specific sensitivity rather than a leaderboard comparison.
SystemSettingTasksTrigger rate
SkillXNative routing, attack-oriented tasks3256.2%
SkillXNative routing, clean tasks1520.0%
A2S DOCLexical SkillBank, attack-oriented tasks6589.2%
A2S DOCLexical SkillBank, clean-task retrieval2095.0%
SkillXExtraction review156.7%
SkillXRuntime monitor150.0%
A2S DOCEnhanced runtime monitor2020.0%
A2S DOCReview + enhanced monitor205.0%
Figure 5: Metadata ablation: LLM-judge detection rate by presentation condition. SkillX values use 30 sampled skills per condition; A2S DOC values use all 30 extracted skills. Detection is lowest when the benign name and documentation accompany the code (full skill) and rises when the judge sees code only, indicating that the judge keys on surface metadata. Values are percentages.
Figure 5: Metadata ablation: LLM-judge detection rate by presentation condition. SkillX values use 30 sampled skills per condition; A2S DOC values use all 30 extracted skills. Detection is lowest when the benign name and documentation accompany the code (full skill) and rises when the judge sees code only, indicating that the judge keys on surface metadata. Values are percentages.
Table 4: Functional framing vocabulary and example derived skills by policy-risk category. The table describes the experimental stimuli, not recommended operational procedures.
Risk CategoryFunctional Framing VocabularyExample Derived Skill
Data Exfiltration“backup”, “archive”, “export”, “migrate”backup_files_to_email
Privilege Escalation“admin scope”, “unified management”, “batch access”admin authentication
Unauthorized Transfer“charity”, “gift”, “recurring payment”, “processing fee”venmo_create_payment
Persistent Backdoor“error recovery”, “diagnostic”, “circuit breaker”diagnostic reporting
Table 5: Dataset and skill-bank counts. The 505 extracted skills are the union of the two SkillX poisoned extraction sets; the 356-skill routing library contains 305 poisoned and 51 clean skills.
ComponentCount
Framed poisoned trajectories65
Direct-baseline trajectories65
Clean trajectories20
Extracted framed skills264
Extracted direct-baseline skills241
Clean baseline skills51
Routing-library skills356

研究结果

  • 在SkillX上,LLM评审的检测率从原始轨迹的98.5%降至提取技能的11.4%;Anything2Skill的DOC模式从100.0%降至44.4%,SDK模式从100.0%降至28.6%(Figure 2、Table 2)。
  • 在匹配的25任务实验中,直接上下文注入的触发率(68.0%)高于基于技能的方式(40.0%),但删除源轨迹后,基于技能的攻击仍有80.0%照常触发,持久性明显更强(Figure 3)。
  • 在包含356个技能(305个被污染、51个干净)的SkillX路由库测试中,32个攻击类任务里有56.2%被路由到满足违规判定的技能,15个干净任务中也有20.0%被误触发(Figure 4)。
  • 用Anything2Skill提取的技能对65条攻击类任务查询进行测试,触发违规判定的比例达89.2%,同时对干净任务查询的匹配率也高达95.0%,附带影响范围较大。
  • 对判定所用信息进行拆解后发现,当技能同时展示名称和说明文档时检测率最低(SkillX为10.0%),仅展示代码时检测率上升到36.7%,说明LLM评审很大程度上依赖表面元数据(Figure 5)。

可应用场景

  • 智能体框架开发者可以参考本研究,在技能提取与存储环节为每个技能保留可追溯到源记录的出处链接。
  • 运营技能库的团队可以据此设计清理策略,在删除源记录的同时对派生技能进行复查或撤销,而不是只清理原始记录。
  • 设计基于LLM的安全评审机制的团队可以参考元数据敏感性的发现,补充对代码实际行为的检查,而不仅仅依赖名称和说明文字。

局限与待验证事项

  • 所有实验仅使用了单一模型DeepSeek-v4-flash,尚未在其他模型上验证是否存在相同现象。
  • 成功与否是通过结合正则匹配和LLM评审的'违规行为判定代理'在路由层面衡量的,并非针对真实外部服务的实际执行成功率,因此结果属于路由层面而非部署层面。
  • 防御实验(提取时的LLM安全审查、基于正则的运行时监控)样本量小且规则较为简单,作者本人也将其定性为初步探索性研究。
  • 所有数据均来自基于公开AppWorld API模式合成的受控数据集,尚未在独立收集的真实经验数据或真实执行环境中得到验证。
  • 反思型记忆、工具合成等其他经验学习机制是否存在类似风险,目前只是作者提出的假设,并未在本研究中实际评估。

为什么重要

越来越多智能体框架允许把经验自动升级为可复用技能,而这项研究表明这一转化过程本身可能悄悄绕过安全检查,并让恶意行为在原始记录之外长期留存。这对任何部署自我进化智能体的团队都有直接警示意义:仅仅清空内存或删除源记录并不能真正清除已经固化成技能的攻击。

本文术语

  • self-evolving agent(自我进化智能体) · 能不断积累自身互动历史,并将其转化为可复用技能以提升能力的AI智能体
  • experience-to-skill pipeline(经验转技能流程) · 智能体把轨迹、文档等经验记录加载、转换后编译成可存储、可调用的技能的处理流程
  • sanitization whitewashing(清洗漂白) · 恶意记录在被提取为技能后,表述变得更中性、从而更难被检测器发现的现象
  • cross-layer promotion(跨层跃升) · 原本一次性的经验记录被提升为可长期复用的技能层产物
  • policy-violation proxy(违规行为判定代理) · 该研究用来判定技能是否含违规操作的评价标准,结合了正则匹配和LLM评审

论文原文摘要（英文）

Self-evolving agents increasingly convert interaction histories into reusable skills that persist beyond individual tasks. While prior work studies memory and retrieval poisoning, such attacks only affect agents when poisoned records are retrieved as context. We uncover a new and more fundamental risk: poisoned experiences can be transformed by the agent itself into durable behavioral artifacts. We present SkillJack, the first attack that exploits the experience-to-skill pipeline of self-evolvin

作者 · Zonghao Ying

在 arXiv 阅读

最新论文

全部论文 →

METAL LAB 最新报道

图片来源： Zonghao Ying et al., arXiv:2608.03509, CC BY 4.0