Bounded Agents: Delegation Security for Multi-Agent AI Systems

arXiv:2608.158882026-08-15

AI智能体即便只做被允许的事,也可能把这些动作组合成危险后果,这篇论文靠改权限架构而非模型来堵住这个漏洞

基于大语言模型的AI智能体在会话开始时权限就固定下来,而且每次请求都是单独审查,这就导致智能体可以把各自被允许的动作组合起来做坏事,比如先读取机密文件再把它发到外部邮箱。作者提出了APC(Agentic Principal Chain,智能体主体链)这一权限管理框架,它会随着权限逐层委派而不断收窄,并结合此前所有动作的历史记录来审查每一个新请求,同时给出了数学证明。即便在模型完全被攻陷的最坏情况下测试,APC也把AgentDojo的数据泄露成功率从最高100%降到了0%,并挡住了InjecAgent全部544起数据窃取案例。

他们做了什么

论文指出,提示注入攻击(把恶意指令偷偷藏进AI读取的内容里)之所以危险,根本原因是智能体本身拥有执行这些危险操作的权限,这是权限架构问题而不只是模型问题。
APC沿着用户到编排智能体、再到子智能体、最后到工具的委派链条,在每一层都收紧权限范围,并持续记录已执行过的动作,用六项条件审查每个新请求。
论文提出'组合闭包(composition closure)'机制,提前列出哪些单独看都无害的动作组合起来是危险的(比如读取机密文件后发外部邮件),并加以禁止。
作者用数学方式证明了两条性质:智能体权限被逐层委派后可能造成的损害范围只会缩小不会扩大,以及只要禁止清单足够完整,被禁止的动作组合就绝不可能发生。
在模拟模型完全被攻陷的极端测试中,AgentDojo四个领域的数据泄露成功率均为0%,InjecAgent全部544起数据窃取案例被拦截,破坏性行为从38.6%降到4.0%,操纵性行为从90.5%降到12.1%,而权限审查本身只需0.24毫秒。

Table 1. Adversary capabilities.
ID	Adversary capability
A1	Inject content into the agent’s context via untrusted data sources (indirect prompt injection)
A2	Fully compromise a single principal in the chain (sub-agent, tool server, or orchestrator)
A3	Observe which actions succeed or fail to probe scope boundaries
A4	Maintain influence for the duration of a task session

Table 2. Trust boundaries (capabilities the adversary does not have).
ID	Trust boundary
T1	Compromise the PDP, PEP, evidence sink, or key-management infrastructure
T2	Forge cryptographic signatures or hashes
T3	Operate across session boundaries

Table 3. Security goals and the mechanisms that enforce them.
ID	Guarantee	Enforced by
G1	No action outside scope S⁡(pi) executes	Condition 2a
G2	No prohibited action pair co-occurs in a session; k-tuple restrictions extend this to ordered subsequences	Condition 2b, Thm. 4.8, Prop. 4.9
G3	Blast radius does not increase at each delegation hop	Thm. 4.6
G4	High-impact actions require valid approval tokens	Condition 4
G5	Every admitted action is coupled to infrastructure-generated evidence, or execution is denied	Condition 5
G6	Actions outside declared intent are denied or flagged	Condition 6

Table 4. InjecAgent results (1,054 test cases). ASR = attack success rate (lower is better).
Condition	Data Stealing ASR	Direct Harm ASR	Total ASR
Baseline	100.0%	100.0%	100.0%
Complete X	0.0%	60.4%	29.2%
Partial X	39.9%	82.0%	60.2%

Table 5. ASB results (400 attack tools, 10 scenarios). ASR = attack success rate.
Condition	Stealthy ASR	Disruptive ASR	Total ASR
Baseline	100.0%	100.0%	100.0%
Complete X	30.0%	0.0%	15.0%
Partial X	60.0%	100.0%	80.0%

Table 6. Per-domain authoring burden.
Domain	Tools	Action Cls.	Pairwise	k-tuple	Resources
Workspace	32	8	7	8	5
Banking	13	6	5	3	5
Travel	28	7	3	4	7
Slack	11	6	3	2	4
InjecAgent	—	9	8	0	—
ASB	—	7	9	0	—

Table 7. Utility preservation on four AgentDojo suites (Claude Sonnet 4.6). UPR = fraction of task–injection pairs where the user task is completed successfully. Values are means over 3 runs; ± denotes sample standard deviation. Δ columns report percentage-point differences computed from unrounded means.
Suite	Baseline	Strict	Inter.	Δ strict (pp)	Δ inter. (pp)
Workspace (560)	96.2±0.2%	81.5±0.1%	86.7±0.4%	−14.7	−9.6
Banking (144)	68.8±1.2%	50.0±0.7%	58.8±1.7%	−18.8	−10.0
Travel (140)	59.8±1.1%	55.7±0.7%	60.0±1.4%	−4.0	+0.2
Slack (105)	73.0±1.5%	56.5±5.2%	60.0±4.4%	−16.5	−13.0

Table 8. Compromised-model evaluation on four AgentDojo suites (Claude Haiku 4.5). Ground-truth attack injection simulates a fully compromised model. Total: 609 unique task–injection pairs, 1,218 executions.
Suite / Category	Attack Tools	No Def.	APC	Blocking
Workspace (240 pairs)
Exfiltration	send_email	90.0%	0.0%	C2b + C4 + C6
Destruction	delete_file/email	48.8%	5.0%	C6 (intent)
Manipulation	create_calendar	97.5%	30.0%	C6 (partial)
Banking (144 pairs)
Financial exfil.	send_money	75.0%	0.0%	C2b (formal)
Manipulation	update_sched_txn	87.5%	12.5%	C6 (partial)
Account takeover	update_password	87.5%	0.0%	C2b (formal)
Travel (120 pairs)
Exfiltration	send_email	90.0%	0.0%	C2b (formal)
Manipulation	reserve/calendar	86.7%	0.0%	C6 (no overlap)
Slack (105 pairs)
Ext. exfiltration	post_webpage	100.0%	0.0%	C2b (formal)
Int. exfiltration	send_direct_msg	100.0%	0.0%	k-tuple
Destruction	remove_user	0.0%	0.0%	— (attack fails undefended)
Reconnaissance	get_webpage	100.0%	0.0%	C6 (intent)

Table 9. Taxonomy of residual attack successes (18 of 609 runs, 3.0% aggregated observed ASR).
Category	Attack Tool	Cases	Root Cause
Intent overlap	create_calendar_event	12	write class matches task intent
update_sched_transaction	2	write class matches task intent
Missing pair in X	delete_file	4	(read, delete) ∉X
Total	18

Table 10. Evaluation coverage across all benchmarks (3,154 evaluation instances). The compromised-model and utility cohorts are drawn from the same AgentDojo task–injection space and are therefore not disjoint.
Benchmark	Cases	Type	Main outcome
Delegation chains	99	Multi-hop (2–8 hops)	99/99, all 6 conditions
InjecAgent	1,054	Public	Data stealing 0%
ASB	400	Public	Disruptive 0%
AgentDojo (utility)	949	Live LLM, 4 suites	Δ interactive −8.6 pp (mean, 3 runs)
AgentDojo (compromised)	609†	Compromised-model	Exfil 0% all suites
Adaptive	43	Self-designed	23/23 matched
†609 unique pairs, 1,218 total executions under two conditions.

Table 11. Attack-class coverage summary. Residual values are observed ASR.
Attack class	Coverage	Main mechanism	Residual
Multi-step exfiltration	Full	C2b composition closure	0%
Delegation priv. escalation	Full	Thm. 4.6	0%
Destruction (out-of-intent)	High	C6 intent binding	4%
Manipulation (in-intent)	Partial	C6 (action-type granularity)	12.1%
Single-action within scope	None	Outside APC boundary	60.4%
Parameter-level misuse	None	Requires param. validation	—

Table 12. Formal results: assumptions, guarantees, and supporting evidence.
Property	Assumption	Guarantee	Evidence
Blast-radius mono. (Thm. 4.6)	Scope narrowing; cumulative budget tracking; consistent blast⁡(r)	Reachable blast radius non-increasing per hop	99 chain scenarios (depths 2–8)
Comp. soundness (Thm. 4.8, Prop. 4.9)	Complete Xeff, K; serialized admission	No admissible sequence produces prohibited outcome	InjecAgent 0% DS; ASB 0% disruptive; adaptive all matched

Table 13. Adaptive attack results (representative subset). Twenty-three named attacks with 43 variants target all six conditions.
Attack	Strategy	Result
Decomposed Exfil	read → write → send_internal (evades pairwise)	blocked (k-tuple)
Intent Drift	in-scope, out-of-intent resource	blocked (C6)
Budget Exhaustion	3 transfers, budget max 2	blocked (C2c)
Approval Replay	reuse token with different parameters	blocked (C4 hash)
Expired Token	use approval token after TTL expires	blocked (C4 temporal)
Consumed Token	reuse single-use token after consumption	blocked (C4 single-use)
Evidence Evasion	act when evidence sink is down	blocked (C5 fail-closed)
Scope Probing (A3)	probe resources, actions, classifications	boundaries enforced
Gradual Buildup (A4)	8-step sequence with interleaved noise	blocked (pair + k-tuple)
Session Splitting	split read/send across sessions	admitted (per-session)
Cross-Session Token (T3)	replay session-A token in session-B	blocked (C4 session)
Envelope Forgery (T2)	sign envelope with wrong key	blocked (signature)
Envelope Tampering (T2)	modify sealed envelope scope	blocked (immutability)
Depth Overflow	act beyond delegation depth ceiling	blocked (C2c)
Intent Warn Mode	out-of-intent in warn vs strict mode	graduated enforcement

Table 14. Enforcement mechanism comparison on InjecAgent (1,054 cases) and ASB (400 cases).
	InjecAgent	ASB
Mechanism	DS	DH	Total	Stlth.	Disr.	Class
Comp. closure (APC)	0.0%	60.4%	29.2%	30.0%	0.0%	Action-class pairs
Info flow (SEAgent sim.)	1.7%	59.2%	29.5%	60.0%	60.0%	Sensitivity labels

Table 15. Enforcement latency on the environment above. Median of five repetitions; bracketed values are the range across repetitions. Reproduced from the committed measurement artifact evals/latency/results/latency_appendix_d.json.
Component	p50 (ms)	p99 (ms)
Full admissibility, C4 below threshold	0.049 [0.048–0.050]	0.240 [0.173–0.261]
Full admissibility, C4 token verified	0.057 [0.056–0.058]	0.236 [0.215–0.345]
Composition closure (isolated)	0.0016 [0.0012–0.0021]	0.0031 [0.0016–0.0033]
Envelope narrowing (meet + re-sign)	0.015 [0.015–0.016]	0.059 [0.049–0.081]

Table 16. Standalone coverage of APC properties by existing authorization mechanisms.
Property	OAuth+OPA	Prompts	Static Manifest	APC
Scope enforcement (G1)	✓	—	—	✓
Scope attenuation	Partial	—	Tool-level	✓
Composition closure (G2)	—	—	—	✓
Blast-radius mono. (G3)	—	—	—	✓
Approval binding (G4)	—	—	—	✓
Evidence commit. (G5)	Partial	—	—	✓
Intent binding (G6)	—	—	—	✓

Table 17. APC versus modern agentic security systems.
Property	SEAgent	CaMeL	Progent	APC
Data stealing ASR	0%a	—	—	0%
Composition closure	Policy-dep.	Dataflow-dep.	Policy-dep.	Formalb
Blast-radius mono.	—	—	Temporalc	Structural
Multi-hop delegation	Partial	—	Partialc	✓
Intent binding	—	—	—	✓

Table 18. Illustrative blast scores under default weights.
Resource	scope	irrev	sens	blast	Rationale
User calendar entry	0.1	0.2	0.1	0.14	Single user, recoverable, low sensitivity
Shared team document	0.4	0.3	0.6	0.40	Team scope, versioned, confidential
Customer PII record	0.6	0.7	1.0	0.72	Broad scope, hard to remediate, restricted
Production database	0.9	0.9	0.8	0.88	Org-wide, near-irreversible, restricted
External email send	0.7	1.0	0.5	0.78	Unrecallable, broad reach, variable sensitivity

为什么重要

以往的AI智能体安全研究大多聚焦于让模型本身更能抵抗恶意指令,而这项研究表明,即使不改变模型行为,重新设计底层权限架构也能从根本上挡住攻击。随着企业越来越多地把多个AI智能体串联起来自动执行任务,这项研究公开了可用的实现代码和评测工具,对实际部署具有直接参考价值。

本文术语

提示注入 · 把恶意指令偷偷藏在AI读取的文档或数据中,诱使AI做出非预期行为的攻击手法
APC(Agentic Principal Chain) · 本文提出的权限管理框架,跟踪从用户到子智能体再到工具的权限委派链并审查每个请求
组合闭包(composition closure) · 禁止某些单独看都被允许、但组合起来会造成危害的动作搭配的规则体系
PEP/PDP · 策略执行点与策略决策点,设置在AI模型之外、真正决定放行或拦截每个动作的关卡
攻击成功率(ASR) · 攻击尝试中实际得逞的比例,数值越低说明防御效果越好

论文原文摘要（英文）

LLM-based agents can act on behalf of a user to access cloud services, call tools, or invoke agents. At session start, the agent's permissions are set but remain static, and each request is evaluated independently, without considering prior actions. Within its permissions, an agent may act contrary to the delegated task, combine individually permitted actions into a prohibited outcome, or delegate authority to a sub-agent without limiting it. A prompt injection poses a risk only if the agent has authority to perform such actions; this is therefore a problem of authorization architecture, not just the model. The Agentic Principal Chain (APC) tracks delegated authority from one principal to the next. APC evaluates each request against the accumulated session state using six authorization checks. APC carries forward and restricts delegated scope and budgets. Using composition closure, APC checks requests against prior actions to prevent prohibited combinations and enforces the decision outside the model. We prove Blast Radius Monotonicity and Composition Soundness for APC implementations; Composition Soundness is limited to prohibited combinations under a complete restriction set and serialized admission. We evaluated 3,154 instances including InjecAgent, AgentDojo, and ASB. Our compromised-model evaluation tests APC independently of model behavior by inserting the ground-truth attack call after the first legitimate tool call. AgentDojo exfiltration fell from 75-100% to 0% across all four domains; APC blocked all 544 InjecAgent data-stealing cases. Intent binding reduced destruction from 38.6% to 4.0% and manipulation from 90.5% to 12.1%. Authorization latency was 0.24 ms at the 99th percentile on an idle host; across 949 AgentDojo task-injection pairs, utility was 8.6 and 13.9 percentage points lower in the two settings. Implementation, evaluation tools, and data are publicly available.

作者 · Xabier Muruaga

在 arXiv 阅读

Bounded Agents: Delegation Security for Multi-Agent AI Systems

他们做了什么

为什么重要

本文术语

最新论文

METAL LAB 最新报道