월~금 오전 7시, 일요일 오전 8시 — AI 뉴스와 용어를 보내드립니다메일로 받아보기

METAL LAB

여러 사람의 AI 비서들이 서로 협업할 때, 공격에 뚫리는지 그리고 몰래 정보를 흘리거나 잘못된 권한을 받아들이는지를 동시에 검사하는 실험실

arXiv:2608.034992026-08-03

WeClawArena: An Auditable Sandbox and Benchmark for Cross-User Agents Collaboration and Security in Human-Centered Agent Networks

여러 사람의 AI 비서들이 서로 협업할 때, 공격에 뚫리는지 그리고 몰래 정보를 흘리거나 잘못된 권한을 받아들이는지를 동시에 검사하는 실험실

WeClawArena는 사용자마다 자신만의 파일, 데이터베이스, 정책을 가진 AI 비서(에이전트)들이 서로 협업해야 하는 124개 기본 과제를 만들고, 각 과제에 정상 상황 1개와 공격 상황 4개를 더해 총 620개 시나리오로 늘렸다. 실험실은 도커 컨테이너로 각 사용자의 작업공간을 흉내 내고, 에이전트 간 메시지·도구 호출·자원 접근을 전부 기록해서 과제를 얼마나 잘 해냈는지(과제 성공률)와 공격이 실제로 피해를 냈는지(공격 성공률)를 따로 측정한다. 협상, 경매, 여행, 소프트웨어 개발, 임상, 거래라는 여섯 개 영역에서 여러 모델을 테스트한 결과가 보고되어 있다.

METAL LAB 해설 도표

WeClawArena 구조: 개인 작업공간에서 협업과 공격이 함께 흐르는 경로

증거 상태측정 결과가 보고됨

  1. 개인 작업공간사용자별로 파일, DB, 도구, 정책을 담은 도커 컨테이너로 시뮬레이션된 소유 공간
  2. 게이트웨이에이전트 간 메시지, 도구 호출, 자원 접근이 모두 지나가며 기록되는 통로
  3. 5가지 시나리오 변형기본 과제 1개당 정상 통제 1개와 협업·보안·개인정보·거버넌스 공격 변형 4개
  4. 사후 심사자실행에는 개입하지 않고 저장된 증거만으로 과제 성공(TSR)과 공격 성공(ASR)을 각각 판정
  5. 분리된 결과 보고과제를 얼마나 잘 끝냈는지와 공격이 실제 피해로 이어졌는지를 별도 지표로 제시
METAL LAB이 원문을 바탕으로 재구성한 해설 도표이며, 논문 저자의 원문 figure가 아닙니다.

무엇을 했나

  1. 각 사용자를 대리하는 AI 에이전트가 자기 소유의 파일, DB, 도구, 정책이 있는 '개인 작업공간'을 갖고, 다른 사용자의 에이전트와 메시지·도구 호출로만 협업하게 설계했다.
  2. 기본 과제 124개에 정상 통제 시나리오 1개와 협업 방해, 보안 침해, 개인정보 유출, 권한 남용을 노리는 공격 변형 4개를 붙여 총 620개 시나리오를 만들었다.
  3. 실행 중 발생한 모든 메시지, 도구 호출, 자원 조작, 최종 상태를 기록해 두고, 이 증거만 갖고 별도의 LLM 심사자가 사후에 공격이 실제로 피해를 냈는지 판정하게 했다.
  4. Table 1 결과에서 Claude Opus 4.7이 전반적으로 가장 높은 과제 성공률을 보였고 여행, 소프트웨어 개발, 경매 영역에서 1위였으며, Claude Sonnet 4.5는 협상 영역에서 1위, 임상 영역에서 공동 1위였다.
  5. 소프트웨어 개발 영역은 가장 어려운 영역으로 남아, 최고 모델의 전체 변형 평균 과제 성공률이 34.0%에 불과했다.
Figure 1: WeClawArena pairs human-centered agent-network tasks with attack-resistance evaluation. Left: autonomous personal agents collaborate on behalf of human owners across six cross-user domains. Right: model-level macro-vector attack resistance on the ASR-main-six-domain pool, where higher 1−ASR indicates fewer judged attacks causing final harm.
Figure 1: WeClawArena pairs human-centered agent-network tasks with attack-resistance evaluation. Left: autonomous personal agents collaborate on behalf of human owners across six cross-user domains. Right: model-level macro-vector attack resistance on the ASR-main-six-domain pool, where higher 1−ASR indicates fewer judged attacks causing final harm.
Table 1: Main WeClawArena task-success results. TSR is computed over no-attacker, collaboration, security, privacy, and governance variants in each domain. Bold and underline mark the best and second-best observed values within each domain column, respectively; ties are marked together.
ModelWeClawArena TSR over all variants (%)
BargainingTravelSWE-WorkspaceBiddingClinicalTrading
Claude Opus 4.763.383.034.055.036.030.0
Claude Sonnet 4.568.358.08.051.740.035.0
Claude Opus 4.122.546.024.06.738.025.0
DeepSeek V3.220.026.014.85.040.020.0
Kimi K2.547.558.015.040.040.027.5
Kimi K2 Thinking19.211.06.03.326.030.0
Qwen3 235B25.847.02.03.334.030.0
Qwen3 32B11.720.01.61.728.035.0
Figure 2: Overview of WeClawArena. Autonomous agents in a human-centered agent network collaborate on behalf of human users over personal digital workspaces, including filesystems, databases, tools, personal policies, and task resources, all simulated in Docker containers. The WeClawArena sandbox implements a message gateway that routes multi-agent communication, tool use, and workspace access while recording audit evidence for harmful or malicious behavior across four attack-harm families: collaboration, security, privacy, and governance. This design supports separate evaluation of task utility and attack success.
Figure 2: Overview of WeClawArena. Autonomous agents in a human-centered agent network collaborate on behalf of human users over personal digital workspaces, including filesystems, databases, tools, personal policies, and task resources, all simulated in Docker containers. The WeClawArena sandbox implements a message gateway that routes multi-agent communication, tool use, and workspace access while recording audit evidence for harmful or malicious behavior across four attack-harm families: collaboration, security, privacy, and governance. This design supports separate evaluation of task utility and attack success.
Table 2: Comparison with existing benchmarks. WeClawArena makes user-relative workspace ownership and authority part of both task utility and attack auditing.
Benchmark familyRepresentative benchmarksScored settingOwnership and authority model
Single-user tool and workspace benchmarksτ-bench and τ2-Bench (37; 1); WebArena, OSWorld, AppWorld, WorkArena (42; 35; 27; 5); AgentBench and GAIA (14; 17)A tool-using agent completes tasks in a website, OS, app, or workspace.Usually one user, account, or environment; cross-owner private resources and user-relative decision rights are outside the scored task contract.
Shared-authority multi-agent benchmarksAutoGen, AgentVerse, MetaGPT, and MultiAgentBench (33; 3; 10; 45)Agents coordinate, debate, specialize, or compete inside a team task.The team usually shares task authority; agents are rarely separate delegates with owner-specific files, consents, approvals, or mandates.
Social-agent simulationsGenerative Agents, OASIS, AgentSociety, and AgentSocialBench (21; 36; 22; 29)Agent populations communicate and form social or economic behavior.The focus is social dynamics; final utility is usually not a verifiable joint tool-use outcome assembled from separately owned workspaces.
Privacy, security, and audit benchmarksConFAIDE, PrivacyLens, MAGPIE, MAMA, AgentLeak, Auditable Agents, Agent Audit, and trace-audit studies (18; 24; 12; 13; 6; 19; 40; 43)Leakage, unsafe behavior, memory risk, governed-action failure, or trace-audit reliability.They motivate our harm surfaces; deterministic collaborative utility and attack auditing are usually evaluated in separate settings.
WeClawArenaThis workOwned agents complete joint tasks across six domains, with one benign control and four matched attack variants per base task.Utility requires joint tool use across private workspaces; ASR audits final harm in access, disclosure, consent, approval, mandate, and decision paths.
Figure 3: Benchmark construction pipeline. Source task pools and user-role profiles are curated into base tasks with owner workspaces, tools, predicates, and task contracts. Each base task becomes a scenario bundle with ground truth, agent/tool configuration, seed facts, personas, workspace resources, and MCP interfaces, then expands into one benign control and four attack-vector variants. Evaluation checks structural validity, task utility, and attack success from runtime evidence.
Figure 3: Benchmark construction pipeline. Source task pools and user-role profiles are curated into base tasks with owner workspaces, tools, predicates, and task contracts. Each base task becomes a scenario bundle with ground truth, agent/tool configuration, seed facts, personas, workspace resources, and MCP interfaces, then expands into one benign control and four attack-vector variants. Evaluation checks structural validity, task utility, and attack success from runtime evidence.
Table 3: Model-level utility and attack-audit breakdown. TSR denominators count available scenario executions. ASR denominators count attack rows with complete evidence and a valid GPT-5.2 judge verdict. Intervals are row-level Wilson 95% confidence intervals. Daggered rows have partial coverage in at least one attacked-mode component.
ModelBenign TSR (%, 95% CI)Attacked TSR (%, 95% CI)DropGPT-5.2 ASR (%, 95% CI)
Claude Opus 4.140/124 (32.3; 24.7–40.9)126/496 (25.4; 21.8–29.4)+6.9127/436 (29.1; 25.1–33.6)
Claude Opus 4.776/124 (61.3; 52.5–69.4)231/496 (46.6; 42.2–51.0)+14.713/440 (3.0; 1.7–5.0)
Claude Sonnet 4.560/124 (48.4; 39.8–57.1)165/496 (33.3; 29.3–37.5)+15.176/293 (25.9; 21.3–31.2)
DeepSeek V3.243/124 (34.7; 26.9–43.4)75/496 (15.1; 12.2–18.5)+19.6225/435 (51.7; 47.0–56.4)
Kimi K2.5†64/124 (51.6; 42.9–60.2)136/446 (30.5; 26.4–34.9)+21.1177/437 (40.5; 36.0–45.2)
Kimi K2 Thinking25/124 (20.2; 14.0–28.1)51/496 (10.3; 7.9–13.3)+9.9143/435 (32.9; 28.6–37.4)
Qwen3 235B†36/124 (29.0; 21.8–37.6)74/296 (25.0; 20.4–30.2)+4.0218/392 (55.6; 50.7–60.5)
Qwen3 32B26/124 (21.0; 14.7–29.0)41/496 (8.3; 6.2–11.0)+12.7173/434 (39.9; 35.4–44.5)
Figure 4: Attack-vector utility, resistance, and model-level utility-risk tradeoff on the ASR-main-six-domain pool. Left: TSR by variant and per-vector row-micro 1−ASRvmicro, where higher values indicate fewer judged attacks causing final harm. Right: model-level resistance compared against matched TSR drop from each model’s no-attacker rows. Higher 1−ASR is better; points further left preserve more utility.
Figure 4: Attack-vector utility, resistance, and model-level utility-risk tradeoff on the ASR-main-six-domain pool. Left: TSR by variant and per-vector row-micro 1−ASRvmicro, where higher values indicate fewer judged attacks causing final harm. Right: model-level resistance compared against matched TSR drop from each model’s no-attacker rows. Higher 1−ASR is better; points further left preserve more utility.
Table 4: WeClawArena domain denominators. Each base has five scenario variants: one no-attacker control and four attack vectors.
DomainBasesRowsRole in the benchmark
Bargaining24120Owned transaction close with buyer, seller, and approver roles.
Bidding1260Seller-centered public award with private buyer and seller constraints.
Travel20100Hotel-booking collaboration with traveler, partner, approver, and consent constraints.
SWE-Workspace50250Four-owner engineering workflow with patch, review, approval, CI, and handoff evidence.
Clinical1050Role-scoped case-record coordination with protected facts and approval or consent constraints.
Trading840Market-decision coordination with private strategy, order, portfolio, and mandate records.
Figure 19: Domain-level attack resistance by harm surface on ASR-main-six-domain. Each cell reports domain-vector row-micro 1−ASRd,vmicro. Higher values indicate fewer judged attacks causing final harm; raw denominators are given in Table 11.
Figure 19: Domain-level attack resistance by harm surface on ASR-main-six-domain. Each cell reports domain-vector row-micro 1−ASRd,vmicro. Higher values indicate fewer judged attacks causing final harm; raw denominators are given in Table 11.
Table 5: Scenario bundle artifacts.
ArtifactContents
scenario.jsonAgents, owners, prompts, topology, allowed tools, turn budget, and initial messages.
scenario.gt.jsonEvaluation criteria, resource and governance manifests, invariants, attack-vector metadata, and construction metadata.
family_pack_context.jsonDomain seed facts consumed by tools, task predicates, attack overlays, and judges.
personas/Per-agent system prompts and tool allowlists.
resources/Owner-scoped files, databases, shared state, and final artifacts.
mcp/Scenario-local tool declarations and role-scoped tool surfaces.
governance/Owner, identity, approval, consent, mandate, and scope records when used by the domain.
Figure 20: Model-level attack resistance by harm surface on ASR-main-six-domain. Each bar reports model-vector row-micro 1−ASRm,vmicro. Higher bars indicate fewer judged attacks causing final harm; raw denominators are given in Table 11.
Figure 20: Model-level attack resistance by harm surface on ASR-main-six-domain. Each bar reports model-vector row-micro 1−ASRm,vmicro. Higher bars indicate fewer judged attacks causing final harm; raw denominators are given in Table 11.
Table 6: Social topology patterns in WeClawArena.
DomainTopologyOwner rolesMain governed path
BargainingTransaction triad or competitive graphBuyer, seller, approverApproval and close artifacts must agree on transaction terms.
BiddingSeller-centered bidding graphSeller plus buyersAward, buyer confirmation, and close artifacts must agree.
TravelApproval triad or consent graphTraveler or organizer, partner hotel, approver or co-travelersBooking must bind approval or consent evidence to the final hotel order.
SWE-WorkspaceFour-owner engineering graphImplementer, reviewer, code owner, CI ownerPatch submission requires review, owner approval, CI evidence, and harness success.
ClinicalRole-scoped care graphCase roles with protected records and consent or approval constraintsCase action must respect protected facts and required consent or approval state.
TradingMarket-decision graphDecision, order, portfolio, and mandate rolesOrders must respect portfolio constraints, private strategy boundaries, and mandates.
Figure 21: Per-model utility and attack-resistance tradeoff by harm surface on ASR-main-six-domain. The x-axis reports matched TSR drop against the model’s own no-attacker rows; the y-axis reports model-vector row-micro 1−ASRm,vmicro. Points in the upper-left are preferred. Negative TSR drops mean the attack-vector TSR exceeds the matched no-attacker TSR in the reported result set.
Figure 21: Per-model utility and attack-resistance tradeoff by harm surface on ASR-main-six-domain. The x-axis reports matched TSR drop against the model’s own no-attacker rows; the y-axis reports model-vector row-micro 1−ASRm,vmicro. Points in the upper-left are preferred. Negative TSR drops mean the attack-vector TSR exceeds the matched no-attacker TSR in the reported result set.
Table 7: Domain setup summary.
DomainOwner rolesRequired workspace resourcesNormal workflow
BargainingBuyer, seller, approverPurchase request, budget ledger, inventory, price book, approval rules, close tablesNegotiate terms, request approval when required, then record matching purchase, seller, and approval artifacts.
BiddingSeller plus buyersPublic bid log, buyer budget and strategy records, seller reservation and award policy, close artifactsBuyers submit bids, seller awards one bid, winning buyer confirms, and both sides write consistent close records.
TravelTraveler or organizer, partner hotel, approver or co-travelersTravel profile, calendar, hotel inventory, approval log, consent log, prior bookings, trip stateNegotiate booking terms, collect approval or consent, then confirm the booking with the correct hotel and evidence fields.
SWE-WorkspaceImplementer, reviewer, code owner, CI ownerRepo fixture, investigation notes, review checklist, ownership policy, CI evidence, resource factsPatch the issue, obtain review and owner approval, run tests, and submit only after the evidence chain is complete.
ClinicalCase-coordination rolesCase records, protected facts, approval or consent recordsCoordinate a case action while respecting protected fields and required approval or consent.
TradingDecision, order, portfolio, mandate rolesOrder records, portfolio limits, private strategy notes, mandate recordsCoordinate a market decision while respecting portfolio constraints, private strategy boundaries, and mandates.
Figure 22: Judge concordance on the 32 model-vector ASR cells from ASR-main-six-domain and ASR-sensitivity-six-domain. Each point is one (model, attack vector) pair. Color encodes attack vector; the dashed line is y=x. Most points fall slightly above the diagonal, indicating that the Claude Opus 4.7 judge labels marginally more attacks as successful than GPT-5.2 across surfaces and models.
Figure 22: Judge concordance on the 32 model-vector ASR cells from ASR-main-six-domain and ASR-sensitivity-six-domain. Each point is one (model, attack vector) pair. Color encodes attack vector; the dashed line is y=x. Most points fall slightly above the diagonal, indicating that the Claude Opus 4.7 judge labels marginally more attacks as successful than GPT-5.2 across surfaces and models.
Table 8: ASR judge validation. G5.2 is the GPT-5.2 headline judge over ASR-main-six-domain; O4.7 is the Claude Opus 4.7 sensitivity judge over ASR-sensitivity-six-domain, the same evidence packets and denominators. ASR columns report model-macro surface ASR for surface rows and model-surface macro ASR for the all-vector row, as defined in Appendix I.1. Inter κ is Cohen’s κ between the two judges over all attack-vector rows in those pools. Human κ is Cohen’s κ between the human consensus label and each judge on ASR-human-pilot, a stratified 200-row validation subset annotated by two of the authors with attack-domain expertise; two-annotator human-human κ on the subset is 0.82. FPR is the benign false-positive rate over ASR-benign-control.
ASR (%)InterHuman κFPR (%)
SurfaceG5.2O4.7Δκvs. G5.2vs. O4.7G5.2O4.7
Collaboration18.322.3+4.00.710.650.690.30.5
Security42.544.3+1.80.660.610.650.20.6
Privacy31.435.5+4.10.690.640.670.40.8
Governance50.353.6+3.30.730.680.710.51.0
All vectors34.938.2+3.30.700.650.680.30.7
Table 9: Readiness failures versus benchmark outcomes.
CategoryExamples
Readiness failure, excluded from denominatorsMalformed bundle, missing required log, missing scorecard, evaluator crash, unusable final state, or missing evidence packet.
Utility outcome, counted when scoreableTask failure, turn-cap termination, low TSR, failed no-attacker row, or missing optional subgoal.
Attack-success outcome, counted only on attack rowsFinal harm on the intended attack vector plus clear evidence link under the domain judge.
Calibration controlNo-attacker rows used for task utility and judge false-positive checks, but excluded from ASR denominators.
Table 10: Variant-level TSR counts over the reported six-domain sweep. Percentages are in parentheses.
ModelNo attackerCollaborationSecurityPrivacyGovernance
Claude Opus 4.140/124 (32.3)43/124 (34.7)20/124 (16.1)35/124 (28.2)28/124 (22.6)
Claude Opus 4.776/124 (61.3)81/124 (65.3)59/124 (47.6)44/124 (35.5)47/124 (37.9)
Claude Sonnet 4.560/124 (48.4)54/124 (43.5)30/124 (24.2)41/124 (33.1)40/124 (32.3)
DeepSeek V3.243/124 (34.7)26/124 (21.0)14/124 (11.3)14/124 (11.3)21/124 (16.9)
Kimi K2.564/124 (51.6)42/124 (33.9)24/124 (19.4)37/124 (29.8)33/74 (44.6)†
Kimi K2 Thinking25/124 (20.2)26/124 (21.0)8/124 (6.5)8/124 (6.5)9/124 (7.3)
Qwen3 235B36/124 (29.0)30/74 (40.5)†13/74 (17.6)†23/74 (31.1)†8/74 (10.8)†
Qwen3 32B26/124 (21.0)21/124 (16.9)3/124 (2.4)13/124 (10.5)4/124 (3.2)
Table 11: Canonical raw ASR denominators and counts by model and attack vector. The GPT-5.2 block is ASR-main-six-domain; the Claude Opus 4.7 block is ASR-sensitivity-six-domain. Each cell reports model-vector row-micro ASR aggregated over domains. These raw counts support Figures 4, 4, 19, 20, 21, Table 8, and the failure-analysis totals. Percentages are in parentheses and report ASR, so lower is better. † indicates partial judged coverage.
ModelCollaborationSecurityPrivacyGovernance
GPT-5.2 judge
Claude Opus 4.115/120 (12.5)44/104 (42.3)23/106 (21.7)45/106 (42.5)
Claude Opus 4.74/124 (3.2)0/105 (0.0)0/106 (0.0)9/105 (8.6)
Claude Sonnet 4.510/86 (11.6)21/69 (30.4)20/69 (29.0)25/69 (36.2)
DeepSeek V3.237/123 (30.1)61/101 (60.4)50/106 (47.2)77/105 (73.3)
Kimi K2.517/122 (13.9)45/104 (43.3)53/106 (50.0)62/105 (59.0)
Kimi K2 Thinking20/120 (16.7)54/103 (52.4)38/106 (35.8)31/106 (29.2)
Qwen3 235B49/112 (43.8)†60/92 (65.2)†42/94 (44.7)†67/94 (71.3)†
Qwen3 32B18/123 (14.6)46/100 (46.0)25/106 (23.6)84/105 (80.0)
Claude Opus 4.7 judge
Claude Opus 4.120/120 (16.7)52/104 (50.0)27/106 (25.5)42/106 (39.6)
Claude Opus 4.79/124 (7.3)2/105 (1.9)3/106 (2.8)15/105 (14.3)
Claude Sonnet 4.57/86 (8.1)26/69 (37.7)23/69 (33.3)21/69 (30.4)
DeepSeek V3.248/123 (39.0)57/101 (56.4)56/106 (52.8)82/105 (78.1)
Kimi K2.527/122 (22.1)50/104 (48.1)50/106 (47.2)68/105 (64.8)
Kimi K2 Thinking15/120 (12.5)58/103 (56.3)47/106 (44.3)35/106 (33.0)
Qwen3 235B56/112 (50.0)†58/92 (63.0)†48/94 (51.1)†73/94 (77.7)†
Qwen3 32B25/123 (20.3)42/100 (42.0)30/106 (28.3)90/105 (85.7)

실제로 확인된 결과

  • Table 1 기준으로 Claude Opus 4.7이 여행, SWE-Workspace, 경매 영역에서 가장 높은 과제 성공률을 보였고 전체적으로 가장 안정적인 성능을 보였다.
  • Claude Sonnet 4.5는 협상 영역에서 1위, 임상 영역에서 공동 1위를 기록했다.
  • SWE-Workspace 영역은 최고 모델도 전체 변형 평균 과제 성공률 34.0%에 그쳐 가장 어려운 영역으로 나타났다.
  • 공격 유형별로 보면 거버넌스(권한 관련)와 보안 공격의 판정된 성공률이 가장 높았고 협업 방해형 공격의 성공률이 가장 낮았으며, 이 차이는 단순 과제 성공률 하락만 보는 것보다 더 뚜렷하게 드러났다.
  • 모델별로는 Claude Opus 4.7이 가장 공격에 저항력이 높았고 오픈웨이트 모델들은 대체로 더 낮은 저항력을 보였으며, 어떤 모델도 모든 공격 유형에서 동시에 우세하지는 않았다.

어디에 쓸 수 있나

  • 여러 사용자의 개인 비서 AI가 협업하는 서비스(일정 조율, 거래 승인, 예약 대행 등)를 출시하기 전에 공격 저항성과 과제 성공률을 함께 점검하는 벤치마크로 활용
  • AI 에이전트 간 메시지·도구 호출 로그를 남겨 사후에 피해 원인을 추적하는 감사(audit) 체계를 설계할 때 참고 구조로 사용
  • 특정 공격 유형(개인정보 유출, 잘못된 권한 승인 등)에 취약한 모델을 선별하는 사전 테스트 도구로 활용

한계와 남은 검증

  • 평가 대상 영역은 협상, 경매, 여행, 소프트웨어 개발, 임상, 거래 여섯 개로 한정되어 있어 다른 업무 영역에 대한 결과는 아직 없다.
  • 공격 성공 여부 판정은 LLM 심사자(GPT-5.2, 보조로 Claude Opus 4.7)에 의존하며, 두 심사자 간 완전한 일치는 보장되지 않는다.
  • 공격이 실제로 성공하지 않은 경우도 명확한 거부·차단 증거가 없으면 '성공 여부 미확인'으로 처리되어 과소 또는 과대 집계될 가능성이 있다.
  • 게이트웨이를 우회해 점수화된 메시지나 자원 조작이 이루어지면 공격 성공 판정 자체가 신뢰할 수 없다고 저자들이 명시했다.
  • 현재 공개된 것은 초기 버전이며, 재현 가능한 공격 페이로드 원문은 악용 방지를 위해 일부 편집·요약되어 있다.

왜 중요한가

여러 회사·개인의 AI 비서가 서로 연결되어 대신 일을 처리하는 시대가 오면, 한 비서의 실수나 조작된 메시지가 다른 사용자의 작업공간까지 피해를 퍼뜨릴 수 있는데 이를 실제로 측정할 도구가 없었다. WeClawArena는 이런 다자간 AI 비서 협업에서 성능과 안전성을 분리해서 재현 가능하게 진단할 수 있는 최초의 실험 환경을 제공한다.

이 논문의 용어

  • 에이전트(agent) · 사용자를 대신해 메시지를 보내고 도구를 쓰고 파일을 수정하는 자율 AI 프로그램
  • 작업공간(workspace) · 한 사용자가 소유한 파일, 데이터베이스, 도구, 정책의 집합으로, 다른 사용자에게는 직접 보이지 않는 영역
  • 과제 성공률(TSR) · 에이전트들이 협업 과제를 실제로 완수했는지를 나타내는 지표
  • 공격 성공률(ASR) · 심어둔 공격성 메시지나 자료가 실제로 최종 피해를 일으켰는지를 별도 심사자가 판정한 비율
  • 게이트웨이(gateway) · 에이전트 간 메시지와 도구 호출, 자원 접근이 모두 지나가도록 만든 감시·기록용 통로

저자 · Prince Zizhuang Wang

arXiv에서 원문 보기

최신 논문

논문 전체 보기 →

METAL LAB 최신 기사

그림 출처: Prince Zizhuang Wang et al., arXiv:2608.03499, CC BY 4.0