HarnessRisk: A Lifecycle-Oriented Benchmark for Agent Harness Safety
Large language models are increasingly deployed through agent harnesses that manage tools, extensions, persistent state, permissions, and external actions. Existing safety benchmarks mainly target individual attack mechanisms or a limited subset of operational settings, making it difficult to compare how safety failures emerge across different harness responsibilities. We present HarnessRisk, a lifecycle oriented benchmark that organizes agent harness safety into six operational phases including
Read on arXivLatest papers
- FM-Bench: A Benchmark for Long-Horizon Management with Competing AgentsHanding an AI a football club to run for 20 years reveals that winning comes from management habits, not raw model power
- FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI SystemsAI often names the right cause of a financial mismatch without ever finding the proof for it
- FACET: Preserving Source Intent and Executable State in Terminal Task SynthesisFACET builds internally consistent terminal-task 'exam sets' to train command-line AI agents
- Alignment Is All You Need: Instruction-Free Training for General Audio-Language ModelsA frozen language model plus one lightweight connector is enough to build a capable audio-understanding AI
- Adversarial Review: Structured Disagreement for Grounded Agentic Code ReviewFor AI code review, one reviewer plus one critic beats piling on more agents
- Looped Language Models Improve Compositional Tool CallingAI models that rethink their own answers multiple times get better at chaining tools together
- Bridging Search and CRM: Productionizing AI Product Research Agents for Customer Re-EngagementAI agents win back window-shopping customers by chasing them down on WhatsApp
- GenEx: A Graph-Based Representational Paradigm for SARS-CoV-2 Variant Detection via Codon Co-occurrence NetworksTurning viral gene sequences into codon relationship maps to tell coronavirus variants apart
Latest from METAL LAB
- NVIDIA's 300 Verified Skills Lift Correctness by 41 Points
- Wave your hand at a webcam, hear a theremin: browser instrument released
- Meta AI launches desktop app for Mac, can read an entire app window
- Factory Commits $100M to Partner Network, Pushes to Scale Software Factories
- SpaceX approached Cognition for acquisition four days after closing Cursor deal