Symposium: Trust via Auditable Records for Communities of AI Scientist Agents
A record-keeping system that stops AI research agents from quietly faking or hiding how they got their results
Symposium is a framework that gives communities of AI agents doing science an unchangeable publication history, called the CommunityRecord, covering their analyses, hypotheses, data, and arguments. Every published item is tied to a specific author and date, and only content explicitly declared as evidence can be cited as evidence, so agents cannot smuggle in unverified support for a claim. The author released a working implementation of the publication infrastructure, agent prompt components, and documentation so others can set up their own Symposium community.
What they did
- The motivation is that AI agents can now do literature analysis, experiment planning, data analysis, and hypothesis generation faster than humans, but they can also fabricate facts or citations, misread sources, or produce persuasive-looking hypotheses built on weak reasoning or cherry-picked data
- It targets communities such as a lab, project, or company that will use many different, changing AI agents over time, where agents need to build on each other's work while still being able to critically review it
- The design borrows from scientific publishing: Members (typically agents) publish immutable units called Artifacts to the CommunityRecord, and corrections or retractions are handled by publishing new Artifacts rather than editing old ones
- The central Artifact type, Argument, structures claims (Assertions), evidence links (Grounds), and explicit Assumptions, and enforces that only content a publisher has explicitly marked as usable evidence (groundable) can be cited to support a claim
- Trust is treated not as a single true/false score but as a judgment tied to a stated purpose and stakes, and the paper provides a working publication infrastructure, agent prompt components, and documentation for setting up a real Symposium community
Why it matters
As AI agents start generating dozens or hundreds of hypotheses and analyses per day, without a shared, auditable record the errors and misaligned incentives of individual agents can compound across a whole research community. This framework offers a way to keep that fast-moving output inspectable by both humans and future, more capable agents, independent of which specific AI systems are being used.
Terms in this paper
- Artifact · A single published unit of work in Symposium, such as data, an analysis, or an argument
- CommunityRecord · The immutable, time-ordered history of all Artifacts published by a community's Members
- Argument · The main structured Artifact type that captures a claim along with its evidence and assumptions
- Ground · A citation object linking an Assertion to evidence that supports or bears on it
- groundable · A flag a publisher must set to declare that specific content may be cited as evidence
Original abstract (English)
Symposium is a formal framework and practical implementation to record the operation of AI agents deployed by small scientific research communities. Symposium provides long-term, immutable histories of agent-driven research activity, leaving auditable trails of analyses, hypotheses, data, and scientific discourse. This shared record of published artifacts enables agents to build on prior work and preserves the evidence researchers and agents need to make purpose-dependent trust assessments. Symposium captures scientific argument, including structured claims, fine-grained evidence citations, assumptions, and explicit declarations of what material may and may not be used as evidence. Symposium differs from AI co-scientist agents or integrated AI research environments; it is a framework that separates a scientific community's durable history from the agents and other systems that operate on that history. It assumes that a community will use diverse AI systems in a rapidly evolving environment. A working implementation of the publication infrastructure, agent prompt components, and documentation are provided to enable users to rapidly set up and run their own Symposium community.
Read on arXivLatest papers
- Specification-delta-driven data governance: an empirical study of the {\guillemotleft}spec-delta{\guillemotright} as the unit of change in lakehouse data platformsTreating data-platform changes like reviewable spec snippets instead of code diffs: an experiment design paper
- Are LLMs becoming similarly creative? Evidence from three years of modelsNewer AI chatbots are giving increasingly similar answers to each other, three years of data show
- Auditing Cross-Lingual Fairness in Language Model WatermarkingAI text watermarks that are supposed to catch machine-written content work far less reliably in many non-English languages, and the gap tracks language families, not individual languages
- TESTNAV: Pareto-Guided Search for Compositional Robustness TestingA smarter way to test AI models against combined real-world glitches, without checking every possible combination
- Optimal Skill Selection for LLM Agents with Provable Bicriteria GuaranteesA method that picks which 'skill documents' to feed an AI coding agent, with mathematically guaranteed near-optimal results
- Reliable Financial Named Entity Recognition under Domain ShiftAn AI's confidence trained on formal filings turns unreliable once it reads tweets
- FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM ServingMaking sparse attention fast enough and accurate enough for real LLM serving, not just papers
- Robust Incomplete Multimodal Sentiment Analysis via Iterative Proxy CorrectionWhen text input is missing or broken, this AI doesn't guess once and move on—it revises its guess step by step to read emotions more reliably
Latest from METAL LAB
- Google Discover adds chatbot that adjusts your feed based on spoken preferences
- OpenAI Closes In on Anthropic Again in Enterprise Spending Share
- Meta Unveils First 10 Tasks in WildArtifactBench, a Benchmark for AI Agents
- Musk: "Optimus + Grok will one day handle healthcare for all humanity"
- 35% of Web Pages Published Since ChatGPT Show Signs of AI Authorship
Figures: Dexter Pratt et al., arXiv:2608.19511, CC BY 4.0