Scientific Data Skills: Enabling Agent-Ready Scientific Data Services at Scale

arXiv:2608.196252026-08-21

Packaging scientific datasets as ready-to-use 'skill' files so AI agents can find and understand them on their own

AI agents struggle to find, understand, and use scientific datasets because the data are scattered across many repositories and documented mainly for human readers. The researchers built the Scientific Data Skill (SciDSK), a standardized package combining dataset descriptions, file structure, usage steps, quality checks, and provenance, along with a pipeline to build it and an online platform, the Scientific Data Skill Bank, hosting resources across six scientific fields. In tests, agents using SciDSK found relevant datasets more accurately and interpreted their contents more precisely than agents using conventional dataset records.

What they did

Problem: scientific data are fragmented across many repositories and documented for humans, making it hard for AI agents to autonomously discover datasets, understand their file structure, and apply them correctly to a task
Solution: SciDSK bundles dataset description, scientific context, file organization, usage procedures, quality checks, and provenance into one reusable 'agent skill' package, while the actual data stays in its original repository and is only linked, not copied
Design: each SciDSK uses a SKILL.md file with a machine-readable YAML header (name, description, identifiers) for discovery, plus a human-readable body (overview, access steps, field details, usage guidance, quality checks, provenance) for interpretation, built through a three-stage pipeline grounded in authoritative source materials and checked before publication
Platform: the Scientific Data Skill Bank (scidsk.cn) publishes curated SciDSK resources across six disciplines -- physics, chemistry, earth sciences, biology, materials science, and computer science -- each assigned a persistent identifier (CSTR) that traces back to its source dataset
Results: on a 104-query retrieval benchmark, an agent using registered SciDSK skills (Agent-SciDSK) beat conventional metadata search and plain-text SciDSK search on every metric; in four interpretation case studies covering CT scans, population-density maps, table images, and social-media event data, it satisfied 23 of 24 evaluation criteria versus 22 for the conventional dataset-page approach

Figure 1: Key challenges in enabling AI agents to discover, interpret, and invoke scientific data.

Table 1: Comparison of SciDK with existing dataset- and agent-oriented concepts.
	Dataset Metadata	Dataset Card	Agent Skill	Tool/MCP	SciDSK(Ours)
Dataset description	✓	✓	×	×	✓
Scientific context	Partial	✓	Partial	×	✓
Task knowledge	×	Partial	✓	×	✓
Operational guidance	×	Partial	✓	✓	✓
Agent discovery	Partial	Partial	✓	×	✓
Dataset invocation	×	×	Partial	✓	✓

Figure 2: Schematic structure of a Scientific Data Skill.

Table 2: Overall discovery performance on the test sets. All values are percentages, and the best result is shown in bold.
Method	Hit@1	Recall@5	MRR	nDCG@5
BM25-Raw	47.12	69.23	57.59	59.12
Agent-Raw	71.15	90.38	79.04	81.90
Agent-SciDSK-Text	70.19	90.38	79.01	81.92
Agent-SciDSK	80.77	94.23	86.41	88.40

Figure 3: Construction pipeline of a Scientific Data Skill.

Table 3: Results on the four dataset interpretation cases.
Evidence condition	Coverage (%)	Satisfied criteria
ScienceDB page	91.67	22/24
Scientific Data Skill	95.83	23/24

Figure 4: The online Scientific Data Skill Bank, which can be visited at https://scidsk.cn/.

Table 4: Comparison on the CT skull reconstruction case.
Evidence condition	TIFF sequence interpretation	Reported file-count handling	Pre-use checks	Coverage
Dataset information page	Incorrectly described the visible sequence as “200+ slices”	Noted that the visible file tree may be incomplete	Provided general checks for the image sequence and parameters	5/6
Scientific Data Skill	Correctly identified 196 consecutive slices	Distinguished the reported total from the visible portion while preserving uncertainty	Specified continuity, non-empty-file, readability, parameter-file, and directory-completeness checks	6/6

Table 5: Comparison on the Weibo rumor-event mapping case.
Evidence condition	File organization	Cross-file relationship	Validation checks	Coverage
Dataset information page	Identified events.txt as the label file and posts.zip as containing event-organized JSON content	Linked event identifiers to event-named JSON files but left corpus-wide completeness unresolved	Suggested general coverage and schema checks without explicit count and label-domain validation	5/6
Scientific Data Skill	Identified 4,664 labeled events and clearly distinguished the roles of the two files	Specified a one-to-one mapping between event records and event-named JSON files	Required count validation, binary-label checks, archive extraction, orphan detection, and sampled content verification	6/6

Why it matters

For AI agents to be useful in data-driven scientific research, they need to find the right dataset, correctly interpret what each file means, and know how to use it properly -- something existing documentation was not built to support. This work offers both a concrete specification and a working platform that could serve as real infrastructure for making scientific data usable by autonomous AI systems.

Figure 5: Human annotation interface for reviewing discovery benchmark queries.

Terms in this paper

AI agent · an AI system that can plan and carry out multi-step tasks autonomously, often using external tools
SciDSK (Scientific Data Skill) · a packaged representation bundling a dataset's knowledge and usage instructions for reuse by AI agents
Agent Skill · a modular, reusable package that gives an AI agent task-specific knowledge and instructions
FAIR principles · guidelines making data Findable, Accessible, Interoperable, and Reusable
CSTR · a persistent identifier assigned to a resource so it can be independently identified and cited

Original abstract (English)

Scientific data are increasingly used by AI agents, yet existing dataset representations provide limited support for autonomous discovery, interpretation, and invocation. This limitation stems from the fragmentation of scientific data across heterogeneous repositories and from dataset representations designed primarily for human use. To address this limitation, we introduce the Scientific Data Skill (SciDSK), an agent-ready representation that packages dataset-specific knowledge and operational guidance as a reusable agent skill. A SciDSK integrates dataset descriptions, scientific context, file organization, usage procedures, quality checks, and provenance information while retaining the underlying data in its original repository. We define a structured SciDSK specification and develop a systematic construction pipeline that grounds each SciDSK in authoritative dataset records and associated supporting materials. We further establish the Scientific Data Skill Bank, a unified platform that publishes SciDSK resources across six scientific disciplines and supports package access, persistent identification, and traceability to source datasets. We evaluate SciDSK through a retrieval benchmark for dataset discovery and controlled cases for dataset interpretation. The results show that SciDSK improves agent-driven dataset discovery and provides more precise and actionable support for dataset interpretation. These findings support the value of organizing dataset-specific knowledge in an agent-ready representation.

Authors · Xiaohan Huang, Qingqing Long, Xiaolei Du, Siyu Pu, Jiawen Xu, Haotian Chen, Chenyang Zhao, Jinbiao Liu, Xuezhi Wang, Hao

Read on arXiv

Latest papers

All papers →

Latest from METAL LAB

Figures: Xiaohan Huang et al., arXiv:2608.19625, arxiv-nonexclusive