Specification-delta-driven data governance: an empirical study of the {\guillemotleft}spec-delta{\guillemotright} as the unit of change in lakehouse data platforms

arXiv:2608.198382026-08-21

Treating data-platform changes like reviewable spec snippets instead of code diffs: an experiment design paper

This paper works out the idea of a 'spec-delta' — a small, reviewable chunk of requirements attached to a change in a data lakehouse (a layered data platform that refines raw data into clean, analysis-ready data) — as an alternative to reviewing changes purely as code pull requests. The author formalizes what a spec-delta looks like, builds a taxonomy of which kinds of data-platform changes benefit from this approach, and designs a controlled experiment comparing spec-delta-driven review against conventional code-PR review. Importantly, this is a design paper: the actual laboratory run and results are explicitly left for future work.

What they did

Building on Spec-Driven Development tools like GitHub Spec Kit and OpenSpec, which argue specifications rather than code should be the primary reviewed artifact, the author extends this idea to data platforms, where many changes are 'contractual' (new datasets, SLAs, metric definitions, access policies) rather than pure code edits.
A spec-delta is defined with five parts: SHALL requirements (using MUST/SHOULD/MAY levels), GIVEN/WHEN/THEN acceptance scenarios, an architecture decision record, a validation checklist, and expected artifacts — and the paper proposes that no change should be promoted to the Silver or Gold data layers (the cleaned and business-ready layers in a medallion lakehouse) without an approved spec-delta.
A taxonomy classifies change types by suitability: new data products, metric-semantics changes, SLA changes, and access-policy changes are rated high suitability for spec-deltas (they're 'contractual'), schema evolution and quality rules are medium ('mixed'), while internal refactors and performance tuning are rated low suitability since they're pure code changes.
The proposed experiment uses a within-subject crossover design where the same participants do equivalent tasks under both the spec-delta workflow and the conventional code-PR workflow, measuring discovery-to-deployment time, defect density reaching Silver/Gold, divergence of the same metric computed across different BI tools, and reviewer cognitive load via the NASA-TLX questionnaire.
A concrete reference lab environment is specified (Azure Databricks, Unity Catalog, dbt Semantic Layer, Great Expectations, OpenLineage, Power BI/Microsoft Fabric) along with eight example change tasks and one fully worked spec-delta example for redefining a profit-margin metric, but the author explicitly states the actual lab run and results are reserved for future execution, not reported here.

Why it matters

Data teams often struggle to judge, from a code diff alone, which downstream consumers or guarantees a change to a dataset, SLA, or metric definition affects; this work proposes a concrete way to make such contractual changes reviewable and testable, plus an experimental framework to find out when it's actually worth the extra specification effort. Once run, the proposed experiment could give practical evidence for which types of data-platform changes deserve upfront spec review versus which don't, helping avoid both silent data-quality failures and wasted over-specification effort.

Terms in this paper

spec-delta · a minimal, self-contained increment of specification attached to one change, used as the unit reviewed for approval instead of the raw code
medallion lakehouse architecture · a data storage pattern with Bronze (raw), Silver (cleaned/validated), and Gold (aggregated, business-ready) layers of increasing quality
data contract · a formal agreement between data producers and consumers on schema, semantics, and quality, enforced at run time
NASA-TLX · a standardized questionnaire (NASA Task Load Index) used to measure a person's subjective mental workload after a task
SHALL requirements · normative statements using RFC 2119 levels (MUST/SHOULD/MAY) that specify how strictly a guarantee must be met

Figures we cannot republish

Figure 0

See the figures in the original paper →

Original abstract (English)

Spec Driven Development SDD has consolidated the idea that the specification rather than the code should be the primary artefact governing AI assisted work. Tools such as GitHub Spec Kit, and proposals such as Constitutional SDD, have formalised this principle in the software domain, while the executable data-contracts literature has extended it to schema and quality enforcement at run time. Nevertheless, the treatment of the specification delta OpenSpec's core idea that every change should produce a reviewable increment of requirements as the unit of change in data platforms remains empirically unexplored, even though many data-platform changes are contractual (new datasets, service-level agreements, metric semantics, access policies) rather than purely code changes. This work formalises the spec-delta concept, proposes a taxonomy of data platform changes according to their suitability for incremental specification, and defines a controlled experiment comparing a spec-delta-driven workflow against a conventional code pull-request workflow without a delta. The response variables are discovery to deployment time, the density of defects reaching the Silver and Gold lakehouse layers, cross-tool metric divergence, and reviewer cognitive load measured with NASA TLX. The paper explicitly reserves a demonstration-and-laboratory section for instantiation on a real lakehouse environment. The contribution is not a tool but reproducible evidence and an applicability guide that helps to avoid the up front over specification antipattern.

Authors · Pablo Ramirez Amador

Read on arXiv

Latest papers

All papers →