A Table Is Not a Data Product: Ownership, SLIs, and the Consumer Contract

A table becomes a data product when someone can depend on it deliberately.

The distinction is not marketing. A table is a storage object. A data product is an interface with a purpose, an owner, a contract, measurable service levels, and a way to respond when reality deviates from the promise.

Over the last several weeks we have looked at the components:

• Contracts preserve meaning
• Idempotency makes reruns safe
• Observability detects bad data
• Schema evolution protects consumers
• Backfills preserve a recovery path
• Lineage, CDC, table formats, metadata, and cost controls make the state explainable and operable

Now we assemble those properties into a service that a consumer can use with confidence.

Start With the Consumer Decision

“Orders table” is not a purpose. A useful product definition names the decision it supports:

Finance uses daily_revenue to close the ledger by 11:00 UTC each business day. It contains captured payments, grouped by business date and ISO currency. It is not a real-time balance.

That sentence establishes grain, inclusion rules, timing, and a boundary. Without it, two teams can query the same table and reach different conclusions while both believe they are correct.

The Product Card

Keep the public contract compact and executable.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

name: finance.daily_revenue
purpose: Close the captured-payment ledger by business date and currency.
grain: one row per business_date and currency
owner:
  team: finance-data
  escalation: finance-data-oncall
  product_manager: finance-platform
inputs:
  - payments.captured_events
  - reference.iso_currencies
contract:
  version: 3
  compatibility: backward-compatible additions
  deprecation_notice_days: 14
service_levels:
  freshness:
    target: 60 minutes
    critical_after: 180 minutes
  completeness:
    target: 99.9 percent of expected partitions
  correctness:
    target: reconciliation variance below 0.1 percent
  availability:
    target: 99.5 percent of scheduled publications
recovery:
  replay_window: 30 days
  rollback_reference: warehouse.finance.daily_revenue_previous

The exact fields will differ by organisation. The important part is that the values are decisions someone is willing to defend.

SLIs Turn Promises Into Measurements

An SLI is an observed indicator. An SLO is the target for that indicator.

Freshness

1

freshness lag = current time − latest valid source event time

Measure both source freshness and publication freshness. A successful job that loaded stale input should not receive full credit.

Completeness

1

completeness = valid expected records or partitions / expected records or partitions

The denominator must be defined. A row count by itself cannot distinguish a quiet business day from a missing partition.

Correctness

Correctness should use domain invariants and reconciliation:

1
2
3

reconciliation variance
= abs(product total − trusted source total)
  / nullif(trusted source total, 0)

For finance, compare against a ledger. For logistics, compare shipment state transitions. For a feature table, compare distributions and key coverage. A generic “tests passed” percentage is not a substitute for the consumer’s real risk.

Availability

1

availability = successful publications / scheduled publications

Define what counts as a successful publication. A job that writes a staging table but never updates the consumer-facing view should not count as available.

Error Budgets Create a Decision Rule

If an SLO is 99.5% for 200 scheduled publications in a month, the error budget is one failed publication. The number is not a punishment. It tells the team when to spend time on reliability rather than adding another feature.

1

monthly budget = scheduled publications × (1 − SLO)

Use the budget to guide behaviour:

• Healthy budget: ship planned changes, improve the product, or retire unused checks.
• Budget nearly exhausted: prioritise root-cause work and reduce risky releases.
• Budget exceeded: pause non-essential changes until the service is stable.

Do not hide incidents by changing the denominator after the fact. If the schedule changes, version the SLO and explain the change.

Publication Is a Protocol

A data product should move through a visible release sequence:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

source event
    ↓
ingest and preserve
    ↓
transform and validate
    ↓
reconcile and record lineage
    ↓
publish consumer-facing state
    ↓
announce version and evidence
    ↓
observe consumers and error budget

Each boundary needs a failure policy. If reconciliation fails, do we block publication, publish with a warning, or route the output to a quarantine table? The correct answer depends on the product’s risk, but silence is never a policy.

The Consumer Contract Includes Change

Availability today does not protect a consumer from tomorrow’s breaking change.

Every product should state:

• Which fields are stable and which are experimental
• What a null means
• What timezone and units apply
• How historical rows are corrected
• How long old fields remain available
• Where consumers can see usage and migration status
• Who approves a breaking change

An additive column may be safe for many consumers. A semantic change from gross to net revenue is not. Schema evolution is part of service management, not a cleanup task after deployment.

The Operator’s Readiness Check

Before calling a dataset production-ready, ask:

If the answer is “we would need to ask someone,” the service is not yet ready for an important decision.

The Data Product Rule

A data product is a promise with evidence.

Give it a purpose, an owner, a contract, measurable service levels, and a recovery path. Then let pipeline runs, quality checks, lineage events, and usage history update the evidence automatically.

The mature platform is not the one with the most tables. It is the one where consumers know which tables to trust, operators know what to do when trust is lost, and both groups can see the same state.

References: OpenLineage overview for run and dataset metadata, BigQuery INFORMATION_SCHEMA.JOBS for operational evidence, and the data observability vital signs already in this archive.

This closes the current weekly run. The next useful question is not which tool to add, but which promise the system still cannot prove.