CDC Is Not a Sync Button: Log-Based Ingestion With Debezium

Change Data Capture (CDC) is often described as a way to keep a warehouse in sync with an operational database.

That description is convenient and incomplete.

CDC does not provide a magical copy of the current state. It gives you a stream of changes, an initial baseline, and a set of recovery obligations. The data team still has to decide how to apply inserts, updates, deletes, retries, schema changes, and out-of-order delivery.

The useful mental model is:

1
2
3
4
5

Operational database
        │
        ├── consistent snapshot ──────┐
        │                             ▼
        └── transaction log / WAL → raw change events → curated tables

This is the second post in the platform-state phase of the Reliable Data Systems series.

Why Polling Stops Scaling

A polling job asks the source database a question repeatedly:

1
2
3

select *
from orders
where updated_at > :last_seen_timestamp;

It looks simple, but the timestamp is not a complete change protocol.

• Two rows may have the same timestamp.
• A clock can move backward or have insufficient precision.
• Deletes may disappear completely from the query result.
• An update can commit between pages of a long scan.
• The source has to execute repeated range queries against production tables.

Log-based CDC reads the database’s committed change log instead. For PostgreSQL, Debezium uses logical decoding of the write-ahead log (WAL). For MySQL, the connector reads the binlog. The exact mechanism depends on the source, but the principle is the same: consume the database’s record of committed changes rather than repeatedly guessing what changed.

The Snapshot Boundary

The first CDC run needs a baseline. The log normally does not contain an indefinitely complete history, so a connector performs a consistent snapshot and then continues from the log position associated with that snapshot.

1
2
3
4

time ─────────────────────────────────────────────────────────▶
       snapshot starts       snapshot boundary       continuous log
       ├─────────────────────┤──────────────────────▶
       rows at one view      position recorded      INSERT/UPDATE/DELETE

The boundary is the critical property. If the snapshot and the log stream are not coordinated, changes committed during the snapshot can be lost or applied twice.

Debezium documents this as a two-phase lifecycle: read a consistent view, record the source position, then stream changes from that position. A restart may repeat work. The consumer must therefore be safe to replay, which is the same idempotency principle described in Run It Again.

A Change Event Is Not a Row

A CDC event carries operation metadata around the row image.

The important fields are:

Do not throw away source.lsn, transaction identifiers, or event timestamps when flattening the event. Those fields are how an operator proves ordering, measures lag, and resumes safely.

Three Useful Tables, Not One

A durable CDC landing zone usually keeps three representations.

1. The Raw Event Log

Append every source event with its envelope intact. This is the replay boundary and the audit record.

1
2
3
4
5
6
7
8

raw_cdc.orders
├── event_key
├── operation
├── before_json
├── after_json
├── source_lsn
├── source_commit_time
└── ingested_at

2. The Current-State Table

Apply the latest event for each primary key. This is convenient for analytics and dimensions, but it must handle deletes and duplicate events explicitly.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

merge into analytics.orders as target
using staged_latest_orders as source
on target.order_id = source.order_id
when matched and source.operation = 'd' then delete
when matched then update set
  status = source.status,
  updated_at = source.source_commit_time
when not matched and source.operation <> 'd' then insert (
  order_id, status, updated_at
) values (
  source.order_id, source.status, source.source_commit_time
);

3. The History Table

Keep each version when the business needs auditability, slowly changing dimensions, or reconstruction of a past view. The current-state table is a projection; it is not the history.

Deletes and Tombstones

Deletes are where many “working” CDC pipelines become wrong.

A delete event says that the row no longer exists. Kafka-compatible pipelines may also emit a tombstone event: the same key with a null value, allowing log compaction to remove older records for that key. A sink that treats both records as ordinary JSON rows can create a null-valued phantom record instead of removing the row.

The consumer contract should state:

• Which field is the stable primary key?
• Does a delete remove current state or only mark it deleted?
• Are tombstones expected and filtered?
• What happens if a delete arrives before the consumer has materialized the row?
• Can historical consumers replay the event log independently?

Monitor the Source, Stream, and Sink

CDC has failure modes at every boundary.

Alerting on connector health alone is not enough. A connector can be connected while its offset stops advancing, or it can be producing events while the sink rejects every delete.

The CDC Rule

CDC is an event-ingestion system with a consistent baseline, not a sync button.

Preserve the raw envelope. Treat the source position as data. Apply changes idempotently. Model deletes explicitly. Retain enough log history to recover, and measure lag from commit time to consumer publication.

References: Debezium overview, Debezium architecture, and Debezium PostgreSQL connector.

Next: what makes a collection of Parquet files behave like a table that can evolve, travel through time, and roll back safely.