← EnergyScope blog
Storage · Table Format · Deep Dive

Apache Iceberg — and the snapshot-to-snapshot delta EnergyScope already computes by hand

Iceberg turns a directory of Parquet files into a table with ACID commits, time-travel, and safe schema change — no database. But the reason it matters here is narrower and sharper: EnergyScope's Dagster pipeline is a hand-rolled, weaker version of Iceberg's delta engine. This document explains the format, then goes deep on the one mechanism that would replace 200 lines of Python row-diffing with a metadata read.

01 Orientation

What Iceberg is

Apache Iceberg (born at Netflix, now an Apache top-level project) is an open table format — a metadata layer that sits on top of a pile of Parquet files in object storage and makes that pile behave like a real database table.

Before Iceberg, a "table" in a data lake was just a directory of Parquet you agreed to treat as one thing. That convention is brittle: you have to list the directory to know what's in it (slow and, on eventually-consistent object storage, sometimes wrong); you can't write to it atomically (a reader mid-write sees half a table); and you can't roll back a bad load. Iceberg replaces the directory-as-table convention with an explicit, versioned metadata tree — and gets ACID transactions, time-travel, and schema evolution as a result.

Crucially, Iceberg is not a query engine and not a database. It stores no data of its own beyond metadata, and runs no compute. Spark, Trino, DuckDB, Snowflake, Flink and pyiceberg all read and write the same Iceberg table. It is the shared contract, not the processor.

02 Motivation

vs a filesystem of Parquet

The obvious objection: EnergyScope already has Parquet files in folders — why is a table format better than the filesystem I've got? The answer is one reframing. A filesystem gives you files. It does not give you a table.

Everything Iceberg adds is the gap between "a folder of Parquet" and "a table." And here is the part that lands: everything EnergyScope hand-rolled — the .manifest.json, the prev/ copy, the drop-guard, the Python row-diff — is you building table semantics on a filesystem that doesn't have them. Those aren't features; they're symptoms of the missing layer.

The gapWhy a filesystem can'tEnergyScope's hand-rolled patch
Atomic publishYou can atomically replace one file (write-temp-then-rename), never a set of 168 year files. On R2 there's no rename at all — just copy+delete. A crash mid-write leaves a half-table.drop-guard + prev/ snapshot — patches over "there is no commit"
"Which files are the table?"The table is an implicit convention — "whatever .parquet is under here" — discovered by a slow, eventually-consistent LIST. Two versions can't cleanly coexist in one folder.the Flight autoload glob, re-deriving the layout at startup
What's inside a fileA file is opaque. To know if it holds WTI or covers 2026, you open it. A hash says that it changed, never what's in it.MD5-per-file manifest — detects change, can't prune
HistoryOverwrite destroys the old bytes. To keep history you copy directories — and you keep one.a single prev/ copy — "yesterday," nothing older
Which rows changedThe filesystem knows a file changed; it has no idea which rows.the Phase-2 to_pylist() + Python row loop
Schema changeNothing coordinates a column add / type change across 168 files; rename is by name/position, fragile.none — which is why the date32 migration is a risky coordinated rewrite
Self-descriptionThe "table" is tribal knowledge encoded in whatever reads it.the layout is hardcoded independently in the cook, the autoload, and the delta function
The honest other half

For what EnergyScope has right now — one writer (Dagster), one box (MS-02), local NVMe — a filesystem of Parquet genuinely works. That's not a mistake; it's the right call at this scale, and it's why it's what runs. The metadata layer earns its keep at specific thresholds: object storage (R2, where multi-file atomicity and listing bite), readers during writes (the reload gap), history / vintages (the product feature), multiple writers, or millions of files (planning cost). EnergyScope is now brushing against three of those — the R2 origin, the reload gap, and the vintage opportunity — which is exactly why Iceberg moved from "no" to "next."

The one-liner

The filesystem stores your bytes; a table format makes those same bytes behave like a database — atomic, versioned, self-describing, queryable through time — without being one. EnergyScope currently supplies a thin, weaker slice of that behaviour in Python, which is fine until the thresholds above, at which point hand-rolling it stops being cheaper than adopting it.

03 Capabilities

The four moves

Snapshots → ACID + time-travel

Every write is an atomic snapshot, committed by swapping one metadata pointer. Readers always see a consistent version; query AS OF any past timestamp; roll back a bad write by re-pointing to an old snapshot.

Manifests, not directory listing

Planning reads metadata manifests — each listing every data file with per-column min/max/null stats — instead of listing the filesystem. Fast on millions of files, correct on object storage where a listing can lie.

Hidden, evolvable partitioning

Partitions are derived from a column by a transform (days(period)) and tracked in metadata. Users query WHERE period > … and pruning is automatic. The partition scheme can change without rewriting old data.

Schema evolution by column ID

Columns carry stable IDs, not positions or names — so add / drop / rename / reorder is a metadata-only change that never breaks a reader or rewrites a file.

All four rest on the same foundation: a rich, versioned metadata tree describing exactly which files make up the table right now, and at every past commit. Understanding that tree is the key to the delta — so let's open it up.

04 Structure

Anatomy of a table

An Iceberg table is a four-level tree. Data at the bottom is ordinary Parquet; everything above it is small metadata that records history and statistics.

metadata.json ← table root: schema, partition spec, and thelist of ALL snapshots + the current pointer ├─ snapshot (per commit) — snapshot-id, timestamp, parent, summary │ │ │ └─ manifest list (Avro) — the manifests live in this snapshot │ │ │ └─ manifest file (Avro) — lists data files; per entry: │ ├─ status: ADDED │ EXISTING │ DELETED │ ├─ record_count │ └─ per-column min / max / null_count │ │ │ └─ data file (Parquet) ← your actual rows

Two things about this tree do all the work:

05 The core idea

The delta, in depth

Here is the one sentence the whole document turns on: Iceberg records the delta at write time, in metadata. EnergyScope's pipeline recomputes it at read time, by scanning.

When Dagster commits today's data into an Iceberg table, the commit itself knows which files are new versus carried over — and writes that fact into the manifest as the status field. So the question "what changed between yesterday's snapshot and today's?" is answered by reading the manifest entries with status = ADDED in the snapshots since yesterday. A few small Avro reads. You never open, materialize, or diff a single row of the old data.

That is the IncrementalAppendScan. In Spark it reads:

SELECT * FROM prices -- only files ADDED between two snapshots — pure metadata planning FOR SYSTEM_VERSION /* incremental */ .option("start-snapshot-id", yesterday) .option("end-snapshot-id", today)

Now contrast what the pipeline does today, in assets.py Phase 2 — the exact code from the review:

# reconstruct the delta by brute force, every run old = pq.read_table(prev_parquet).to_pylist() # entire prev year → Python new = pq.read_table(curr_parquet).to_pylist() # entire new year → Python lookup = {(r.series_id, r.period): r.value for r in old} for r in new: # O(all rows), in Python if (r.series_id, r.period) not in lookup: new_row() elif abs(lookup[...] - r.value) > 1e-4: revised_row()

The pipeline reconstructs the delta by brute force every run because nothing recorded it. Iceberg recorded it at commit time, so the read is O(changed files), not O(all rows). For a year partition where one price was revised, that's the difference between reading a few kilobytes of manifest and materializing millions of rows into Python dicts.

The asymmetry in one line

Write-time bookkeeping vs read-time scanning. Iceberg pays a tiny cost at commit (write a manifest entry) to make every future "what changed?" nearly free. The hand-rolled pipeline pays nothing at write and everything at read — a full re-scan and diff, on every dataset, every day.

Where the delta key comes from

The diff code raises a question: where is the key (series_id, period) actually defined? The honest answer is that it isn't declared anywhere — it's hardcoded as a Python tuple, once, in the body of _compute_delta:

old_lookup[(old_s[i], old_p[i])] = old_v[i] └──── this tuple literal IS the key definition ────┘

There is no config, no schema, no per-dataset setting. "A row's identity is (series_id, period); its payload is value" lives inside that one function and is applied identically to all 24 datasets. Nothing outside the diff loop knows the key exists, nothing enforces it, and changing it for one source means editing the code. The key is code, not metadata.

Iceberg inverts that. You declare the key once, in the table schema, as the table's identifier fields — the columns a revision's delete matches on:

# pyiceberg — the delta key is a first-class property of the TABLE schema = Schema( NestedField(1, "series_id", StringType(), required=True), NestedField(2, "period", StringType(), required=True), NestedField(3, "value", DoubleType()), identifier_field_ids=[1, 2], # ← the key, declared ONCE )

Now the key is a property of the table — discoverable through the catalog and honoured by every engine that touches it: the MERGE that writes equality-deletes, the changelog scan, a rollback. It can differ per dataset without a code change, and the engine enforces it (an equality-delete on (series_id, period) is only meaningful because those are the declared identifier fields).

Key as code vs key as metadata

Today you define the key — by hand, as a tuple, and only the diff loop knows it. Iceberg has you declare it once as schema metadata, and the whole system honours it. It's the same shift as the delta itself: move the fact out of imperative code and into versioned metadata, where every reader and writer can see and trust it.

06 The subtle case

Appends vs revisions — and how CDC handles both

EnergyScope's data hits two kinds of change, and the pipeline's new vs revised split maps exactly onto Iceberg's two mechanisms.

New periods each day → appends

A fresh daily observation is a new row in a new file, tagged ADDED. The cheap IncrementalAppendScan above handles it completely — read the added files, done.

EIA revising a past value → an update

This is the more interesting case — the pipeline's old_val != new_v[i] branch with the 1e-4 float tolerance. Iceberg v2 represents an update as a delete + an insert, using a second kind of manifest content called delete files:

Delete file typeSaysUsed when
position delete"row 5 of file X is gone"the row's exact location is known
equality delete"any row where series_id='PET.RWTC.D' AND period='2024-03-05' is gone"revising by key — EnergyScope's case

So a MERGE that revises WTI's price for one day writes an equality-delete on the key plus a new data file carrying the corrected row. Readers apply the deletes on the fly — this is merge-on-read. The old value is never mutated in place; it's shadowed by the delete and superseded by the new row.

The changelog scan — surfacing updates as a stream

To get a clean stream of what changed and how, Iceberg offers a changelog scan (Spark's create_changelog_view). It reconstructs change rows across a snapshot range and tags each one:

INSERT

A genuinely new row — a new period. Maps to the pipeline's new_count.

DELETE

A row removed — e.g. a series discontinued or a bad row retracted.

UPDATE_BEFORE / _AFTER

A revision, emitted as a pair: the old value and the new. Maps exactly to the pipeline's revised_count.

Each change row also carries its _change_type and the originating snapshot id. That is precisely the "new vs revised" distinction the Python loop computes by hand — but here it's derived from metadata the writer already produced, not from a row-by-row comparison of two full tables.

The delete-detection gap

There's a real asymmetry hiding here. The pipeline's Phase-2 diff keys rows on (series_id, period) and compares value — but it iterates over the new rows only, checking each against a lookup of the old ones. So it catches a key absent in old → INSERT and a value moved → UPDATE, but it never walks the old rows to find keys that vanished. A (series_id, period) that was in prev/ and is gone from today's cook produces no delta at all — the stale row simply lingers in the Flight server's memory until a full reload.

For EIA-style data this is usually harmless — agencies append and revise, they rarely retract history — which is why it has never bitten. But it is a structural blind spot: the hand-rolled diff computes INSERT and UPDATE and is incapable of emitting DELETE. Iceberg's changelog scan surfaces all three, because the deletion is recorded at write time (a removed data file, or an equality-delete) rather than inferred from a one-directional row comparison. If a vendor ever does drop rows — a discontinued series, a retracted print — this is exactly the case the current design misses and Iceberg would catch for free.

07 The resemblance

Mapped to EnergyScope today

Line the two systems up and the pipeline reveals itself as a hand-rolled subset of Iceberg — one snapshot deep, hashes instead of statistics, a Python loop instead of a metadata read.

Today — by hand (assets.py)

  • .manifest.json — an MD5 hash per year file
  • prev/ — exactly one snapshot copy (yesterday)
  • drop-guard — a blunt >20% row-count check to catch partial fetches
  • Phase-2 row-diff — full to_pylist() + Python dict + loop
  • new / revised counters — but no deletes (the loop walks new rows only)

Iceberg — native

  • manifests — per-file entries with per-column min/max stats, not just a hash
  • snapshots — the entire history, not just yesterday
  • atomic commit — a partial write never becomes current, so the guard is unnecessary
  • incremental scan — metadata-only, O(changed files)
  • changelog scan — INSERT / UPDATE and DELETE, straight from metadata

Two capabilities fall out that the hand-rolled version simply cannot do:

  1. Column stats prune before you read. A manifest records that file X's period ranges 2020–2022; a query for 2026 skips it without opening it. An MD5 hash can only tell you that a file changed — never what's inside it.
  2. Every snapshot is a queryable vintage, forever. The delta between any two historical snapshots is reconstructable, because the whole chain is retained — not just "yesterday vs today," which is all a single prev/ copy can express.
08 The payoff

Vintage queries — why a data vendor cares

The delta mechanism isn't just an efficiency win. Retaining every snapshot turns "what did the data look like on any past date?" into a first-class query — and for energy market data, that is a product feature, not plumbing.

EIA and most statistical agencies revise figures constantly — a weekly inventory number published Wednesday is refined over the following weeks. With Iceberg snapshots, every daily commit is a queryable vintage:

SELECT * FROM prices FOR TIMESTAMP AS OF '2024-03-05 00:00:00' -- data as known that day WHERE series_id = 'PET.WCESTUS1.W' -- US crude stocks, weekly

This returns the data exactly as it was known on 5 March 2024 — before any later revision. For a quant, that is the difference between an honest backtest and a lie:

Lookahead bias, eliminated

Backtesting a strategy on today's revised history secretly feeds it numbers no one had at the time — the model looks prescient because it saw corrected data. Querying the vintage as-of each decision date removes that lookahead bias entirely. Selling revision-safe history is exactly what data vendors like Bloomberg and Haver charge a premium for. Today, EnergyScope's single prev/ snapshot can't answer "what did you show me last Tuesday"; Iceberg makes it a one-line query.

Three more capabilities come from the same snapshot chain, at no extra cost:

09 The honest part

Caveats & adoption path

Iceberg is the strongest architectural upgrade on EnergyScope's board — but it earns a next, not a now, and the reasons are specific.

It's a storage change, invisible to serving

Iceberg is a lake/table format, not a query engine. The Flight server still reads a snapshot into RAM and serves it exactly as it does now. The in-memory O(1) index — the thing that wins point queries at 0.6 ms — is untouched. Iceberg improves ingest, storage, and history; it does not touch the hot path.

It needs a catalog

Something has to track "which metadata.json is current." That's a catalog — a REST catalog service, or pyiceberg's lighter SQL / filesystem catalog. One more moving part to run and back up.

pyiceberg's write path is maturing

The native, mature engine is Java/Spark/Trino. EnergyScope is pure Python → pyiceberg, whose read / append / snapshot-inspection are solid, but full merge-on-read CDC (equality deletes, changelog scan) is where you'd verify current support before betting on it.

Its home is object storage, not local NVMe

Iceberg shines over R2 / S3 — the cold origin tier. The hot local-NVMe scan tier is Vortex's turf. They compose rather than compete.

Recommended path — read-first

Start read-only: register the existing R2 Parquet as an Iceberg table and prove vintage / time-travel queries against real history. Then move writes: have Dagster commit snapshots instead of maintaining .manifest.json + prev/, and retire the Phase-2 row-diff in favour of the incremental scan. Because Iceberg wraps the Parquet you already have, this is a staged adoption, not a rewrite — the same files, a metadata layer on top.

The line to remember

Where dbt is the right tool for a job EnergyScope's architecture deliberately avoided (a warehouse full of interdependent SQL), Iceberg is the right tool for a job EnergyScope is currently doing badly by hand — the delta. That contrast is the whole case for prioritising it.

10 The field

The neighbours — and a wishlist

Iceberg didn't invent the category; it won a race. Understanding the runners — what they all fight, how each fights it, and where each is strong and weak — is what turns "we picked Iceberg" into a defensible decision rather than a fashion.

The problem they all exist to solve

Object storage (S3, R2) is a bucket of immutable blobs: cheap, durable, effectively infinite — and dumb. It has no transactions, no schema, no atomic multi-file operations, and a directory listing that is slow and, being eventually consistent, occasionally a liar. A "table" is a fiction you paint over a folder of Parquet. Every table format is an answer to the same question: how do you get database guarantees on top of a dumb object store? Concretely, each must solve some subset of:

Iceberg's answers are the subject of this whole document — a metadata tree, atomic pointer swap, v2 delete files, snapshot history, ID-based schema evolution, and a required external catalog. Here is how the neighbours answer differently.

The direct rivals — open table formats

Before the differences, the surprising part: the three rivals share almost everything. Learn one and ~80% transfers to the others — they diverge in exactly one place.

What all three share

The same substrate — immutable Parquet data files in object storage, interchangeable between them (and translatable by XTable). The same guarantees — atomic ACID commits, snapshot time-travel + rollback, schema evolution, and hidden / evolvable partitioning. The same dependency — an external catalog to hold the current-version pointer. The same purpose — database guarantees on a dumb object store. None of that is a differentiator; it's the price of admission to the category.

They differ on a single axis: how each records change — and therefore what it is fastest at. That one choice cascades into everything else. Here is each, and then the axis laid out plainly.

Delta Lake

Databricks → Linux Foundation

Where Iceberg keeps a tree of snapshots, Delta keeps an ordered transaction log — a _delta_log/ directory of numbered JSON commits, each listing add / remove file actions, with a Parquet checkpoint every ~10 commits so readers don't replay the whole log. The log is the single source of truth; a commit is an atomic write of the next log entry. Row-level change is handled by deletion vectors (mark rows deleted without rewriting the file, like Iceberg's position deletes) plus MERGE INTO.

Strengths
Dead-simple mental model — a linear log you can read top to bottom.
Most mature tooling; deletion vectors are efficient; delta-rs gives real non-Spark (Rust/Python) access.
UniForm exposes Iceberg-readable metadata — hedges the format war.
Weaknesses
Databricks gravity — the best features historically landed in their fork before OSS caught up.
The log model plans worse than Iceberg's manifest stats at extreme file counts.
Governance/catalog was Databricks-centric (Unity); less engine-neutral momentum than Iceberg now.

Apache Hudi

Uber

Built for a different pain: high-volume upserts and CDC from operational databases. Its differentiator is a record-level index — a map from record key to the file that holds it — so an update is targeted to one file group, never a full-table scan. It offers two table types you pick per workload: Copy-on-Write (rewrite the file on update — read-fast, write-heavy) and Merge-on-Read (append delta logs, merge at read — write-fast, read-heavier). Ships its own ingestion tooling (Hudi Streamer) and async compaction/clustering.

Strengths
Best-in-class upsert / delete throughput and incremental pull — the record-level index is unique here.
Deletes are first-class (fixing exactly EnergyScope's blind spot).
Mature streaming-ingestion story.
Weaknesses
The most operationally complex of the three — many knobs; tuning the index + compaction is real work.
Historically tightly bound to Spark and its own writer; smaller engine ecosystem than Iceberg.
The record-level index adds write overhead and state to maintain.

Apache Paimon

Flink community (was Flink Table Store)

A database storage engine on a lake: primary-key tables backed by an LSM-tree (log-structured merge tree — sorted runs that compact in the background), the same structure that powers write-heavy databases. Streaming-first and Flink-native, it treats a table and a changelog stream as the same object, so real-time upserts land with low latency.

Strengths
Genuinely low-latency, high-frequency upserts — the LSM design is built for it.
Unifies streaming and batch natively; built-in changelog producers.
Weaknesses
Newest and least mature — smallest community and engine support.
Flink-centric; ad-hoc / batch query engines are thinner than Iceberg's.
LSM compaction is its own tuning burden — overkill for daily batch data.

What's unique to each — the one axis

Given the shared foundation above, each format's identity reduces to one design choice — how it records change — and the single strength that follows. Iceberg is in the table too, to place it among its rivals:

FormatRecords change as…Uniquely best at
Delta Lakea linear transaction logsimplicity + the most mature tooling
Iceberga snapshot tree with per-file statspruning + planning at massive file counts
Hudia record-level indextargeted upserts + native deletes
Paimonan LSM-treehigh-frequency streaming upserts

So the choice is not really "which format" — it's "which pain": scale of files points to Iceberg, simplicity to Delta, update frequency to Hudi or Paimon. For EnergyScope — many files, revision-heavy, mostly-daily batch — Iceberg's file-scale pruning is the fit, with Hudi's upsert index the one genuinely tempting thing it doesn't match.

The layers around them

Catalogs — the format needs an external authority to hold the atomic "current pointer" and answer discovery/governance. Nessie is the interesting one: git-like branches, tags, commits, and merges over tables, with multi-table transactions — "data as code," and a natural home for tagging vintages or branching a migration. Apache Polaris (Snowflake) and Unity Catalog (Databricks, open-sourced) are the big open REST catalogs; AWS Glue is the managed default. InteropApache XTable (formerly OneTable) translates metadata between Iceberg / Delta / Hudi, and Delta UniForm writes Delta while exposing Iceberg metadata. The trend is unmistakable: the data (Parquet) is shared; the formats are competing metadata layers, and translation is making the choice less permanent every year.

The wishlist — if one were designing it from EnergyScope's chair

No existing format is ideal for a time-series data vendor that serves from RAM across an edge fleet. Each solves part of it. Imagining the format that fits this exact shape — and noting who does each piece today:

  1. Iceberg's engine-neutral metadata tree as the substrate. Broad read ecosystem, hidden partitioning, schema-by-ID, snapshot time-travel — the winning base to build on.today: Iceberg
  2. Hudi's record-level index, so revisions are targeted and deletes are never missed. "Update WTI for 2024-03-05" should touch one file, and a vanished key should always register — closing both the delete-detection gap and Iceberg's clunkier delete-file model.today: Hudi (but bolted to Spark)
  3. Nessie's branch/tag semantics as a first-class primitive, not a bolted-on catalog. For a data vendor, releases and vintages are the product: a vintage should be a tag, a schema migration a branch you merge, and a multi-dataset publish one atomic commit.today: Nessie, external
  4. Delta's single-source-of-truth clarity. One legible history, not a four-level tree you need a diagram to explain. Approachability is a feature.today: Delta
  5. A native Python/Rust writer — no JVM assumed. EnergyScope is Python; the ideal doesn't presume Spark. delta-rs and pyiceberg are crawling this way; it should be the default, not the hard path.today: partial (delta-rs, pyiceberg)
  6. Pluggable hot/cold file encodings under one table. Formats default to Parquet everywhere. The ideal lets the same logical table store Vortex on local NVMe (fast, low-memory serving) and Parquet on R2 (cheap, portable cold) under one metadata layer — "Iceberg tables whose data files can be Vortex," an idea already emerging.today: nobody — the composable gap
  7. Time-series-native primitives. Understand (series_id, period) as a natural composite key, encode value columns with ALP-style float compression, and cluster by (series, period) automatically — instead of treating time-series as generic rows.today: nobody (specialist TSDBs, but those are databases)
  8. A serving-tier handoff. The formats stop at "here are the files." The ideal would expose a snapshot as something a Flight server can mmap and index directly — closing the gap between a lake table and an in-RAM serving index. → Explored in full: The Serving Snapshot — design, the v0 EnergyScope already ships, base+delta liveness, prior art, and a weekend experiment to validate it.today: nobody — the unclaimed seam
The honest synthesis

No single project is that format, and probably none should be — items 1–5 are convergence bets the big three are already racing toward (XTable interop, delta-rs/pyiceberg, deletion vectors), while 6–8 are the genuinely open frontier where a time-series serving vendor could contribute. The closest real assembly today is the composable stack the ecosystem doc keeps returning to: Iceberg + Nessie for the table and its vintages, Vortex + Parquet as the hot/cold file formats beneath it, a Python/Rust writer, and the Flight server as the serving handoff. The wishlist isn't a product to go build — it's the map of which seams to watch, and which two or three (hot/cold file pluggability, the serving handoff) are worth prototyping because nobody has closed them yet.