Skip to content

dbt-hodor

A free, self-hosted alternative to dbt State. It reuses the objects that are already in your warehouse and rebuilds only what changed.

Get started Interactive guide

dbt-hodor is a free, self-hosted alternative to dbt Labs' dbt State. It runs inside dbt. For every selected node it decides whether to reuse the object that is already in the warehouse or to build it again. A node is reused when its logic has not changed and its upstream data has not changed since the last build.

It needs no dbt platform account and sends nothing to external services. State lives in a Delta table on Databricks, in a Postgres database such as Databricks Lakebase, or in a local DuckDB file.

Status: early development

hodor has end-to-end tests on dbt-duckdb and live tests on a Databricks SQL warehouse and on Databricks Lakebase; the test docs list what they cover. It has not run in a large production project yet. Before enabling reuse in production, run in shadow mode for at least a week of scheduled runs and review its decisions. See the rollout guidance. Changes between releases, including the breaking ones, are in Upgrading.

Why "hodor"

The name is a backronym: Hold Old Data, Or Rebuild. That is the only decision hodor makes, once per node.

hodor holds the door in front of your warehouse. A node gets through to a rebuild only when its code changed or new data arrived upstream. Everything else keeps the data it already has.

What it does for each node

  1. Hashes the logic. The compiled SQL is normalized with sqlglot, so case, quoting, comments and table aliases do not count as changes.
  2. Checks the upstream data. Each upstream table has a data version: the Delta commit history on Databricks, a content fingerprint on DuckDB.
  3. Decides. It compares both against the record of the node's last build and returns reuse or build, with a reason code.
  4. Records. It appends one record per node to the state table.

Every failure leads to a build, never to a reuse. hodor never fails a dbt run.

Slim CI

A pull request that builds state:modified+ into a CI catalog or a schema per pull request, under any naming scheme, and reads the rest from a base target with --defer --state, works with hodor as it is. The base target is what the pull request is compared with: the main catalog of the same environment, or prod.

  • Each push to the PR reuses what the last push built, when the code and the data it reads are unchanged.
  • Different catalog and schema names between targets are not code changes.
  • A deploy compares only with the deployed target's own history.

A pull request builds its changed model in the CI catalog, reads unchanged parents from the base target through defer, and keeps its build records apart from the base target's A pull request builds its changed model in the CI catalog, reads unchanged parents from the base target through defer, and keeps its build records apart from the base target's

The Slim CI page has the setup. The interactive guide plays the scenarios: pushes, new base data, dbt clone, the deploy.

Explore

  • Getting started

    Install hodor and run the local DuckDB example.

    Getting started

  • Decision rules

    The rules and their reason codes.

    Decision rules

  • Slim CI

    Defer, CI schemas, dbt clone, deploys.

    Slim CI

  • Configuration

    Node configs and settings.

    Configuration

  • State stores

    Delta on Databricks, Postgres or Lakebase, or a local DuckDB file.

    State stores

  • Operations

    Modes, rollout, fail-open behavior, known limits.

    Operations

  • Interactive guide

    A single-page walkthrough, with a run simulator and slim CI scenarios.

    Open the guide

What is built and what is planned

Part State
Decisions for table, incremental and view models, seeds, snapshots and data tests built, tested end to end on dbt-duckdb
delta store and Delta history data versions built, tested live on a Databricks workspace (pytest -m databricks)
duckdb store and content fingerprint built, tested
dbt-hodor explain CLI built, tested end to end on dbt-duckdb; not yet run live on Databricks
dbt-hodor prune CLI built, tested end to end on dbt-duckdb
postgres store, Lakebase included built, tested end to end against Lakebase (pytest -m postgres)
Slim CI: --defer, --favor-state, environment-neutral hash built, tested end to end on dbt-duckdb and live on Databricks
PyPI package published