Scheduled runs rebuild everything
A scheduled dbt run builds every model in scope, even when its code and its inputs are the same as yesterday. hodor checks each node first and reuses what did not change.
An example
The daily run of a large Databricks project (illustrative figures):
Every night the run builds all of it again. Most nodes read the same data as the day before.
What dbt-hodor prints (simulated)
What hodor changes
For each node hodor asks two questions: did the code change, and did new data arrive upstream? If both answers are no, the node keeps the table it already has.
Illustration only, not a measurement. Move the slider to change the share of nodes with changed code or new upstream data.
The name
hodor is a backronym. One name, one job.
One job: hold the door
hodor holds the door in front of your warehouse, and that is the only decision it makes, once per node.
A node gets through to a rebuild only when its code changed or new data arrived upstream. Everything else keeps the data it already has.
The logo is original artwork.
What dbt-hodor prints (simulated)
Teal nodes stop at the bar and are reused. Amber nodes pass and are built.
Animations are off in your system settings.
How a run works
hodor runs inside the dbt process as a plugin. Step through one run, or press Play.
Play runs the remaining steps by itself; press Pause to stop. Step shows exactly one more step. Back goes one step back. Restart returns to step 1.
Fast path
When the compiled SQL text equals the text of the last build, hodor takes the stored analysis and skips sqlglot. Hashing a project of about 15,000 models then takes about 0.2 s instead of about 7 minutes (measured on a sample, single thread).
What dbt-hodor prints (simulated)
Simulator
Pick a scenario and watch a dbt run, line by line. The log follows the output format of the end-to-end tests on dbt-duckdb. The lines are simulated, not recorded.
How the numbers are simulated
Queries: hodor reads the state once, runs one DESCRIBE HISTORY per upstream table (2 here), one refresh read per built table and one insert per 500 records. A built model counts 2 queries, a built test 1, a reused node 0. Wall clock: built nodes take fixed example times (views 1.2 s, tables 4.8 to 6.5 s, tests 2.4 s), a reused node 0.01 s, plus 1.1 s for hodor at the start and 0.4 s for the insert. Compute: 4 compute-seconds per second of a built node, 0.5 per hodor query. The comparison column is the same run with every node built and no hodor.
dbt-hodor explain -s
Play prints the rest, Pause stops. Step adds one line, Back removes one, Restart clears the log.
Run metrics (simulated, illustrative, not measured)
Decision rules
The engine checks the rules in order. The first match decides. Pick a scenario to see the path.
What dbt-hodor prints (simulated)
Data versions
hodor does not trust file timestamps. It asks each table for a data version, a token it compares for equality with the token recorded at the last build.
Delta table history
The token is the version and the commit time of the latest commit that changed data, so a recreated table whose versions restart still counts as changed. Maintenance commits never count. Other commits count, unless all their change metrics are 0.
Maintenance operations that never count
OPTIMIZE, VACUUM START, VACUUM END, ANALYZE, COMPUTE STATS, SET TBLPROPERTIES, UNSET TBLPROPERTIES, CHANGE COLUMN, ADD CONSTRAINT, DROP CONSTRAINT, UPGRADE PROTOCOL, SET TAGS, UNSET TAGS, CLUSTER BY, REORG, FSCK.
Change metrics and the commit window
Change metrics that decide the zero-row case: numOutputRows, numRemovedFiles, numTargetRowsInserted, numTargetRowsUpdated, numTargetRowsDeleted, numDeletedRows, numUpdatedRows, numCopiedRows. A commit with none of them counts as a change. A replace that writes 0 rows still removes the old files, so it counts: the table is now empty.
Only the newest 20 commits are read (DESCRIBE HISTORY ... LIMIT 20). If none of them changed data, the version is unknown and dependents build. An unknown operation name counts as a change: that can cause an extra build, never a wrong reuse.
DuckDB content fingerprint
DuckDB has no table history. The token is the row count plus an order-independent sum of row hashes. Reordering rows keeps the token. Changing a value changes it. This suits local test data.
What dbt-hodor prints (simulated)
Lineage
To know if upstream data changed, hodor needs the tables a node really reads. Sources and tables have their own version. Views and ephemerals have none, so hodor looks through them.
- A
delta.`path`input has a version from its own Delta history. read_files(...)and other table functions have no version. The node always builds.- One unknown input is enough to build the node (
unknown_upstream). - A non-Delta source, such as a foreign table, is unknown too.
What dbt-hodor prints (simulated)
Modes and safety
Installing the package skips nothing. The default mode is shadow.
Mode Illustration with made-up nodes.
Run metrics (simulated, illustrative, not measured; same model as the Simulator)
Fail open
Every error leads to a build, never to a reuse. hodor never fails a dbt run.
| Failure | Behavior |
|---|---|
| Exception while deciding a node | Warning, normal build, reason hodor_error |
| State store cannot be read | Warning, the whole run builds |
| State write fails | Warning; those nodes build next run (no_history) |
| Metadata query fails for a table | That table is unknown; dependents build |
| Invalid config value | Warning, default is used |
| Ctrl+C or job cancel | Queued records are flushed if possible |
Tests are replayed
When nothing a test checks has changed, hodor returns the recorded result (pass, warn or fail) with the recorded failure count. The result line shows replayed. lag_tolerance is always 0 for tests.
7 of 8 PASS not_null_fct_orders_id ......... [PASS (replayed) in 0.02s]What dbt-hodor prints (simulated)
Slim CI
A pull request builds its changes into its own schema and reads the rest from a base target with --defer --state. The base target is what the pull request is compared with: the main catalog of the same environment, or prod. Pick a scenario to see what hodor does on each push and on the deploy. The lines are simulated, not recorded.
The setup behind the scenarios
The base target builds into analytics.sales; the CI target builds into analytics_ci.<ci_schema> through a generate_schema_name macro. The name of that schema can follow any scheme, for example pr_42_sales or mr_17_sales; hodor does not parse it. The PR edits fct_orders and adds rpt_orders. The CI job runs dbt build -s state:modified+ dim_parity --defer --state base-state; dim_parity is selected by name, though the PR does not change it. stg_orders is not selected, so dbt reads the base table through defer. See Slim CI in the docs.
Docs
What it is
A free, self-hosted alternative to dbt Labs' dbt State. It runs inside dbt as an auto-loaded plugin (module dbt_hodor). It needs no dbt platform account and sends nothing to external services. dbt-core 1.12.x; adapters dbt-databricks 1.12.7, and dbt-duckdb for local development and tests.
Install
Install dbt-hodor from PyPI in the environment where dbt runs. dbt loads any installed module named dbt_* that has a plugins list, so there is nothing to configure in dbt.
pip install dbt-hodor
# local DuckDB state store:
pip install "dbt-hodor[duckdb]"Requires Python 3.10 or newer. If upstream dbt_state is also installed, hodor disables itself with a warning.
Node configs
Set them under state: in YAML (same block as dbt State).
# dbt_project.yml
models:
my_project:
+state:
lag_tolerance: 4h
require_fresh_data_from: any| Key | Default | Values |
|---|---|---|
lag_tolerance | 45m | 0s, 45m, 4h, 1d, 1w, integer seconds, or {count: 4, period: hour} |
require_fresh_data_from | any | any: build when any upstream has new data. all: build only when all have. |
compare_unrendered_code | false | true: build only when both the rendered and the unrendered code changed |
execute_hooks_on_any_reuse | false | true: run pre- and post-hooks on a reused node (execute_hooks_on_reuse also accepted) |
evaluate_volatile_sql | auto | auto: current_date(), current_user(), current_timestamp() and similar functions count as a changing upstream unless they only fill a stamp column; true: stamps count too; false: never (dbt State default) |
pre_clone, allow_clones and defer_to_target are accepted and ignored, with a warning.
Settings
Set them on the profiles.yml target. An environment variable overrides the profile key. Name parts may contain letters, digits, _ and -; hodor always quotes them.
| Profile key | Env var | Default | Values |
|---|---|---|---|
hodor_mode | HODOR_MODE | shadow | off, shadow, on |
hodor_store | HODOR_STORE | delta | delta, duckdb, postgres (alias lakebase) |
hodor_database | HODOR_DATABASE | the target's catalog, for delta only | Databricks catalog, or Postgres database (required); not used by duckdb |
hodor_schema | HODOR_SCHEMA | none for delta and postgres (required); main for duckdb | schema name |
hodor_table | HODOR_TABLE | hodor_state | table name |
hodor_path | HODOR_PATH | .hodor/state.duckdb | DuckDB file, for duckdb |
hodor_metadata_threads | HODOR_METADATA_THREADS | 8 | 1 to 64 |
hodor_lookback_days | HODOR_LOOKBACK_DAYS | 90 | 1 to 3650 |
hodor_endpoint | HODOR_ENDPOINT | none | Lakebase endpoint path, projects/<p>/branches/<b>/endpoints/<e>. Supplies host, user and OAuth token |
hodor_host, hodor_port, hodor_user, hodor_password, hodor_sslmode | HODOR_HOST and so on | port 5432, sslmode require | For a Postgres that is not Lakebase. Keep the password in an environment variable |
# profiles.yml
my_project:
target: prod
outputs:
prod:
type: databricks
catalog: my_catalog
# ... connection settings ...
hodor_mode: shadow
hodor_schema: my_ops_schema
local:
type: duckdb
path: dev.duckdb
hodor_store: duckdbState store
An append-only table with one record per node per run. The delta table is <catalog>.<schema>.hodor_state, clustered by (target_name, node_key). The schema must exist: hodor creates only the table. The duckdb store is a local file that hodor creates. The postgres store holds state in any Postgres, Databricks Lakebase included, and creates its schema and table on first use.
The state table decides what is skipped in prod. Grant MODIFY on it only to the job service principals. Without hodor_schema, the delta store turns hodor off with a warning. Cleanup is manual: DELETE FROM hodor_state WHERE recorded_at_ms < <cutoff>, then VACUUM on Delta.
Reason codes
First match wins. Shadow mode builds anyway and records the decision.
| # | Reason | Decision |
|---|
CLI
dbt-hodor explain shows why each node of the latest run was reused or built: the reason, the last build it was compared with, the code and config hashes then and now, and each upstream data version then and now. It only reads the state table. Options: -s (a unique_id, a relation or a name; repeat for more nodes), --invocation, --json, and dbt's --project-dir, --profiles-dir, --profile and -t. Try it in the Simulator tab.
$ dbt-hodor explain -s fct_orders
dbt-hodor explain: hodor_e2e/local, invocation 7d1e4b90-... (2026-10-21 20:42 UTC), 1 node
model.hodor_e2e.fct_orders: build (new_upstream_data)
last build 2026-10-20 20:41 UTC, invocation 1f0c2a7e-...
code hash same (qualify:6c1f02aa)
config hash same (e56f2a83)
upstream raw.orders: 14@1792465200000 -> 15@1792551600000dbt-hodor ddl prints the CREATE TABLE statement of the state table, for manual creation.
dbt-hodor prune deletes the old records of this project and target. It is a dry run unless you pass --yes, and --keep-days N sets the window to keep (default hodor_lookback_days). Records of other projects and targets in the same table are never touched.
$ dbt-hodor prune --keep-days 30
Would delete 1284 records: hodor_e2e/local, records older than 30 days.
Run again with --yes to delete them.Result lines carry the reason too:
4 of 8 REUSED sql table model main.fct_orders ......... [REUSED (no_new_data) in 0.01s]
4 of 8 OK created sql table model main.fct_orders ......... [OK (new_upstream_data) in 0.07s]
4 of 8 OK created sql table model main.fct_orders ......... [OK (shadow, could reuse: no_new_data) in 0.06s]
7 of 8 PASS not_null_fct_orders_id ......... [PASS (replayed) in 0.02s]The last line of a run: dbt-hodor: N reused, M built, K fail-open errors. In shadow mode: dbt-hodor: N could be reused, M need a build (shadow: all built), K fail-open errors.
Slim CI
hodor works with -s state:modified+ --defer --state <prod manifest>, with or without --favor-state. Catalog and schema names of the project's own nodes are replaced by the node's id before hashing, so a CI catalog or a schema per pull request, under any naming scheme, is not a code change. Each push reuses what the last push built. A table made by dbt clone has no hodor record, so it builds once, then is reused. A deploy compares only with the target's own history. Try it in the Slim CI tab.
Scope of v1
Commands: run, build, test, seed and snapshot.
Supported: models materialized as table, incremental and view, plus seeds, snapshots and data tests. Incremental strategies: append, merge, delete+insert, insert_overwrite and replace_where; an unset strategy counts as merge.
Built as plain dbt: Python models, microbatch, materialized views, streaming tables, metric views, custom materializations, custom incremental strategies and unit tests.
A seed is reused when the CSV content (sha256 of the file, line endings normalized) and its config are unchanged and its table was not written outside dbt. A snapshot follows the table rules: code, config and upstream data unchanged, and its target not written outside dbt. lag_tolerance applies to snapshots too.
Known limit
HODOR_MODE=off after such an upgrade.