Data catalog

The catalog is the searchable inventory of every table DataRunner has seen across your connections — schema, columns, last scan time — annotated with the descriptions and tags your team adds.

Read: 3 min
Sections: 4
Plans: All plans

What the catalog tracks

Every time DataRunner introspects a connection, it records each table as a catalog asset: the table name, its schema, the originating connection, and the timestamp of the last scan. The catalog view lets you search across all connections and filter by connection.

Table — the introspected table name
Schema — owning schema (e.g. public)
Description — human-written context (you add this)
Connection — which connection it came from
Last scan — when the schema was last refreshed

Populating the catalog

Assets appear automatically as connections are introspected — testing a connection or running a schema refresh fills the catalog. If a catalog is empty, run a schema refresh on the connection to populate it. Workspace-wide and per-connection refreshes both require an admin.

Only engines with schema introspection appear here. The catalog is built from introspection, which is implemented for the PostgreSQL, MySQL, SQL Server and SQLite families. A ClickHouse, MongoDB, Redis, DuckDB or BigQuery connection runs queries normally but contributes no catalog assets — an empty catalog on one of those is expected, not a misconfiguration.

Describe and tag

Add a description and tags to any asset so teammates know what a table means and who owns it. An asset can also carry an owner, so "who do I ask about this table" has an answer.

http

PATCH /api/v1/catalog/assets/{id}
Authorization: Bearer <token>
Content-Type: application/json

{
  "description": "One row per fulfilled order. Source of truth for revenue.",
  "tags": ["finance", "core"]
}

Lineage and impact analysis

Alongside the asset inventory, DataRunner records a lineage graph: directed edges from one object to another, detected as queries and downstream objects are saved. Nodes are identified by a kind and an id, and the graph spans more than tables:

table — a catalog asset
query — a saved query that reads it
report and dashboard — built on those queries
notebook, watch, monitor, campaign, sync — anything else that consumes them

The impact endpoint answers the question that matters before a migration: if this table changes or goes away, what breaks? It walks the graph downstream from a node and returns everything that depends on it.

http

GET /api/v1/lineage/impact?kind=table&id=public.orders
Authorization: Bearer <token>

# Everything downstream of that table: the queries that read it, the
# reports and dashboards built on those queries, the watches and
# monitors that depend on them.

Pair the catalog with Metrics — once a table is documented, promote the canonical numbers it feeds into named, reusable metric definitions.