GitHub - xorq-labs/xorq: Git-native executable catalog for agentic data work.

Xorq is an executable, composable catalog for tabular data work. It gives agents runnable, content-addressed pipelines instead of markdown notes: memory you can execute, not prose you have to reread. Entries compose, so new pipelines build on cataloged ones instead of starting from scratch. It turns ephemeral agent work such as pandas scripts, sklearn pipelines, ad-hoc tables into durable artifacts that any future agent or human can discover, reproduce and reuse.

It comes with a CLI for agents and a TUI for humans with a git-native catalog.

Xorq in 60 seconds

Think of Xorq as giving a dataframe pipeline a reproducible identity.

1. Define the pipeline

You start with a lazy dataframe expression:

import xorq.api as xo

penguins = xo.examples.penguins.fetch()
penguins_agg = (
    penguins
    .filter(xo._.species.notnull())
    .group_by("species")
    .agg(avg_bill_length=xo._.bill_length_mm.mean())
)

This describes the transformation: filter the penguins data, group it by species, and calculate the average bill length. Unlike a sequence of scripts that must be executed in the right order, the expression gives Xorq a structured representation of the computation.

2. Build it

Save the expression in expr.py, then build it into a reproducible artifact:

Xorq produces a hashed build directory:

builds/fa2122f6a9e9/
├── expr.yaml
├── expr_metadata.json
├── requirements.txt
└── xorq-<version>-py3-none-any.whl

The manifest (expr.yaml + *_metadata.json) is the content-addressed specification of the pipeline. It captures the computation, including its operations, schema, lineage, and inputs. The build packages that specification with the environment needed to reproduce and execute it.

3. Add it to a catalog

Add the build to a catalog:

xorq catalog add builds/fa2122f6a9e9/ -a penguins-agg

The catalog is a Git repository of these entries. An entry can be discovered by its hash or by a human-readable alias such as penguins-agg.

The flow is:

Dataframe expression
        ↓
    xorq uv build
        ↓
Content-addressed specification
        +
Reproducible environment
        ↓
   Catalog entry
        ↓
     Git repo
        ↓
 Humans / CI / agents

Another human or agent can now discover, reproduce, run, or compose that pipeline without reconstructing the original work from scripts, environment setup, and chat history.

That is the core idea behind Xorq's executable catalog: instead of remembering a description of what happened, the catalog keeps the computation itself.

The Problem

Coding agents are great at accomplishing closed-loop task but in the process accumulate tech-debt and unnecessary complexity. For example, if you ask a coding agent to build a dashboard, you are more likely than not to get a folder of one-off Python scripts that import each other in non-obvious ways, an embedded JSON holding intermediate state, and a requirements.txt that was last regenerated two sessions ago. It may also execute end-to-end on your laptop. Verifying by reproducing on another machine, or productionizing any of it, means rewriting some of it. And every time you rewrite, more complexity gets introduced.

Two ways to start

With an agent. Install the Xorq plugin in Claude Code and let it build catalogs for you:

/plugin marketplace add xorq-labs/claude-plugins
/plugin install xorq@xorq-plugins

The plugin adds four slash commands:

• /xorq:init — load CSV or Parquet files as catalog entries
• /xorq:catalog-explore — browse what's already in a catalog
• /xorq:composer — combine entries into new joined/aliased entries
• /xorq:builder — assemble ML pipelines and semantic-layer entries

The agent does the building; you keep the catalog.

Manually. Install the library and start composing expressions in Python:

❯ pip install xorq[examples]
❯ xorq init -t penguins

Design choices

Supported engines

The same expression can run against any of these backends, and into_backend moves data between them.

Comparison

Xorq is complementary to general-purpose data catalogs. Atlan, Collibra, and Unity Catalog inventory and govern the assets you already have: descriptions, owners, glossaries, policies, permissions, harvested lineage. Xorq catalogs the computation, so an entry is something you run, not something you read about.

A governance catalog tells you what data exists, who owns it, and who may read it. Xorq tells an agent what to run to get the same answer twice and humans how it arrived at that answer. Read governed tables through the Databricks or Snowflake backend, and register entries as assets.

Benchmark

On DABStep — 450 data-analysis questions over payment transaction data — a Xorq semantic catalog of 33 named expressions takes Haiku from 50% to 84%, 8pp above the Sonnet baseline.

Where the agent looks for context mattered more than which base model it used. Full write-up: Orientation Over Reasoning.

Under the hood

import xorq.api as xo
from xorq.caching import ParquetCache

penguins = xo.examples.penguins.fetch()

penguins_agg = (
    penguins
    .filter(xo._.species.notnull())
    .group_by("species")
    .agg(avg_bill_length=xo._.bill_length_mm.mean())
)

expr = (
    penguins_agg
    .cache(ParquetCache.from_kwargs())
)

expr = penguins.into_backend(xo.sqlite.connect())
expr.ls.backends

(<xorq.backends.sqlite.Backend at 0x107debda0>,
 <xorq.backends.xorq_datafusion.Backend at 0x1669002c0>)

In [6]: expr.to_pyarrow_batches()
Out[6]: <pyarrow.lib.RecordBatchReader at 0x15dc3f570>

❯ xorq uv build expr.py
Building wheel...
Successfully built ...
builds/fa2122f6a9e9

❯ xorq catalog -p git-catalogs/penguins init
Initialized catalog at /git-catalogs/penguins

❯ xorq catalog add builds/fa2122f6a9e9/ -a penguins-agg
Added fa2122f6a9e9

❯ git -C git-catalogs/penguins reflog
17dd4e9 (HEAD -> main) HEAD@{0}: add: fa2122f6a9e9 (aliases penguins-agg)
9f5d242 HEAD@{1}: add catalog.yaml
9915df3 HEAD@{2}: commit: Switching to main

❯ tree git-catalogs/penguins
git-catalogs/penguins
├── aliases
│   └── penguins-agg.zip -> ../entries/fa2122f6a9e9.zip
├── entries
│   └── fa2122f6a9e9.zip
├── metadata
│   └── fa2122f6a9e9.zip.metadata.yaml
└── catalog.yaml

# List aliased entries
❯ ls git-catalogs/penguins/aliases/

# Find entries that emit an 'avg_bill_length' column
❯ grep -l 'avg_bill_length' git-catalogs/penguins/metadata/*.yaml

# Find entries running on DataFusion
❯ grep -l 'xorq_datafusion' git-catalogs/penguins/metadata/*.yaml

# Find source entries (vs. unbound, expr_builder kinds)
❯ grep -l 'kind: source' git-catalogs/penguins/metadata/*.yaml

❯ tree builds/fa2122f6a9e9
├── build_metadata.json
├── expr.yaml
├── expr_metadata.json
├── profiles.yaml
├── requirements.txt
└── xorq-0.3.24-py3-none-any.whl

# Input-addressed, composable, portable
# Abridged expr.yaml
definitions:
  nodes:
    '@read_b5f228c91f16':
      op: Read
      method_name: read_parquet
      name: penguins
      read_kwargs:
        - [hash_path, .../penguins/20250703T145709Z-c3cde/penguins.parquet]
        - [table_name, penguins]
      schema_ref: schema_f11dda6745cc

    '@filter_fa4a3fde7765':
      op: Filter
      parent: { node_ref: '@read_b5f228c91f16' }
      predicates:
        - { op: NotNull, arg: { op: Field, name: species, ... } }

    '@aggregate_eb3109707390':
      op: Aggregate
      parent: { node_ref: '@filter_fa4a3fde7765' }
      by:
        species: { op: Field, name: species, ... }
      metrics:
        avg_bill_length:
          op: Mean
          arg: { op: Field, name: bill_length_mm, ... }

    '@cachednode_fa2122f6a9e9':
      op: CachedNode
      parent: { node_ref: '@aggregate_eb3109707390' }
      cache:
        type: ParquetCache
        relative_path: parquet
      schema_ref: schema_9271d5e9d443

expression:
  node_ref: '@cachednode_fa2122f6a9e9'
  schema_ref: { schema_ref: schema_9271d5e9d443 }

❯ xorq catalog list-aliases
penguins-agg

❯ xorq catalog list
fa2122f6a9e9

❯ xorq run builds/fa2122f6a9e9 -o out.parquet

Learn more

• Quickstart
• Why xorq?
• Claude Code plugin
• Scikit-learn
• A Git-Native Semantic Layer — building a portable semantic catalog with Xorq
• Orientation Over Reasoning — Haiku + Xorq catalog hits 84% on DABStep, above the Sonnet baseline

Pre-1.0. Expect breaking changes with migration guides.