Data pipelines for TypeScript
Pull data in, transform it, keep it fresh. One CLI, one DuckDB file.
Free and open source · macOS and Linux
One tool, end to end
A croft project is a folder of TypeScript and SQL files. Each file becomes one table, and croft runs them in order.
Ingest
Bring data in
From Stripe, GitHub or any API, or from CSV exports. Incremental, typed, and safe to rerun.
// assets/stripe_charges.ts export default ingest({ key: "id", schedule: "every hour", async *rows({ http }) { … }, });
Transform
Shape it
A SQL query or a TypeScript function per table, with checks that run before anything is saved.
-- assets/revenue_by_month.sql -- check: revenue >= 0 SELECT month, sum(amount) AS revenue FROM stripe_charges GROUP BY month
Schedule
Keep it fresh
A schedule in plain words. Everything downstream updates after it, in order.
$ croft schedule on $ croft status ASSET NEXT stripe_charges every hour revenue_by_month after inputs
Serve
Use it in your app
Query the tables from any Bun or Node app, with no database server to run.
import { query } from "@zabaca/croft/read"; const rows = await query( "from revenue_by_month order by month", );
Typed across SQL and TypeScript
Every table gets a TypeScript type, generated from its real columns. Change a SQL query, and croft checks the TypeScript that reads it, before anything runs.
- Rename a column, see what breaks. Code that reads the old name fails the check, with the columns it could read instead.
- Mistyped SQL is caught early. Every query is checked against the columns it reads, without running it.
- Typed in your app too.
query<"revenue_by_month">()returns that table's columns, and a renamed one breaks your app's type-check. - Autocomplete for your data. Your editor knows each table's columns and their types.
// .croft/types/revenue_by_region.d.ts, written by croft export type RevenueByRegionRow = { region: string; // VARCHAR, the key orders: number | bigint | null; revenue: number | null; // DOUBLE };
After renaming revenue to total in the SQL:
$ croft validate --types error UNKNOWN_INPUT_COLUMN assets/region_labels.ts:9:65 revenue_by_region has no column "revenue" fix: revenue_by_region has these columns: region, orders, total, _loaded_at
What it replaces
A typical data stack is several tools, in Python and YAML, glued together. croft is one, in TypeScript.
| Job | The usual stack | croft |
|---|---|---|
| Load from APIs and files | dlt, Airbyte or custom Python scripts | ingest() in a .ts file |
| Transform | dbt models (SQL and YAML) | a .sql file, or transform() in TypeScript |
| Test the data | dbt tests | checks that run before every save |
| Schedule and orchestrate | Dagster, Airflow or cron | schedule: "every hour" |
| Catch a broken column | when the warehouse runs the model | before anything runs, from SQL into the TypeScript that reads it |
| Store | a warehouse to host and pay for | one DuckDB file on your machine |
| Read from your app | a client and credentials per warehouse | @zabaca/croft/read, typed by table |
| Language | Python, YAML and SQL | TypeScript, and SQL where you want it |
croft does these jobs; it does not read those tools' projects. It is built for one machine and one DuckDB file. If your team already runs a warehouse with dbt, keep it.
And your agent can run it
Tell Claude Code what you want. It writes the pipeline, and croft checks every step.
- Mistakes are caught early. croft checks each change before it runs, and tells the agent exactly what to fix.
- Try before it's real. Every change can be previewed first, without touching your tables.
- You stay in charge. Anything that could lose data stops and waits for your OK.
An example
YouPull my Stripe charges in every hour and show me revenue by month.
ClaudeDone. stripe_charges loads every hour, and revenue_by_month is built from it. I previewed both first: 1,204 charges, 14 months.
ClaudeTurning on the hourly schedule runs it without me. Shall I?
Your data stays safe
Some data can't be fetched again. croft never throws it away on its own.
Nothing half-saved
Each update is saved whole or not at all, and a failed check keeps the old data.
A trash you can undo
Anything replaced or deleted goes to the trash first, and comes back with one command.
On your machine
One file on your computer. No account, no server, no cloud to pay for.