Ibex | Fast, Concise DataFrame Pipelines

3 min read Original article ↗
Ibex logo

Ibex

Ibex gives DataFrame pipelines their own compact language. Explore in a notebook (Jupyter, R/dplyr, or Ibex's own web UI) and compile it to C++ when the pipeline needs to ship.

Concise table expressions Fast execution Interactive and compiled workflows

Explore

Try the thought, see the table

Ibex is designed for interactive data exploration. Type a pipeline into the REPL, evaluate the result, adjust your code, and repeat.

The same pipeline can run in a notebook, script, or compiled binary unchanged, without switching languages or API.

This session groups 10 million generated ticks and shows the actual output and measured runtime on my workstation.

Sample captured locally using :timing on in the REPL: 10M generated ticks. Timings vary by machine and run. Run Ibex live in your browser →

ibex — interactive session 


    

  


  

      

        Performance
        
Exceptional single-core performance

        
Ibex delivers leading per-core
        performance on common columnar queries. Multi-core execution is
        being expanding from that baseline.

      

      

        Language
        
Write columns as columns

        
Named clauses describe each transformation, and columns are
        referenced directly by name. In Ibex you write px, not
        pl.col("px").

      

      

        Deployment
        
One pipeline, many interfaces

        
Use Python or R notebooks, the web UI, or a REPL for exploration and
        modelling, then use C++23 codegen for native binaries. Type errors are
        caught before the pipeline runs.

      

    


  
  

    
Performance

    
Speed, measured

    

      Ibex is built for the hard part of table work: grouped aggregation,
      rolling time windows, joins, filters, null handling, and reshaping. The
       benchmark suite compares each query
      against the same operation in Polars, DuckDB, ClickHouse, DataFusion,
      pandas, data.table, and dplyr.
    

    
    

      

        7.2 ms
        
mean by symbol, 16M rows

        
Polars: 63.8 ms. DataFusion: 35.2 ms. Ibex single-threaded: 34.4 ms.

      

      

        18.9 ms
        
count by symbol x day, 16M rows

        
Polars: 290 ms. DuckDB: 66.1 ms. DataFusion: 49.1 ms.

      

      

        50.5 ms
        
rolling sum 1m, 16M rows

        
Polars: 146 ms. DuckDB: 1.24 s. DataFusion: 14.0 s.

      

    

    
    

      Ibex parallelises some operators by default. The benchmark page shows every
      engine at its default settings alongside a single-threaded run of each —
      including ibex-st, which makes Ibex's own threading gain
      visible and gives a same-core comparison against the other engines.
    

  


  
  

    
The vocabulary

    
A small vocabulary for table work

    

      Clauses compose inside a pipeline. Start with these common operations,
      then see the tutorial or
      reference for the full syntax.
    

    
filter
      select
      update
      by
      order
    

  


  
  

    
Install & run

    
Get Ibex on your machine

    

      Download a prebuilt release and start exploring immediately.
    


    

        
Run your first pipeline


# Unpack the archive for your platform, then:
./ibex --plugin-path ./plugins


import "data_gen";
let ticks = gen_ticks(100);

// Order symbols by total volume
ticks[
    select { traded = sum(volume) },
    by symbol,
    order { traded desc },
];

        

  


  
  

    
Where to go next

    
Keep going

    

      

        
Docs

        
Start with installation and a first pipeline, then work through language, data, and integration topics.

      

      

        
Benchmarks

        
Interactive timings and memory use against Polars, DuckDB, ClickHouse, DataFusion, pandas, and R.

      

      

        
Examples

        
Follow runnable data workflows, comparisons, and time-series examples.

      

      

        
Reference

        
A guided walk through every clause, with runnable snippets for deeper evaluation.