GitHub - tsotchke/eshkol: High-Performance LISP-like language for Scientific Computing and AI written in C++

Eshkol

A Programming Language for Mathematical Computing

Eshkol is a Scheme-based programming language that unifies functional programming with native automatic differentiation, providing a mathematically rigorous foundation for gradient-based optimization, numerical simulation, and machine learning research. Built on Homotopy Type Theory foundations and compiled to native code via LLVM, Eshkol delivers mathematical correctness and deterministic performance without sacrificing the elegance of homoiconic Lisp syntax.

Why Eshkol?

Because gradient-based optimization should be native.

;; Define any differentiable function
(define (loss-function params data)
  (let ((predictions (neural-network params data)))
    (mean-squared-error predictions (data-labels data))))

;; Compute exact gradients - no approximation, no frameworks
(define gradients
  (gradient loss-function initial-params training-data))

;; Eshkol's autodiff system handles the mathematics automatically

Eshkol brings mathematical computing to Lisp and delivers what other languages promise:

• True automatic differentiation - Not numerical approximation. Exact symbolic, forward-mode, and reverse-mode AD with full vector calculus (∇, ∇·, ∇×, ∇²)
• Zero-overhead abstractions - Type-level proofs erase at compile time. Arena allocation is O(1). No runtime penalties for safety
• Deterministic performance - No garbage collector means no unpredictable pauses. Critical for real-time systems and production ML
• Native compilation - LLVM backend generates machine code competitive with hand-written C while preserving high-level expressiveness
• Mathematical rigor - HoTT type foundations ensure correctness properties are mathematically provable, not just tested

Quick Start

Installation

# Prerequisites: CMake 3.14+, LLVM, C17/C++20 compiler
git clone https://github.com/tsotchke/eshkol.git
cd eshkol

# Build
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build

# Optional: Build REPL
cmake --build build --target eshkol-repl

# Add to PATH
export PATH=$PATH:$(pwd)/build

Hello, World!

;; hello.esk
(display "Hello, Eshkol!")
(newline)

Design Philosophy

The language embodies three fundamental principles that distinguish it from existing mathematical computing environments:

1. Differentiation as a Language Primitive

Automatic differentiation is not a library feature—it is intrinsic to the language semantics. Eshkol provides three distinct AD modes (symbolic, forward-mode, reverse-mode) with native support for vector calculus operators (∇, ∇·, ∇×, ∇²) and arbitrary-depth gradient nesting through a sophisticated computational tape stack architecture.

;; Define any differentiable function
(define (loss-function params data)
  (let ((predictions (neural-network params data)))
    (mean-squared-error predictions (data-labels data))))

;; Compute exact gradients - no frameworks, no approximations
(define gradients (gradient loss-function initial-params training-data))

;; Nested gradients to arbitrary depth
(define hessian-trace 
  (trace (gradient (lambda (x) 
                     (car (gradient f (vector x))))
                   (vector x0))))

2. Deterministic Memory Management

Arena-based allocation with Ownership-Aware Lexical Regions (OALR) eliminates garbage collection entirely, providing O(1) allocation and deterministic deallocation. This architecture ensures predictable performance characteristics essential for real-time systems and production machine learning deployments.

;; Automatic scope-based memory management
(with-region 'computation
  (let ((large-dataset (load-training-data)))
    (train-model large-dataset)))
;; All memory automatically freed - no GC pauses, ever

;; Explicit ownership semantics when needed
(define resource (owned (acquire-expensive-resource)))
(transfer-to-subsystem (move resource))  ; Compile-time ownership tracking

3. Mathematical Rigor Through Type Theory

The gradual type system, grounded in Homotopy Type Theory, enables compile-time verification of dimensional correctness, resource linearity, and functional purity while preserving Scheme's dynamic flexibility. Type violations produce warnings without preventing compilation, allowing rapid prototyping with optional formal verification.

;; Types provide compile-time guarantees without runtime overhead
(define (matrix-multiply (A : Matrix<Float64, m, n>) 
                        (B : Matrix<Float64, n, p>)) : Matrix<Float64, m, p>
  (matmul A B))  ; Dimensions verified at compile-time

;; Linear resources with compile-time consumption tracking
(define quantum-state : (Linear (Superposition 8))
  (prepare-quantum-register))
(define measurement (measure quantum-state))  ; quantum-state consumed

Technical Implementation

Compiler Architecture

Eshkol is implemented as a production compiler written in C17/C++20, utilizing LLVM for native code generation. The implementation comprises:

• Recursive descent parser with comprehensive macro expansion (syntax-rules)
• HoTT type checker with bidirectional inference and dependent type support
• Modular LLVM backend with 20+ specialized code generation components
• Arena memory allocator with optimized allocation primitives
• Production JIT REPL enabling interactive development with persistent state

Runtime Representation

All values are represented as 16-byte tagged structures with 8-bit type tags and efficient union-based storage. Heap objects utilize 8-byte headers with subtype information, enabling type consolidation while preserving fine-grained semantic distinctions:

typedef struct eshkol_tagged_value {
    uint8_t type;        // Primary type classification
    uint8_t flags;       // Exactness, linearity, ownership flags  
    uint16_t reserved;   // Future extensibility
    union {
        int64_t int_val;     // Exact integers, symbols, characters
        double double_val;   // Inexact real numbers
        uint64_t ptr_val;    // Heap object pointers
    } data;
} eshkol_tagged_value_t;  // Exactly 16 bytes for cache efficiency

Memory Architecture

The arena allocator implements bump-pointer allocation with lexical scope tracking, delivering O(1) allocation performance and deterministic cleanup:

• 64KB default block size optimized for CPU cache behavior
• Scope stack management for nested region tracking
• Header-prefixed objects enabling type introspection and metadata
• Zero fragmentation through sequential allocation patterns

Automatic Differentiation System

Forward-Mode (Dual Numbers)

Utilizes dual number arithmetic for efficient first-derivative computation:

typedef struct eshkol_dual_number {
    double value;       // f(x)
    double derivative;  // f'(x)
} eshkol_dual_number_t;  // 16 bytes, perfect for SIMD

Reverse-Mode (Computational Graphs)

Implements tape-based automatic differentiation with support for arbitrary nesting:

typedef struct ad_tape {
    ad_node_t** nodes;       // Nodes in evaluation order
    size_t num_nodes;        // Current node count
    ad_node_t** variables;   // Input variable references
    size_t num_variables;    // Variable count
} ad_tape_t;

// Global tape stack for nested gradient computation
extern ad_tape_t* __ad_tape_stack[32];
extern int __ad_tape_depth;

Symbolic Mode

Performs compile-time AST transformation for symbolic differentiation with algebraic simplification.

Language Capabilities

Core Scheme Compatibility

Eshkol implements R5RS-compatible Scheme with modern extensions:

• 39 special forms: define, lambda, let/let*/letrec, if/cond/case/match, quote/quasiquote
• 300+ built-in functions: Complete numeric tower, list operations, string manipulation, I/O
• Hygienic macros: Full syntax-rules implementation with pattern matching
• Lexical closures: First-class functions with captured environment support
• Tail call optimization: Direct elimination and trampoline-based constant-stack recursion

Extended Capabilities

Automatic Differentiation (8 Operators)

(derivative f x)                    ; Forward-mode: ℝ → ℝ
(gradient f point)                  ; Reverse-mode: ℝⁿ → ℝⁿ  
(jacobian F point)                  ; Vector function: ℝⁿ → ℝᵐˣⁿ
(hessian f point)                   ; Second derivatives: ℝⁿ → ℝⁿˣⁿ
(divergence F point)                ; ∇·F: ℝⁿ → ℝ
(curl F point)                      ; ∇×F: ℝ³ → ℝ³
(laplacian f point)                 ; ∇²f: ℝⁿ → ℝ
(directional-derivative f p dir)    ; D_v f: directional derivative

N-Dimensional Tensors (30+ Operations)

;; Tensor creation and manipulation
(zeros 100)                         ; 1D zero vector
(ones 10 10)                        ; 2D identity preparation
(eye 4)                             ; 4×4 identity matrix
(arange 0 100 0.1)                  ; 1000-element range
(linspace -1 1 1000)                ; Linearly-spaced values

;; Linear algebra operations
(matmul A B)                        ; Matrix multiplication
(tensor-dot u v)                    ; Dot product / contraction
(transpose M)                       ; Matrix transposition
(solve A b)                         ; Linear system solution (LU decomposition)
(det M)                             ; Determinant (Gaussian elimination)
(inv M)                             ; Matrix inverse (Gauss-Jordan)

Advanced Memory Management

;; Arena-based regions with automatic cleanup
(with-region 'training-session
  (let ((model (initialize-large-model))
        (data (load-training-batch)))
    (gradient-descent-step model data)))
;; Memory deterministically freed

;; Ownership and borrowing semantics
(define resource (owned (acquire-gpu-buffer)))
(define processed (move resource))              ; Transfer ownership
(borrow processed (lambda (r) (analyze r)))    ; Temporary access
(define shared-ref (shared processed))          ; Reference counting

Advanced Type System

;; HoTT-based gradual typing with dependent types
(define (safe-array-access (arr : Vector<Float64, n>) 
                          (idx : Nat) {idx < n}) : Float64
  (vector-ref arr idx))  ; Bounds verified statically

;; Polymorphic function types
(define map : (∀ (A B) (-> (-> A B) (List A) (List B)))
  (lambda (f lst) ...))

;; Linear resource tracking
(define quantum-state : (Linear Superposition)
  (prepare-quantum-bits 8))
(measure quantum-state)  ; Consumption verified at compile-time

Demonstrated Applications

Neural Network Training

Complete multi-layer perceptron with automatic differentiation:

(require stdlib)

;; Define network architecture
(define (neural-network weights inputs)
  (fold (lambda (layer input)
          (relu (tensor-add (matmul layer input) 
                           (layer-bias layer))))
        inputs
        weights))

;; Loss function with L2 regularization
(define (total-loss weights data targets lambda-reg)
  (+ (mse (neural-network weights data) targets)
     (* lambda-reg (sum-squares weights))))

;; Training step with exact gradients
(define (train-step weights data targets learning-rate)
  (let ((grad (gradient (lambda (w) (total-loss w data targets 0.01)) weights)))
    (tensor-sub weights (tensor-mul grad (vector learning-rate)))))

;; Training loop
(define final-weights
  (fold train-step initial-weights training-batches))

Scientific Computing

Linear algebra and numerical methods implemented in pure Eshkol:

;; Solve system of equations: Ax = b
(define A (reshape (vector 4.0 1.0 2.0 3.0) 2 2))
(define b (vector 7.0 10.0))
(define x (solve A b 2))  ; Uses LU decomposition with partial pivoting

;; Numerical integration (Simpson's rule)
(define area (integrate (lambda (x) (* x x x)) 0.0 2.0 1000))

;; Root finding (Newton-Raphson method)
(define sqrt-2 
  (newton (lambda (x) (- (* x x) 2.0))     ; f(x) = x² - 2
          (lambda (x) (* 2.0 x))           ; f'(x) = 2x  
          1.0 1e-10 100))

;; Eigenvalue estimation (power iteration)
(define dominant-eigenvalue 
  (power-iteration covariance-matrix n 1000 1e-12))

Vector Calculus

Physical field analysis with native differential operators:

;; Electric field: E = ∇V (gradient of potential)
(define (electric-field potential point)
  (gradient potential point))

;; Divergence theorem verification: ∇·E  
(define (field-divergence E-field region-points)
  (map (lambda (p) (divergence E-field p)) region-points))

;; Curl for magnetic fields: B = ∇×A
(define (magnetic-field vector-potential point)
  (curl vector-potential point))

;; Wave equation: ∇²φ - (1/c²)(∂²φ/∂t²) = 0
(define (laplacian-operator scalar-field point)
  (laplacian scalar-field point))

Performance Characteristics

Compilation Performance

Memory Performance

• Arena allocation: O(1) bump-pointer, zero fragmentation
• Cache efficiency: 16-byte tagged values, 32-byte cons cells
• Deterministic cleanup: No GC pauses, predictable latency
• Memory usage: Competitive with manually managed C++

Autodiff Overhead

• Forward-mode: 2-3x slowdown (industry standard)
• Reverse-mode: 3-5x slowdown with O(n) memory (optimal)
• Symbolic: Zero runtime overhead (compile-time transformation)
• Nested gradients: Logarithmic space complexity via tape stack

Unique Technical Achievements

1. True Homoiconicity with Native Compilation

Unlike interpreted Lisps, Eshkol preserves code-as-data semantics while generating native machine code. Lambda functions maintain their S-expression representations through a runtime registry, enabling full introspection of compiled code.

2. Mixed-Type Lists with Complete Type Preservation

Each cons cell stores complete type information in both car and cdr positions, enabling heterogeneous lists with zero type erasure:

;; Each element retains full type information
(define mixed-list (list 42 "hello" (lambda (x) x) #(1.0 2.0 3.0)))
(map type-of mixed-list)  ; => (int64 string closure tensor)

3. Zero-Overhead Memory Safety

Arena allocation with compile-time ownership tracking provides memory safety without runtime cost. Linear types and borrow checking prevent use-after-free and double-free errors at compilation time.

4. Three-Mode Automatic Differentiation

The only language providing symbolic, forward-mode, and reverse-mode AD with seamless interoperability:

;; Symbolic (compile-time, zero overhead)
(diff (* x x) x)  ; → (* 2 x)

;; Forward-mode (dual numbers, exact derivatives)  
(derivative (lambda (x) (* x x x)) 2.0)  ; → 12.0

;; Reverse-mode (computational graphs, scalable to large functions)
(gradient (lambda (v) (complex-loss-function v)) parameter-vector)

5. HoTT-Based Type System

Dependent types enable compile-time verification of array bounds, matrix dimensions, and resource consumption:

;; Matrix multiplication with dimension checking
(define (safe-matmul (A : Matrix<Float64, m, k>) 
                    (B : Matrix<Float64, k, n>)) : Matrix<Float64, m, n>
  (matmul A B))  ; k dimensions must match - verified statically

Installation and Quick Start

Prerequisites

• CMake 3.14+ (build system)
• LLVM 10.0+ (backend and JIT)
• C17/C++20 compiler (GCC 8+, Clang 6+)
• Readline (optional, for REPL enhancements)

Build from Source

git clone https://github.com/tsotchke/eshkol.git
cd eshkol

# Configure build
cmake -B build -DCMAKE_BUILD_TYPE=Release

# Compile (parallel build recommended)
cmake --build build -j$(nproc)

# Optional: Build interactive REPL
cmake --build build --target eshkol-repl

# Add to path
export PATH=$PATH:$(pwd)/build

Verification

# Test basic functionality
echo '(+ 1 2 3)' | eshkol-run  # Should output: 6

# Test automatic differentiation  
echo '(derivative (lambda (x) (* x x)) 5.0)' | eshkol-run  # Should output: 10.0

# Test neural network capability
eshkol-run tests/neural/nn_working.esk

# Interactive REPL
eshkol-repl

First Program

Create hello.esk:

;; gradient.esk - Gradient computation demonstration
(require stdlib)

(define (quadratic x) (+ (* x x) (* 2 x) 1))
(define point 3.0)

(display "f(x) = x² + 2x + 1")
(newline)
(display "f(3) = ") (display (quadratic point)) (newline)
(display "f'(3) = ") (display (derivative quadratic point)) (newline)

;; Expected output:
;; f(x) = x² + 2x + 1  
;; f(3) = 16.0
;; f'(3) = 8.0

Execute: eshkol-run gradient.esk -o gradient && ./gradient

Language Features

Complete Scheme Foundation

• Lexical scoping with proper closures and captured environments
• Proper tail calls with direct elimination and trampoline support
• Hygienic macros via syntax-rules with pattern matching
• Delimited continuations (planned for v1.1)
• R7RS compatibility for core semantics and standard procedures

Mathematical Computing Extensions

• N-dimensional tensors with element-wise operations and broadcasting
• Linear algebra: LU decomposition, Gauss-Jordan elimination, eigenvalue estimation
• Numerical methods: Simpson's integration, Newton-Raphson root finding
• Vector field analysis: divergence, curl, Laplacian operators
• Statistical functions: variance, covariance, correlation analysis

Modern Language Features

• Pattern matching with algebraic data type support
• Module system with dependency resolution and namespace management
• Exception handling via guard/raise with typed exception hierarchies
• Multiple return values with destructuring assignment
• Hash tables with generic key/value types and O(1) average access

Ecosystem and Tooling

Interactive Development

The REPL provides full compilation and execution via LLVM JIT:

$ eshkol-repl

Welcome to Eshkol REPL v1.0.0-foundation
Type :help for commands, :quit to exit

eshkol> (define (f x) (* x x x))
eshkol> (gradient f (vector 2.0))
#(12.0)

eshkol> :type (gradient f (vector 2.0))
Vector<Float64, 1>

eshkol> :ast (lambda (x) (* x x))
(λ (x) (* x x))

eshkol> :load my-program.esk
Loaded 15 expressions from my-program.esk

Standard Library

Comprehensive mathematical libraries implemented in pure Eshkol:

• core.functional.*: composition, currying, combinators
• core.list.*: higher-order functions, transformations, queries
• core.data.*: JSON, CSV, Base64 parsing and serialization
• core.strings.*: 30+ string manipulation utilities
• math.esk: Linear algebra, numerical methods, statistics

Build System Integration

# Compile to native executable
eshkol-run program.esk -o program

# Compile to object file for linking
eshkol-run library.esk -c -o library.o

# Link with external libraries
eshkol-run main.esk -l linear-algebra -l graphics -o app

# Generate LLVM IR for analysis
eshkol-run --dump-ir program.esk  # Produces program.ll

Research Context and Comparisons

Positioning in the Language Landscape

Eshkol occupies a unique position combining the mathematical rigor of Julia, the functional elegance of Racket, the memory safety of Rust, and the AD capabilities of JAX, while maintaining true Lisp homoiconicity.

Research Contributions

1. OALR Memory Model: First practical implementation of ownership-aware lexical regions in a functional language
2. Native AD Integration: Language-level integration of three AD modes with nested computation support
3. HoTT Gradual Typing: Practical application of Homotopy Type Theory principles to gradual type checking
4. Homoiconic Compilation: Preservation of code-as-data semantics through native compilation

Documentation

For Users

• Language Reference: Complete function and syntax reference
• Architecture Guide: Technical implementation overview
• Quickstart Tutorial: Hands-on introduction with examples
• API Reference: Comprehensive function documentation

For Researchers

• Automatic Differentiation: Mathematical foundations and implementation
• Type System: HoTT theory and practical realization
• Memory Architecture: Arena allocation and OALR semantics
• Compiler Design: LLVM backend and optimization strategies

For Developers

• Contributing Guide: Architecture overview and development workflow
• Test Coverage: Comprehensive test documentation
• Overview: Start Here

Future Directions

Version 1.1 (Q1 2026): Complete Type Enforcement

• Full HoTT type checking with compile-time errors
• Dependent type refinement with theorem proving
• Linear resource verification with effect tracking

Version 1.2 (Q2 2026): Quantum Computing Integration

• Native quantum types with no-cloning enforcement
• Quantum circuit compilation to hardware backends
• Quantum-classical hybrid algorithm support

Version 1.5 (Q3 2026): Neuro-Symbolic AI

• Logic programming integration (miniKanren-style)
• Knowledge base representation and reasoning
• Differentiable programming with symbolic constraints

Version 2.0+ (Q4 2026): Multimedia Computing

• Real-time audio/video processing with zero-copy semantics
• GPU compute integration with memory safety
• Hardware I/O with resource lifetime management

Community and Contributions

Contributing

Eshkol welcomes contributions from researchers and practitioners. The codebase is architected for extensibility:

• Modular codegen: Add new backends by implementing CodegenModule interface
• Type system: Extend HoTT types through TypeFamily registration
• Standard library: Pure Eshkol implementations in lib/core/
• Testing: Comprehensive test coverage required for all features

See CONTRIBUTING.md for development setup and coding standards.

Research Applications

Eshkol is designed for:

• Integrated AI systems programming: Platform native self-improving, long-running agents and robotics
• Machine learning research: Native AD, deterministic performance, mathematical correctness
• Numerical analysis: High-precision computing, algorithm development, performance optimization
• Programming language research: Type theory, memory management, compilation techniques
• Computer graphics: Differentiable rendering, optimization-based animation

License and Citation

Eshkol is released under the MIT License. For academic use, please cite:

@software{eshkol2025,
  title = {Eshkol: A Programming Language for Mathematical Computing},
  author = {tsotchke},
  version = {1.0.0-foundation},
  year = {2025},
  url = {https://github.com/tsotchke/eshkol},
  note = {Scheme-based language with native automatic differentiation}
}

Technical Specifications

• Language: C17 runtime, C++20 compiler implementation
• Backend: LLVM 10.0+ with native code generation and JIT support
• Memory: Arena-based allocation with deterministic cleanup
• Types: HoTT-based gradual typing with dependent type support
• AD: Forward/reverse/symbolic modes with nested computation
• Testing: 300+ comprehensive tests with automated verification
• Platform: macOS (Intel/Apple Silicon), Linux (x86_64/ARM64), Windows (WSL)

Eshkol represents a synthesis of functional programming elegance, mathematical rigor, and systems programming performance. It is designed for researchers, engineers, and practitioners who require both expressive power and computational efficiency in their mathematical computing workflows.

Where Lisp meets differential geometry, and performance meets mathematical correctness.

Engineered for researchers and engineers building machine learning systems, numerical simulations, and differentiable algorithms where mathematical correctness and performance are non-negotiable.