GitHub - thevoxium/bare-lm: lightweight autograd lib in C

A minimal autograd tensor library in C.

Installation

You can install bare-lm in two ways.

Option 1: Build and install from source

Requirements:

• A C compiler (cc, gcc, or clang)
• make
• OpenBLAS (headers + library)

Install OpenBLAS first:

# macOS (Homebrew)
brew install openblas

# Ubuntu / Debian
sudo apt update && sudo apt install -y libopenblas-dev

Then build and install:

git clone https://github.com/thevoxium/bare-lm.git
cd bare-lm

# macOS
make OPENBLAS_PREFIX=/opt/homebrew/opt/openblas

# Linux
make

sudo make install

By default this installs to /usr/local/include/bare.h and /usr/local/lib/libbare.{so,dylib}.

Option 2: Install from GitHub Release tarball

Each tagged release publishes prebuilt archives for Linux and macOS:

• bare-<version>-linux.tar.gz
• bare-<version>-macos.tar.gz

After downloading and extracting a release tarball, install headers and library:

tar -xzf bare-<version>-<artifact>.tar.gz
cd bare-<version>-<artifact>
sudo cp include/bare.h /usr/local/include/
sudo cp lib/libbare.* /usr/local/lib/

Using the library

If your system does not search /usr/local/lib by default, add -L/usr/local/lib and -I/usr/local/include.

Example: XOR with a 2-layer MLP

#include "bare.h"

int main() {
  Memory *mem = create_global_mem(1 << 28);
  ParameterList *pl = create_param_list(mem);

  int x_shape[] = {4, 2};
  int y_shape[] = {4, 1};

  Tensor *x = tensor_init(mem, x_shape, 2, PERM);
  Tensor *y = tensor_init(mem, y_shape, 2, PERM);

  float x_data[] = {0, 0, 0, 1, 1, 0, 1, 1};
  float y_data[] = {0, 1, 1, 0};
  for (int i = 0; i < 8; i++) x->data[i] = x_data[i];
  for (int i = 0; i < 4; i++) y->data[i] = y_data[i];

  Linear *l1 = create_linear(mem, pl, 2, 8);
  Linear *l2 = create_linear(mem, pl, 8, 1);

  load_checkpoint(pl, "xor.btw");

  for (int epoch = 0; epoch < 500; epoch++) {
    zero_grad(pl);

    Tensor *h = linear_t(mem, l1, x);
    h = relu_t(mem, h);
    Tensor *o = linear_t(mem, l2, h);
    o = sigmoid_t(mem, o);

    Tensor *loss = mseloss_t(mem, o, y);
    backward(mem, loss);

    if (epoch % 100 == 0)
      printf("epoch %3d  loss=%.4f\n", epoch, loss->data[0]);

    sgd_step(pl, 0.1f);
    reset_temp_mem(mem);
  }

  save_checkpoint(pl, "xor.btw");

  free_global_mem(mem);
}

Walkthrough

Step 1: Create memory and a parameter list. A single Memory object holds two arenas (permanent and temporary). 1 << 28 = 256 MB per arena. A ParameterList tracks all trainable tensors.

Memory *mem = create_global_mem(1 << 28);
ParameterList *pl = create_param_list(mem);

Step 2: Allocate persistent tensors. Inputs x and y are plain PERM tensors (not trainable). Pass PERM so they survive reset_temp_mem.

Tensor *x = tensor_init(mem, x_shape, 2, PERM);
Tensor *y = tensor_init(mem, y_shape, 2, PERM);

Step 3: Create layers with auto-registration. create_linear allocates weights and bias as PERM tensors and automatically adds them to the parameter list.

Linear *l1 = create_linear(mem, pl, 2, 8);
Linear *l2 = create_linear(mem, pl, 8, 1);
// pl now contains l1->weights, l1->bias, l2->weights, l2->bias

Step 4: Forward pass. Every operation (linear_t, relu_t, sigmoid_t, mseloss_t) allocates its result from the temp arena. No malloc calls, no cleanup code.

Tensor *h = linear_t(mem, l1, x);
h = relu_t(mem, h);
Tensor *o = linear_t(mem, l2, h);
o = sigmoid_t(mem, o);
Tensor *loss = mseloss_t(mem, o, y);

Step 5: Backward pass. backward builds a topological sort of the computation graph (also temp-allocated) and propagates gradients. Intermediate tensor data is still valid at this point.

Step 6: Zero grads, update, reset. zero_grad clears all parameter gradients. sgd_step applies SGD to all parameters in one call. After gradients are consumed, reset_temp_mem zeroes the temp arena in O(1). All intermediate tensors from the forward pass are gone.

zero_grad(pl);
sgd_step(pl, 0.1f);
reset_temp_mem(mem);

Step 7: Cleanup. free_global_mem releases both arenas and the Memory struct.

Memory Model

All allocations go through allocate_mem(mem, size, perm). There are no individual free calls.

reset_temp_mem resets the temp arena pointer to 0. It is called after both forward and backward complete, because backward reads intermediate tensor data.

epoch N:  forward (temp fills) → backward (reads temp) → update → reset_temp_mem
epoch N+1: forward (temp fills from 0) → ...

ParameterList

ParameterList is a dynamic array of trainable tensors. It is a typedef of Dt_array allocated in the PERM arena.

ParameterList *pl = create_param_list(mem);

// Manual registration
param_list_add(mem, pl, my_tensor);

// Or automatic — create_linear registers weights and bias for you
Linear *l = create_linear(mem, pl, 128, 64);

Once built, the parameter list drives the training loop:

zero_grad(pl);       // zero all parameter gradients
backward(mem, loss); // backprop
sgd_step(pl, 0.01f); // update all parameters

API Reference

Memory

ParameterList

Tensor Creation

Arithmetic

Math

Reductions

Activations

Masking

Linear Algebra

Embedding

Shape Operations

Loss Functions

Layers

Autograd

All operations register backward passes and track parent tensors automatically.