a tiny deep learning library, grown from scratch

// numpy-only · zero dependencies

A deep learning library, grown one tensor at a time.

picodl is a from-scratch autograd engine and neural network stack - every gradient traced by hand, every op built on plain numpy. No hidden framework underneath.

$ pip install picodl-nn

loss ↓ as optimizer.step() runs

// what's inside

Seven small modules, one coherent stack

Each piece does one job and hands a plain Tensor to the next. Nothing is generated for you - every layer, loss, and optimizer is code you can read start to finish.

tensor.py

Autograd engine

Tracks every op into a graph, topologically sorts on .backward(). 25+ differentiable ops: matmul, conv2d, softmax, attention primitives, and more.

layers.py

Layers

Linear, Conv2D, Embedding, LayerNorm, BatchNorm2D, MaxPool2D, AvgPool2D, GlobalAvgPool, Dropout, GELU, TiedLinear.

loss.py

Loss functions

MSE, BinaryCrossEntropy, NLLLoss, CrossEntropyLoss - built from the same primitive ops as everything else.

optim.py

Optimizers

SGD, RMSprop, Adam, AdamW - decoupled weight decay included, per-parameter state tracked by identity.

nn.py

NeuralNet

Chains layers, exposes params for the optimizer, saves and loads weights to any file extension you like.

data.py / train.py

Data + training loop

BatchIterator for mini-batches, a train loop that accepts raw numpy or Tensor input directly.

// thirty seconds in

Build, train, save

A small classifier - GELU hidden layer, Adam optimizer, saved to disk when done.

from picodl.nn import NeuralNet
from picodl.layers import Linear, GELU
from picodl.loss import CrossEntropyLoss
from picodl.optim import AdamW
from picodl.train import train

net = NeuralNet([
    Linear(784, 128),
    GELU(),
    Linear(128, 10),
])

train(net, x_train, y_train,
      num_epochs=10,
      loss=CrossEntropyLoss(),
      optimizer=AdamW(lr=0.001, weight_decay=0.01))

net.save("model.picodl")