Releases · lightly-ai/lightly-train

20 min read Original article ↗

v0.17.0

[0.17.0] - 2026-07-28

Added

Changed

  • Raise the default mixup and mosaic probability from 0.5 to 0.75 for
    ltdetrv2-s/m/l/x object detection models. Other LTDETR models are unaffected.
  • Lower the default backbone_lr_factor from 0.05 to 0.0025 for ltdetrv2-m/l/x
    object detection models. ltdetrv2-s and other LTDETR models are unaffected.

Deprecated

Removed

Fixed

  • Fix incorrect TensorRT inference for LT-DETR object detection and instance
    segmentation. TensorRT's optimizer/fusion pass around the GridSample ops used by
    deformable attention silently produced wrong activations, corrupting detections and
    masks, even with parser-compatible mode names. Deployment/export now replaces
    grid_sample with a gather-based bilinear equivalent that contains no GridSample
    op, so neither the parser mode-name issue nor the optimizer bug can apply; training
    keeps the faster fused grid_sample.

Security

What's Changed

Full Changelog: v0.16.4...v0.17.0

v0.16.4

[0.16.4] - 2026-07-24

Added

  • Add TIPSv2 vision backbones: dinov2/vitb14-tipsv2, dinov2/vitl14-tipsv2,
    dinov2/vitso400m14-tipsv2, and dinov2/vitg14-tipsv2.
  • Add LTDETRv2 instance segmentation with ltdetrv2-seg-s/m/l/x models and
    COCO-pretrained checkpoints for fine-tuning or out-of-the-box inference.
  • Add SAHI inference for LTDETRv2 instance segmentation through model.predict_sahi().

Changed

  • Warn when a built-in distillation teacher is used with non-ImageNet input
    normalization, which can produce invalid teacher features.

Removed

  • Remove the DINOv3.1 pretraining method.

Fixed

  • Preserve exponential-moving-average updates when resuming training, preventing a
    spurious validation-metric drop after the first resumed update.
  • Skip degenerate predicted boxes during validation visualization and log a warning
    instead of crashing.
  • Restore installation and package imports by using a released LightlySSL dependency.

What's Changed

Full Changelog: v0.16.3...v0.16.4

v0.16.3

[0.16.3] - 2026-07-22

Added

  • Add support for LingBot Vision backbones
    dinov3/vits16-lingbot, dinov3/vitb16-lingbot, and dinov3/vitl16-lingbot.
  • Add LingBot Vision backbones to the DINOv3 EoMT semantic, panoptic, and instance
    segmentation tasks: dinov3/vits16-lingbot-eomt, dinov3/vitb16-lingbot-eomt, and
    dinov3/vitl16-lingbot-eomt.
  • Add tiny, tiny-plus, and small Depth Anything V3 depth estimation models — the
    smallest and fastest so far, distilled from ViT-L: dinov3/dav3-relative-tiny,
    dinov3/dav3-relative-tiny-plus, dinov2/dav3-relative-small,
    dinov3/dav3-metric-tiny, dinov3/dav3-metric-tiny-plus, and
    dinov2/dav3-metric-small.

Changed

  • ONNX export for LT-DETR object detection now returns raw logits and normalized
    bounding boxes. Postprocessing is supposed to be applied outside the graph.

Deprecated

Removed

  • Remove the DINOv3 EoMT semantic segmentation training model_args.patch_size option.
    The patch size is now determined by the selected model name; use a
    dinov3/vit*32-eomt model, such as dinov3/vits32-eomt-coco, to train with patch
    size 32.

Fixed

  • Fix ONNX export verification for task models: Tensor.is_floating_point was
    referenced without calling it, so the always-truthy bound method forced integer
    outputs (e.g. labels) through the float comparison path instead of the intended
    exact-match check.

Security

What's Changed

Full Changelog: v0.16.2...v0.16.3

v0.16.2

[0.16.2] - 2026-07-10

Added

  • Add NaNCapture for fine-tuning debugging: when a NaN/Inf is detected in parameter
    gradients during training, save a self-contained capture (model state dict +
    TrainModel class/init kwargs + the step's microbatches + RNG state) to
    out_dir/debug/nan_capture/rank{R}/nan_capture.pt and halt training. Replay via
    lightly_train._debug.nan_capture.load_nan_capture(dir).replay() to deterministically
    reproduce the failure in a notebook/REPL. Enable with
    debug_args.nancapture.enabled=True.
  • Add ONNX and TensorRT export for depth estimation models via the export_onnx and
    export_tensorrt methods of DepthAnythingDepthEstimation.
  • Add a process_res_method argument to depth estimation predict/predict_batch:
    "square_resize" (default), "upper_bound_resize", or "lower_bound_resize".

Changed

  • Consolidate the separate Depth Anything V2/V3 depth estimation task models into a
    single config-driven DepthAnythingDepthEstimation model.
  • Depth estimation predict/predict_batch now default to square_resize (previously
    aspect-preserving upper/lower-bound per model), changing default depth outputs. Pass
    process_res_method="upper_bound_resize"/"lower_bound_resize" to restore the
    previous geometry.

Removed

  • Removes the DINOv3LTDETRObjectDetection and DINOv2LTDETRObjectDetection classes in
    favor of the new LTDETRObjectDetection class. For the user, this changes nothing,
    since they don't use the class directly, but only its methods, which will remain
    available.

Fixed

  • Fixed an issue with legacy LT-DETR checkpoints that used a [0,1] normalization instead
    of the now-default ImageNet normalization.

What's Changed

Full Changelog: v0.16.1...v0.16.2

v0.16.1

[0.16.1] - 2026-06-26

Added

  • Log the total gradient norm (gradient_norm) during finetuning. It is shown in the
    console as grad_norm and written to all configured loggers (JSONL, TensorBoard,
    Weights & Biases, MLflow). It is the pre-clipping norm when gradient clipping is
    enabled and the total gradient norm computed without clipping otherwise.

Fixed

  • Fix checkpoint loading with load_model for legacy checkpoints of LT-DETR.

What's Changed

Full Changelog: v0.16.0...v0.16.1

v0.16.0

[0.16.0] - 2026-06-25

Added

  • Add LTDETRv2, an improved object detection model that reaches 50.7
    mAP50:95 on the COCO 2017 validation set (+1 mAP50:95 over the
    previous LTDETR with a 55% shorter training schedule) and 5.4ms latency on an NVIDIA
    T4 (TensorRT, FP16, batch size 1, 640x640). Use the compact ltdetrv2-s/m/l/x models,
    which are built on EdgeCrafter ECViT backbones.
  • Add depth estimation inference with Depth Anything V2 and V3 models, covering both
    relative and metric depth (dinov2/dav2-relative-*, dinov2/dav2-metric-*,
    dinov2/dav3-relative-large, dinov2/dav3-metric-large). Checkpoints are converted
    to the LightlyTrain format; the Apache-2.0 models are hosted for download, while the
    CC-BY-NC-4.0 Depth Anything V2 variants must be converted locally with
    convert_checkpoint_dav2.
  • Add the benchmark_object_detection command (beta) to measure inference
    performance of an object detection model on a validation dataset. It reports detection
    accuracy (mAP/mAR, including per-class mAP) and timing statistics (latency and
    throughput), and writes a JSON result and a human-readable Markdown report. This is
    useful to compare inference backends and precisions before deployment. See the
    benchmarking documentation
    for details.
  • Add Slicing Aided Hyper Inference (SAHI) for EoMT instance segmentation to improve
    small instance recall at inference via model.predict_sahi() method.

Changed

  • Update LTDETRv2 training defaults: the default batch_size is now 32 (was 16),
    the default training schedule is 266_112 steps (6x ECDet-S, ~72 epochs at batch size
    32), backbone_lr_factor is now 0.05 (was 1e-2), and lr_warmup_steps defaults
    to "auto" so short runs no longer warm up for longer than they train.

What's Changed

Full Changelog: v0.15.1...v0.16.0

v0.15.1

[0.15.1] - 2026-05-28

Added

  • Add image logging for all fine-tuning tasks. Sample predictions are saved locally and
    forwarded to configured loggers (TensorBoard, Weights & Biases, MLflow).
  • Add predict_batch for batched inference across all fine-tuning tasks.
  • Configurable patch_size for DINOv3 LT-DETR models.
  • Added decoder and losses from D-FINE.
  • Add support for choosing LR scheduler for LTDETR object detection. You can specify the
    scheduler with model_args.scheduler_name, choosing either linear or flat-cosine.
  • ONNX export for models that support dynamic batch sizes can now specify if the batch
    size should be dynamic with the dynamic_batch_size keyword argument.
  • Explicit support for all YOLO26 variants.

Fixed

  • Fix PicoDet fine-tuning with mismatched num_classes.
  • Fix DINOv3 LT-DETR patch size precedence so model_args.patch_size overrides the
    backbone default.

What's Changed

Full Changelog: v0.15.0...v0.15.1

v0.15.0

[0.15.0] - 2026-04-16

New Distillation Method and Custom Teacher Models: We release the new
Distillationv3
method that achieves better generalization across fine-tuning tasks and works better
with DINOv3 teacher models. The new method also supports using
custom teacher models

Added

  • Add distillationv3 method for dense as well as global feature distillation.
  • Add support for custom teacher models with distillationv3.
  • Add support for the new
    EUPE models from Meta for all
    distillation, pretraining, and fine-tuning tasks. For example, use
    dinov3/vits16-eupe instead of dinov3/vits16 to load the EUPE pretrained ViT-S/16
    model. See the
    documentation
    for all supported models.
  • Add Mosaic augmentation for LTDETR object detection training.
  • Add CopyBlend augmentation for LTDETR object detection training.
  • Add MixUp augmentation for LTDETR object detection training.
  • Add logging of completed epochs to the console and the loggers.
  • Add support for COCO object detection dataset format.
  • Semantic segmentation now allows one to specify classes from a JSON file.

Changed

  • Default distillation method is now v3 (previously v2), with a DINOv3 teacher instead
    of a DINOv2 teacher. Previous default still available with method="distillationv2".
  • Make ScaleJitter in LTDETR step-aware. Now you can stop the augmentation by adding a
    step_stop args like the following
    transform_args={"scale_jitter": {"step_stop": 10000}}

Removed

  • Remove StopPolicy and use ActivationPolicy instead for more fine-grained control
    over the step-aware augmentations.

What's Changed

New Contributors

Full Changelog: v0.14.3...v0.15.0

v0.14.3

@guarin guarin released this

26 Mar 08:52

Added

  • Add support for DINOv2 panoptic segmentation inference and fine-tuning.
  • Add support for metric_args in all fine-tuning commands to allow configuring the metrics used for validation and best model checkpointing. See the documentation for details.
  • Add option to freeze the backbone for all EoMT models during training with the model_args={"backbone_freeze": True} argument.
  • Add YOLOOrientedObjectDetectionDataset for loading YOLO oriented object detection datasets with (cx, cy, w, h, angle) bounding boxes.

Changed

  • PicoDet switched to O2O NMS-free inference/export, updated L preset to picodet/l-640, and improved ONNX/TensorRT export robustness.

Removed

  • It is no longer possible to set seed=None. Instead, an integer seed must be provided for reproducibility. This fixes a bug where recent PyTorch Lightning versions (>=2.2) no longer generate random seeds when seed=None is set.
  • LTDETR no longer supports the detector_weight_decay and backbone_weight_decay arguments. Instead use the general weight_decay argument.

Fixed

  • Fix incorrect model name format in export_model() log example for DINOv2 and DINOv3 packages. The example now shows the correct format (without prefix) that works with get_model().
  • Fix the wrong config of ScaleJitter sizes of LT-DETR.
  • Fix a bug when loading DINOv3 LTDETR checkpoints that were not pretrained on COCO which resulted in backbone weights not being loaded.

What's Changed

New Contributors

Full Changelog: v0.14.2...v0.14.3

v0.14.2

@guarin guarin released this

24 Feb 15:18

[0.14.2] - 2026-02-24

New Classification Support: You can now train image classification models with LightlyTrain! See the classification documentation for more information.

Added

  • Add
    classification support
  • Add support for frozen backbone training in LTDETR and Picodet object detection
    models. Set model_args={"backbone_freeze": True} in train_object_detection to
    freeze the backbone and reduce VRAM usage.
  • Add LTDETR support for DINOv3 ViT-B/L and DINOv2 ViT-L/B/G models. Pretrained weights
    are not yet available for these models.
  • Add support for fine-tuning DINOv2 models for instance segmentation with the
    train_instance_segmentation command. See the
    instance segmentation documentation
    for more information.

Fixed

  • Filter invalid bounding boxes in instance segmentation
  • Fix incorrect logging of training times.

What's Changed

New Contributors

Full Changelog: v0.14.1...v0.14.2