Optimizers on N-Dimensional Terrain

Playback

Optimizer

SGD Momentum RMSProp Adam AdamW Muon

Learning rate Eff: 0.20

Batch size 16

LR schedule

Step size k: 40 Gamma: 0.5

Min factor: 0.1

Gamma: 0.995

Warmup frac: 0.10 Min factor: 0.1

Dimensions

n_dimensions 2D

Info

Steps: 0 / 640
Grad clip: 6.0 | h: 1e-3

All Time Best Loss: --
Cur. Best Loss: --
Cur. Loss: --

Loss curve