Noise: Speedrun for Pretraining Without Backpropagation

https://x.com/industriaalist/status/2113365351455219760

> In the 10 days since we released the Noise speedrun, the time to pretrain a transformer went from 1.76 hours to 1.6 seconds, a 4,000x (!!!) speedup. Dust itself was up to 10,000x more compute-efficient than the previous state-of-the-art.