Using Fable to one-shot weights outperforms training (97.8% vs. 94.6%)

This is just a huge if/then/else chain setup as model weights.

Seems like it made if/then/else decisions until it was optimized to the test. I bet there's a massive over-fitting.

Shared by Jacob Lauritzen (@jacsebl), CTO at Legora.

They got Fable to directly hand-write the weights of a 100k-parameter neural network to classify real vs. fake legal citations, outperforming traditional gradient descent training while staying super sparse.