Neural Thickets · MIT

1 min read Original article ↗

θ (pretrained) Gaussian search window Evaluate on D_train Score each seed Top-K Select best K seeds Majority Vote Ensemble K

# Training: Select top-K seeds based on D_train performance

seeds = [sample_seed() for _ in range(N)]

sigmas_per_seed = [sigmas[i // (N // len(sigmas))]

for i in range(N)]

## evaluate all perturbed models

scores = [evaluate(theta + sigmas_per_seed[i] * eps(seed[i]), D_train)

for i in range(N)]

top_indices = topk(scores, K).indices

# Inference: Ensemble predictions on test input x

answers = [generate(theta + sigmas_per_seed[i] * eps(seed[i]), x)

for i in top_indices]

prediction = majority_vote(answers)