Show HN: Deploy Hugging Face Models to AWS Lambda

1 points by cnuss a year ago · 0 comments · 1 min read

Reader

I've been working on Scaffoldly since 2020 to simplify AWS Lambda deployments. Recently discovered you can run Hugging Face models efficiently using EFS for caching. Here's what's interesting:

   - Uses EFS for model file persistence
   - Pre-downloads models after deployment for faster cold starts
   - Cold start: ~20s (model loading), warm requests: 5-20s (CPU inference)
   - Fully automated container builds and deployment
   - Works with private/gated models via HF_TOKEN

Example deployment:

  npx scaffoldly create app --template python-huggingface
  cd python-huggingface && npx scaffoldly deploy

Scaffoldly is Open Source and I'm excited for all feedback and contributions from the community!

https://github.com/scaffoldly/scaffoldly

https://github.com/scaffoldly/scaffoldly-examples/tree/pytho...

No comments yet.

Settings

Show HN: Deploy Hugging Face Models to AWS Lambda

Keyboard Shortcuts