Lightweight Model Serving

Hello,

I recently completed a small project where I explored the potential improvements in model serving by switching from naive PyTorch setups to the ones utilizing ONNX Runtime, as well as ONNX Runtime with Rust's Actix web server.

Here are the findings:

https://github.com/martynas-subonis/model-serving

Additionally, I wrote a broader article on Substack for anyone interested:

https://github.com/martynas-subonis/model-serving

Hope somebody finds it interesting/useful as well.