Show HN: On-device embedding and vector search for Apple Devices, built in Zig
github.comdve handles embedding, vector storage, and search all in one library with a simple interface.
This last year I was working on an app for Macs which required doing vector search. The local AI/ML libraries for Apple devices are lacking, which means the only real option is to use an API. I didn’t want to do that, so I started working on my own library. It worked well enough for my uses that I eventually decided to split off my library from the app which used it.
I reached for Zig for its C-level performance and portability, but with modern conveniences. I used CoreML for the ML runtime, as it is the most natural way to run ML models on Apple devices. Unfortunately, it's not particularly common to release CoreML versions of models. I had to manually convert ML models from PyTorch/ONNX to CoreML in order to make embedding seamless.
Getting started with the library is straightforward, there are a few Zig examples in the repo. It also has experimental Swift and C/C++ bindings. It only supports Macs for now, but support for other platforms is planned. Feedback is greatly appreciated!
No comments yet.