Ask HN: What do you use local models for?
What are the use-cases for using small 8B-26B models? How are these helpful versus more specific ML algorithms or just simple if/else. I am thinking about smart home environment, or how to apply it to my SWE work.
2 threads
I use this https://arkasentinel.dev, it is not meant to code but to ensure the code is within the repo structure and architectural boundary's its completely local, zero egress.
ive been experimenting with pointing some custom opencode agents and commands specifically at local models for really small things like commit messages which definitely don't need an LLM