Something has happened in the last few months, and I'm not sure how many in the space have noticed. Open source LLMs have gotten very good at tool calling specifically, and very cheap. We're talking less than 30 cents for a million tokens of output! And, these cheap models can reliably call tools like executing some apple script or calling reminders. The 'harness' concept has also really taken off, providing models with skills to perform critical functions like operating Apple Reminders. This all coalesces into a personal assistant that costs $500 for setup (the cost of a MacBook Neo or Mac mini) and ~$5 a month in ongoing costs that can manage some email, update your calendar, create a morning briefing, and much more. You sand down the UX a little (easier setup, less terminal, no accounts) and you can replace systems that companies like apple and google are spending billions of dollars to do.
Monetary Cost
To access a personal AI assistant is now cheap and good. You need some sort of mac computer running that has access to your calendar, reminders, common things an assistant would need access to. A Mac is nice because the AI can interact with these services directly, adding items to your calendar because you log in with your Apple account. So what's the cheapest Apple computer? Either the new Macbook Neo or the Mac Mini. Both have their trade-offs and benefits, so it's mostly up to user preference. You just want something you can leave running so when you text it, there is a computer to answer/route your question. Then comes StepFun — it's a very cheap model that has excellent tool calling capabilities and even moderate programming ones. I estimate the monthly cost to be under ~$5 for AI usage, which is incredible! If you want to not depend on 3rd party LLM services, you'd need around ~128GB of VRAM to run StepFun locally. This equates to around $3500 for a Mac Studio or $5000 for a Macbook Pro with that much RAM. So there isn't a time horizon where it pays off to do one, but would make for a good maybe threshold in the future of how small we can get real frontier performance?
Skill Cost
Unfortunately, this is where we are now. The skills involved to set all this up are just too difficult for a regular person. I can evangelize the benefits, but I can't turn to my hair stylist wife and have her also set this up. You need to know what an API key is, what an agent harness is, have an opinion on LLMs. Normal people do NOT care about any of that. They just want to be able to talk to their assistant in plain English to add 15 things to their calendar for next week.
Current Tech Stack
hermesagent harnessStepFun 3.5 FlashLLM- LLM accessed via OpenRouter
- API key provisioned for hermes harness to access LLM on OpenRouter
- Google Voice Phone #
- Signal-CLI setup
- Update
hermesto reference signal as messanger - Probably some dumb debugging
And I Feel Fine
It's fulfilling the promise we got from Siri over a decade ago. I have a personal assistant in my pocket that can actually manage my schedule. As a fresh dad of 2 new twins and a senior engineer, I would love an automated way to find as much time to relax/spend time with family as possible.
Out Loud
The quiet part out loud here is that all the parts to a workable AI assistant exist, have established workable patterns, and are cheap. I need to devote real time to game out what I want to set up and how without burning too much money. Maybe find out what I'm happy with is my "assistant budget" (the Claude/Codex $20 seems like too much for some of the reliably simple stuff I want) and change it up from there. I don't know if I have the bandwidth/budget to really build out the actually simple "assistant for your grandma that she can talk to" that would take the world by storm. And I think the current toolset that exists in the wild can do this, it just needs to be arranged. I think the issue is that engineers generally fail to solve really difficult last-mile UX issues that lead to mass adoption like the Wii or the iPhone.
OpenClaw/Hermes prove that there is an agentic workable toolset for nerds or people who are interested enough to get past the hurdles. I think there's still an iPhone moment out there. Good luck everyone!