How much cost would it take to run a LLMs locally
If I want to run LLMs model like Qwen or gpt locally how much would it cost me and I want to connect it to my main website creating an API link, also which models would be best
First, check what LLMs your system can actually handle based on your RAM/VRAM, then choose the model variant accordingly.
If you want to use the model through an API for your website, a good setup would be a dedicated machine/server to host it, since running LLMs locally can consume a lot of memory.
Depending on your hardware, you can look at smaller variants of Gemma or Qwen Coder.
Another option is to use a hosted inference provider. It’ll cost a little, but you’ll usually get much faster inference compared to running locally especially if your system isn’t high-end.
What should be the minimum hardware requirements I got Nvidia processor with a high quality graphic card and 16 gb ram
Try TurboLLM (https://turbollm.dev/).
go to https://huggingface.co/settings/hardware?next=%2Fmodels and add your hardware, it will show models you can run on it