Ask HN: AI at home with multiple graphics cards
I was wondering if anyone has been testing and running AI locally, what applications and models have you been using and what have you been able to find useful, currently I have a not so old pseudoserver with a 3060 TI and a 2060 as well as a second computer with a 3060, what should or can I run, are there any models I can use over my local network that can benefit from feedback between two computers (agency models) ? According to this, you should be able to leverage multi-GPU machines using the stock-and-standard Llama.cpp: https://github.com/ggerganov/llama.cpp/pull/1703