If you are curious about large language models.

You can run a very interesting model that was created by Facebook ( https://ai.facebook.com/blog/large-language-model-llama-meta-ai/ archive) and fine-tuned by a Stanford group ( https://github.com/tatsu-lab/stanford_alpaca archive ) locally. Amazingly it requires nothing more than ~10GB of RAM or so and a decent CPU. Soon enough we will have such things running in our phones…

1. Install a recent version of node (18 or above).
   
2. Run:
   

npx dalai alpaca install 13B

This will take a while and will download the model (a multi-GB file).


3. Test this with the command npx dalai serve -> this will open a simple web interface under http://localhost:3000/
   
4. Clone the following repository:
   

https://github.com/antimatter15/alpaca.cpp archive


5. Enter inside it and use make chat to compile.
   
6. To run it in a chat interface use a command similar to
   

./chat -m /home/igorhvr/dalai/alpaca/models/13B/ggml-model-q4_0.bin

You will need to adjust your path to the place where the prior command npx dalai alpaca install 13B put your model.


7. Voilá!