Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF · Hugging Face

2 min read Original article ↗
  • Libraries
  • Transformers

    How to use Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF with Transformers:

    # pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF", device_map="auto")
  • Notebooks
  • Google Colab
  • Kaggle
  • Local Apps Settings
  • llama.cpp

    How to use Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF with llama.cpp:

    Install (macOS, Linux)
    curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
    Install from WinGet (Windows)
    winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
    Use pre-built binary
    # Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
    Build from source code
    git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
    Use Docker
    docker model run hf.co/Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
  • LM Studio
  • Jan
  • Ollama

    How to use Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF with Ollama:

    ollama run hf.co/Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
  • Unsloth Desktop
  • Docker Model Runner

    How to use Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF with Docker Model Runner:

    docker model run hf.co/Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
  • Lemonade

    How to use Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF with Lemonade:

    Pull the model
    # Download Lemonade from https://lemonade-server.ai/
lemonade pull Quazim0t0/Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF:Q4_K_M
    Run and chat with the model
    lemonade run user.Phi4.Turn.R1Distill_v1.5.1_Q4_k-GGUF-Q4_K_M
    List all available models
    lemonade list
  • Atomic Chat