Phlox — A full-featured, self-hostable AI platform

Self-hostable · Open source · Apache 2.0

A full-featured AI platform
you actually own.

Phlox is a self-hostable AI platform — an agentic tool-using harness with a live artifact canvas, custom assistants, reusable skills, document RAG, configurable code execution, an OpenAI-compatible gateway, and per-user cost accounting and budgets — running over any model provider: AWS Bedrock or any OpenAI-compatible endpoint, including fully local models.

localhost:5173

Bring your own model. Or run it all locally.

Named provider profiles cover AWS Bedrock and any OpenAI-compatible endpoint — OpenAI, LiteLLM, or a local runtime. Point Phlox at Ollama, LM Studio, or vLLM and the whole stack — chat and RAG embeddings — runs offline with no cloud API key. Switch profiles live, with a built-in connection tester.

• Define as many provider profiles as you like, switch between them instantly
• Local embeddings (e.g. nomic-embed-text) keep RAG fully offline
• Edit profiles, pricing, and limits live — no server restart required

config.yml

default_profile: local-ollama
profiles:
  local-ollama:
    type: openai
    label: "Ollama (local)"
    endpoint: http://localhost:11434/v1
    api_key: ollama       # ignored by Ollama
    model: qwen3.6:35b
    supports_tools: true

Everything in one app

A complete assistant, not just a chat box

Phlox bundles the pieces you'd otherwise stitch together yourself — each one self-hosted and under your control.

💬

Streaming chat

Conversation history with rename, delete, search & export. Edit/regenerate messages, markdown with highlighted, copyable code, plus Mermaid diagrams and LaTeX math.

🤖

Agentic harness

The model uses tools in a loop — filesystem, shell, Python/Node execution, document search, plus planning, sub-agents, memory, and checkpoints — in a per-conversation workspace. Isolated execution uses the container or AgentCore runner.

🤝

Human-in-the-loop

Pause on sensitive tools, approve or deny, then resume. The run state is persisted, so approvals survive disconnects.

🧰

Code execution & artifacts

Run code with captured output and inline artifacts. A Workspace Files panel lets you browse and download everything the agent created.

🖥️

Artifact canvas

HTML and Markdown files the agent writes open in a live, resizable side panel — a sandboxed preview with a raw-source toggle — so web apps, reports, and diagrams run right next to the chat.

✨

Agent skills

Named, reusable instruction sets that teach the agent a workflow. Type / to invoke one, or let the agent auto-activate the right skill — import/export-compatible with the Anthropic SKILL.md format.

🎭

Custom assistants

Admins publish named personas on any configured model — custom system prompt, starter prompts, a shared knowledge base, and per-assistant capability limits. Users pick one on the new-chat screen.

📚

Documents & RAG

Upload PDF, DOCX, TXT, MD, or code — attach documents to a message or reference them inline with @. Hybrid dense + sparse search over Qdrant with reranking and citations. Works offline.

🌐

Opt-in web search

A per-prompt composer toggle exposes web_search (zero-config ddgs or SearXNG) so the agent can discover current sources before fetching pages with web_fetch.

🧠

Cross-conversation memory

Durable facts are saved and semantically recalled across chats, so the assistant remembers you from one conversation to the next.

🖼️

Multimodal

Attach images to messages for vision models, persisted and replayed into the provider as image content parts.

🔌

MCP integration

Connect Model Context Protocol servers from the UI — stdio or streamable-HTTP with bearer/header auth; their tools join the model's toolset automatically, no code required.

🚪

OpenAI-compatible gateway

Mint per-user API keys and call Phlox from any OpenAI SDK via /v1/chat/completions — with the same per-user cost accounting.

💵

Usage & cost accounting

Per-message token and cost in the UI, plus an admin chargeback view by month × user × department × model, with CSV export for finance.

🧮

Spend budgets

Set a monthly cost cap per user or department. Warn at an adjustable threshold, then block priced models once the budget is reached — across chat and the API gateway. Resets each month.

🛡️

Optional PII pattern guardrails

When enabled, redact or block emails, phone numbers, SSNs, credit cards, API keys, and custom regex matches — in provider-bound text and streamed output across chat and the gateway.

🎨

Theming

Phlox Dark by default, with Light, Fred Hutch, Hutch Night, Sandstone, and a CRT phosphor-green Terminal — instant switching via a CSS-variable token system.

The agentic core

A real agent, not "chat that calls tools"

Each turn, the model works in a loop — calling tools, planning, and recovering — inside a per-conversation workspace you can inspect, snapshot, and roll back. Choose container or AgentCore when execution isolation is required.

Custom assistants

Ship curated personas to your whole team

Turn any configured model into a purpose-built assistant: a name, an avatar, a custom system prompt, and starter prompts — published once by an admin, picked by users on the new-chat screen, and pinned per conversation.

• Built on any provider profile and model you've configured — cloud or local
• Optional shared knowledge base — documents every user of the assistant can search
• Per-assistant capability limits: allow or block web search, personal documents, and agent tools
• Public to all users or restricted — prompt edits apply everywhere, instantly

Knowledge & memory

Your documents, searched the right way

Upload PDFs, Office docs, markdown, or source code. Phlox parses, chunks, and embeds them into Qdrant, then retrieves with true hybrid search — a dense semantic vector and a sparse lexical vector per chunk, fused with RRF and reranked, returning numbered citations the model is instructed to cite.

• Attach documents to a message, or reference existing ones inline with @
• Global knowledge base, per-conversation, or per-assistant document scoping
• Dependency-free sparse vectors and reranker work fully offline
• The database stays the source of truth — the index can always be rebuilt
• Cross-conversation memory recalls durable facts into every turn

Built for teams

Multi-user controls and accountability

Auth is on by default. Owner-scoped routes and cross-user regression tests protect private user content, while sensitive tools run behind a permission gate you control.

🔐

Auth & SSO

Local accounts (bcrypt + JWT) or Microsoft Entra ID SSO. The Entra authorization-code flow validates one-use state, nonce, PKCE, issuer, tenant, and audience. Administrators manage accounts without a bypass to users' private content.

📦

Three execution runners

The fast local subprocess trusts the Phlox host and is not an isolation boundary; it is for trusted single-user use only. Shared deployments use an ephemeral Podman/Docker container with resource/network controls or off-host AWS Bedrock AgentCore execution. Configured isolation fails closed rather than silently running locally.

🛡️

Per-tool permissions

Each tool is auto, ask, or deny. Mutating and execution tools default to ask; Agent mode is off by default and auto-approves only when explicitly enabled for a turn.

⚙️

Live admin config

Edit provider profiles, model pricing, resilience, and sandbox limits from an admin panel — applied without a restart. API keys are write-only and masked.

📊

Observability

Per-request structured logs, an optional OpenTelemetry tracing seam, and per-turn token/cost capture in a durable ledger.

💵

Departmental chargeback

A durable usage ledger outlives the accounts it tracks — a departed user's costs stay billable after their account is deleted. Usage by month × user × department × model, CSV-exportable.

🧮

Spend budgets

Cap monthly spend per user or department. Users see a warning as they near the limit; priced models are blocked once it's reached — enforced for both chat and the API gateway, with a monthly reset.

🧱

Hardened harness

Sub-agents inherit the turn's approval state instead of bypassing it, and web_fetch refuses private, loopback, and cloud-metadata addresses by default — the agent can't be steered into your internal network.

🗄️

SQLite or PostgreSQL

SQLite by default for a zero-setup start; point DATABASE_URL at PostgreSQL for production deployments. Deploy bare, under systemd, or as a Docker/Podman container.

Guardrails

Configurable controls for provider-bound text

Guardrails are optional and disabled by default. When enabled, an administrator can apply text/regex-based rules to provider-bound messages, replayed history, and tool results, and to model output streaming back. Built-in patterns cover emails, phone numbers, SSNs, credit cards, and API keys; custom regular expressions can cover organization-specific identifiers. The original user transcript may still be stored by Phlox.

• Redact or block, set per direction — filter what reaches the provider, what comes back to users, or both
• Streaming-safe — a holdback redactor means a match can't slip through split across streamed chunks
• Per-pattern actions: redact PII globally while hard-blocking project codenames or record numbers
• Enforced identically in interactive chat and the OpenAI-compatible /v1 gateway
• Built-in test-pattern preview — try a policy against sample text before you save it
• Not a DLP guarantee — pattern matching can miss sensitive data; review provider retention, logging, and deployment controls separately

Phlox does not currently support PHI or regulated-data deployments. Those uses remain out of scope until the documented governance, managed-secrets, audit, and independent-review gates are complete.

The platform layer

An LLM gateway and cost ledger for the whole team

Beyond chat, Phlox is an OpenAI-compatible gateway with per-user API keys, live model pricing, department-level chargeback, and monthly spend budgets — the governance layer that turns a chat app into shared infrastructure.

Spend budgets

Set a cap. Warn, then enforce.

Give a user or a whole department a monthly cost ceiling. Phlox warns as they approach it and blocks priced models once it's reached — while free, locally-hosted models keep working. Enforcement is shared by chat and the API gateway, and budgets reset each month. Most-restrictive budget wins when both a user and department cap apply.

Under the hood

Two clean processes

A FastAPI backend handles LLM orchestration, the agent harness, MCP, RAG, code execution, auth, and persistence — SQLite by default, PostgreSQL for production. A React + Vite frontend renders the rich, streaming UI.

Make it yours

Eight themes, instant switching

A semantic CSS-variable token layer means themes change with no rebuild — and adding your own is two small edits.

Phlox Dark

Phlox Light

Fred Hutch

Hutch Night

Dark

Light

Sandstone

Terminal

+ your own

Up and running in minutes

Quick start

One command builds and starts everything — it checks your prerequisites, installs dependencies on first run, seeds backend/config.yml, and opens your browser.