vLLM Agentic API — Your agent harness. Now on vLLM.

3 min read Original article ↗
BUILT FOR AGENTIC INFERENCE

vLLM Agentic API is the agentic server that lets you run Codex and Claude Code on top of vLLM.

Everything you need for agentic inference: conversation state, built-in tools, and multi-turn execution with the open models you choose.

Get started View on GitHub

APACHE 2.0BUILT IN RUSTPOWERED BY vLLM

THE AGENTIC STACK01 — 03

Codex

Claude Code

Your app

HTTP · SSE · WebSockets

Agentic APITHE APPLICATION LAYER

RUST

State Tools Execution

Reason → call tools → continue↻

MODEL INFERENCE

vLLMINFERENCE ENGINE

Open models. Your GPUs.

(SGLang and Dynamo also supported)

You own every layer.

One application layer.Conversations+Built-in tools+Multi-turn execution+WebSockets

BUILT FOR THE AGENT LOOP

Inference is the start.
Keep the agent going.

Agentic inference brings model calls, built-in tool execution, and conversation state together. vLLM serves the model; Agentic API coordinates the loop.

A conversation that carries forward.

Keep item history and response state on the server. Continue with previous_response_id, without rebuilding the transcript on every call.

response_01→response_02→response_03

Tools, with the server in the loop.

Execute built-in web search and tools from Model Context Protocol (MCP) servers on the gateway. Tool call outputs flow back to the model so it can take the next step.

web_search MCP toolsGATEWAY-EXECUTED

One request.
Multiple inference rounds.

Let the gateway coordinate model calls and built-in tools until the response is ready or the client needs to act.

REASON → ACT → CONTINUE

Stay connected.
Keep streaming.

Use HTTP for requests, server-sent events (SSE) for streamed responses, and WebSockets for interactive Responses API clients.

HTTPSSEWS

Open models.
Your choice.

Connect to compatible open models served by vLLM. Choose the model and tool-calling configuration for your workload.

EXPLORE SETUP

Clear execution boundaries. Codex and Claude Code still execute their own shell and editor tools. Agentic API runs the tools assigned to the gateway.

ACROSS THE OPEN INFERENCE STACK

Bring your serving stack.

Built for vLLM, with recorded integration tests for SGLang and NVIDIA Dynamo covering streaming, multi-turn state, and client-executed function calls.

FROM MODEL TO AGENT

Same workflow.
Your infrastructure.

Point your coding agent at Agentic API. Keep your familiar workflow, with an open model behind it.

  1. 01

    Serve your model

    Run a tool-capable model with vLLM. The example uses an upstream on port 5050.

  2. 02

    Get Agentic API

    Run v0.9.0 from PyPI with uvx or install from crates.io, with source builds available for development.

  3. 03

    Launch your client

    Install Codex or Claude Code, then choose your client and served model.

Read the full setup guide

Install from crates.io (v0.9.0)

cargo install agentic-server --version 0.9.0 --locked

Install the released gateway and agentic CLI. Requires Rust and Cargo.

Run either command to start Agentic API and launch the harness:

Launch Codex CLI

agentic run codex \
  --upstream http://127.0.0.1:5050 \
  --model Qwen/Qwen3-30B-A3B-FP8

Launch Claude Code

agentic run claude \
  --upstream http://127.0.0.1:5050 \
  --model Qwen/Qwen3-30B-A3B-FP8

For the desktop app, see the Codex Desktop setup guide.

Use the model ID served by your vLLM instance.