GitHub - glama-ai/lightport: A lightweight AI gateway that makes LLM providers OpenAI-compatible.

A lightweight AI gateway that makes LLM providers OpenAI-compatible.

Goal

Lightport does one thing: it accepts OpenAI-compatible requests, transforms them for the target provider, and returns the response. That's it.

Retries, secret management, caching, rate limiting, and other operational concerns are explicitly non-goals. Those are better handled either at a service layer above Lightport or as custom middleware.

Supported endpoints:

• POST /v1/chat/completions
• POST /v1/completions
• POST /v1/responses (+ GET, DELETE, input_items)

Supported providers: OpenAI, Anthropic, Azure OpenAI, Google Gemini, Vertex AI, Bedrock, Cohere, Mistral, Groq, Deepseek, Together AI, Fireworks, Ollama, and more (77 total).

Background

Lightport started as a fork of Portkey AI Gateway. Our sole use case for the gateway has always been making AI providers OpenAI-compatible – we only needed the request/response transformation layer.

Since then, Portkey has evolved into a full-featured AI gateway with guardrails, fallbacks, automatic retries, load balancing, request timeouts, smart caching, usage analytics, cost management, and more. We believe those capabilities belong at a higher abstraction level – which is what Glama provides – rather than in the gateway itself.

Since forking, we have fixed numerous bugs, added integration tests for every provider, and continue to actively maintain the gateway as it directly powers Glama.

If you need a lightweight proxy that makes LLM providers OpenAI-compatible, Lightport is for you. If you need an enterprise gateway with all the bells and whistles, consider Portkey Gateway.

Quickstart

The gateway runs on http://localhost:8787.

From source

Make a request

curl http://localhost:8787/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-lightport-provider: openai" \
  -H "Authorization: Bearer sk-YOUR-KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Set the provider via x-lightport-provider header and pass credentials via Authorization (or provider-specific headers like x-api-key for Anthropic).

Provider-specific headers

Some providers require additional headers:

HTTP proxy

Route provider requests through an HTTP proxy by setting the x-lightport-proxy-url header:

curl http://localhost:8787/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-lightport-provider: openai" \
  -H "x-lightport-proxy-url: http://user:pass@proxy.internal:8080" \
  -H "Authorization: Bearer sk-YOUR-KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Request timing

Every response carries a Server-Timing header breaking the request into stages, and the same breakdown is logged once per request:

server-timing: total;dur=3020.2, parse;dur=0.2, validate;dur=0.3, transform;dur=0.2,
               socket;dur=3.3, ttfb;dur=3007.5, read;dur=2.2

socket is time spent acquiring an upstream connection (queueing, DNS, TCP, TLS); ttfb is time spent waiting for the provider to answer. A slow gateway and a slow provider are indistinguishable from the outside — these tell them apart. The log line also reports send (writing the response back) and covers requests whose caller hung up before reading any header.

Scripts

pnpm dev           # Development server with hot reload
pnpm build         # Production build
pnpm start:node    # Start production server
pnpm test          # Run tests
pnpm lint          # Lint code
pnpm format        # Format and auto-fix
pnpm knip          # Find unused code/dependencies

Testing with provider credentials

Copy .env.example to .env and fill in API keys for the providers you want to test. Tests automatically load .env and skip providers without credentials.

cp .env.example .env
# fill in your keys
pnpm test

Architecture

Request
  -> bodyParser middleware (parse JSON/FormData)
  -> requestValidator (require provider header)
  -> handler (chatCompletions / completions / modelResponses)
    -> constructConfigFromRequestHeaders()
    -> tryPost()
      -> adapter transform (if needed for responses/messages API)
      -> provider lookup + transformToProviderRequest()
      -> fetch to provider
      -> responseHandler() (transform response back)
    -> Response

The provider system (src/providers/) contains 77 provider implementations. Each defines:

• API config (base URL, endpoints, headers)
• Request parameter transforms
• Response transforms (streaming + non-streaming)