GitHub - gumieri/nenya: A lightweight, highly secure AI API Gateway/Proxy written in Go. Acts as transparent middleware between local AI coding clients (OpenCode/Pi/Cursor) and upstream LLM providers (Gemini, DeepSeek, Zhipu z.ai).

A lightweight, zero-dependency AI API Gateway written in Go. Nenya sits between your AI coding clients and upstream LLM providers, adding secret redaction, context management, agent routing, and MCP tool integration — all with transparent SSE streaming. Security-hardened: non-root execution, mlock for secrets, seccomp + no-new-privileges.

Compatible with any provider that implements the OpenAI Or Anthropic Chat Completions API. For 23 providers we ship built-in adapters with specialized handling.

How Nenya handles the requests

flowchart TD
    CLIENT["Client<br/>Cursor / OpenCode / Aider / etc.<br/>POST /v1/chat/completions · /v1/messages<br/>Bearer token"]

    subgraph GW["Nenya Gateway"]
        direction TB
        AUTH["Auth + RBAC"]
        RESOLVE["Parse body · resolve agent + targets<br/>strategy: fallback · round-robin · sticky"]
        CACHE{"Response cache"}
        MCPINJ["MCP auto-search + tool injection"]
    end

    CHAIN["Interceptor chain (best-effort)<br/>redact → entropy → TF-IDF → bouncer"]
    TRIM["Token budget trim (hard limit)"]

    subgraph LOOP["Dispatch loop — per target"]
        direction TB
        GUARDS["Circuit breaker · rate limits · cost guard"]
        MODES["A standard forward<br/>B MCP multi-turn tool loop<br/>C context-limit retry"]
    end

    UPSTREAM["Upstream LLM providers<br/>23 built-in adapters"]

    subgraph SSE["SSE pipeline"]
        direction TB
        PROBE["Stream-head probe (pre-header)<br/>empty / early-error failover"]
        XFORM["Adapter transforms · format conversion"]
        WATCH["Stall watchdog · stream continuation"]
        ACCT["Usage accounting · cache capture · MCP auto-save"]
    end

    OUT["Client receives transparent SSE"]

    CLIENT --> AUTH --> RESOLVE --> CACHE
    CACHE -- "HIT → replay" --> OUT
    CACHE -- "miss" --> MCPINJ --> CHAIN --> TRIM --> GUARDS --> MODES --> UPSTREAM
    UPSTREAM --> PROBE --> XFORM --> WATCH --> ACCT --> OUT
    PROBE -. "failover → next target" .-> GUARDS

    classDef io fill:#e8ebf0,stroke:#57606a,color:#1f2328
    classDef gw fill:#ddf4ff,stroke:#0969da,color:#1f2328
    classDef pipe fill:#fff8c5,stroke:#9a6700,color:#1f2328
    classDef sse fill:#dafbe1,stroke:#1a7f37,color:#1f2328

    class CLIENT,OUT io
    class GW,LOOP gw
    class CHAIN,TRIM pipe
    class SSE sse

Flow notes:

• /v1/* endpoints require client bearer auth; /healthz, /statsz, /metrics do not.
• Pipeline failures degrade gracefully and forward the request instead of returning a 500.
• MCP-enabled agents can run local/remote tools without exposing MCP complexity to the client.
• Sticky strategy pins a session (agent + system prompt + first user message) to one provider/model, keeping provider-side prefix caches warm across turns.
• Upstream streams are probed before headers commit: empty streams or early SSE errors fail over to the next target, and interrupted streams are auto-resumed via stream continuation.

Features

Routing & Agents

• Config-driven provider registry — add providers via JSON, zero code changes
• 23 built-in providers with specialized adapters for wire format differences
• Dynamic model discovery — fetches live model catalogs from providers at startup and on reload
• Model registry — reference models by string shorthand with automatic provider/context resolution
• Multi-provider model resolution — when a model exists in multiple providers, all are added to the agent's fallback chain
• Three-tier model resolution — config overrides > discovered models > static registry
• Per-model wire format — models from multi-format gateways (OpenCode Zen) auto-convert between OpenAI, Anthropic, and Gemini wire formats based on the model's format attribute
• Agent fallback chains — round-robin or sequential with circuit breaker and automatic failover
• Latency-aware routing — auto-reorder targets by historical median response time with ±5% jitter to prevent thundering herd
• Per-agent system prompts — inline or file-based

Security & Privacy

• Tier-0 regex secret filter — always-on redaction of AWS keys, GitHub tokens, passwords, etc.
• 3-Tier content pipeline — pluggable interceptor chain: regex redaction, entropy filtering, TF-IDF relevance scoring, engine summarization
• Context window compaction — sliding window summarization with configurable engine
• Stale tool call pruning — compact old assistant+tool response pairs to save tokens
• Thought pruning — strip reasoning blocks from assistant message history
• Input validation — strict body limits, JSON sanitization, header filtering
• Graceful degradation — never blocks requests due to engine or pipeline failures
• Role-Based Access Control (RBAC) — per-API key roles (admin, user, read-only) with agent and endpoint restrictions
• Secure memory — mlock-protected token storage, read-only sealing, core dump prevention

Hardening (Deployment Security)

• Secure memory (default): All tokens stored in mlock-protected RAM, sealed read-only after init, core dumps disabled
• Non-root execution — runs as UID 65532 with dropped capabilities
• Memory protection — LimitMEMLOCK=infinity and LimitCORE=0 in systemd
• Read-only filesystem — immutable root + private /tmp
• Seccomp + no-new-privileges — restricted syscalls, prevents privilege escalation
• Zero-trust secrets — loaded via systemd credentials or container mounts, never to disk
• Socket activation — seamless restarts with zero dropped connections

Reliability

• Zero external dependencies — Go standard library only
• Hot reload — systemctl reload nenya for zero-downtime config changes
• Circuit breaker — per agent+provider+model with automatic failover, exponential backoff, and semantic error classification
• Rate limiting — per upstream host (RPM/TPM) with per-provider overrides
• Response cache — in-memory LRU with SHA-256 fingerprinting and optional semantic similarity search
• Graceful shutdown — 30s grace period for in-flight requests, MCP client cleanup
• Context-limit auto-retry — upstream context-length errors trigger summarization and retry
• Local engine lifecycle — pre-load and manage local Ollama models with LRU eviction
• Structured errors — all error responses include error_kind field for programmatic diagnostics

MCP Tool Integration

• Tool discovery — connect to MCP servers for automatic tool injection
• Multi-turn execution — intercept tool calls, execute against MCP servers, forward results
• Auto-search — pre-fetch relevant context from MCP servers before forwarding
• Auto-save — persist assistant responses to MCP memory servers

Quick Start

Run with Podman

Create minimal config and secrets:

mkdir -p config secrets
cat > config/config.json << 'EOF'
{
  "server": { "listen_addr": ":8080" },
  "agents": {
    "default": {
      "strategy": "fallback",
      "models": ["gemini-2.5-flash"]
    }
  }
}
EOF

cat > secrets/provider_keys.json << 'EOF'
{
  "provider_keys": {
    "gemini": "AIza..."
  }
}
EOF

cat > secrets/client.json << 'EOF'
{
  "client_token": "nk-$(openssl rand -hex 32)"
}
EOF

Run the container:

podman run -d \
  --name nenya \
  -p 8080:8080 \
  -v ./config:/etc/nenya:ro \
  -v ./secrets:/run/secrets/nenya:ro \
  -e NENYA_SECRETS_DIR=/run/secrets/nenya \
  --cap-drop=ALL \
  --cap-add=IPC_LOCK \
  --security-opt=no-new-privileges:true \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64M \
  ghcr.io/gumieri/nenya:latest

Test it:

curl -H "Authorization: Bearer $(jq -r '.client_token' secrets/client.json)" \
  http://localhost:8080/healthz

Or Install via Package Manager

Nenya provides native packages for major Linux distributions and community package managers:

All packages install the binary to /usr/bin/nenya and include systemd service and socket units. After install, enable and start:

sudo systemctl enable --now nenya.socket
sudo systemctl enable --now nenya.service

Runtime Configuration

Nenya supports standard environment variables for deployment portability:

Example usage:

PORT=9090 HOST=127.0.0.1 ./nenya --config /path/to/config.json

Or in Docker:

docker run -e PORT=9090 -p 9090:9090 ghcr.io/gumieri/nenya:latest

Or Choose Your Deployment

• Deploy Bare Metal (systemd) — Direct binary install, socket activation, hot reload
• Deploy Container (Podman/Docker Compose) — compose.yml, image verification, security hardening
• Deploy Kubernetes (Helm) — Helm chart, ConfigMap/Secret, ingress setup

API Endpoints

All /v1/* endpoints require Authorization: Bearer <client_token> or Bearer <api_key_token>. API keys support RBAC enforcement — agent scoping, endpoint allowlists, role-based permissions (admin bypasses all checks).

See docs/PASSTHROUGH_PROXY.md for detailed passthrough proxy usage.

Documentation

License

Apache 2.0. See LICENSE.