Skip to main content

Connect Clients

Clients reach the LLM Gateway with two values, both on Control > LLM Gateway > Connect:

ValueWhat it is
Gateway URLThe gateway's base URL, for example https://acme.withwillow.ai/llm
API key commandnpx -y @mcp-s/mcp@latest auth token --base-url <your MCP URL>. It prints the user's Willow credential, which the client sends as its API key

Every person gets their own key, so usage, budgets, and logs are attributed to the person who signed in. Nobody needs a provider API key.

How the key works​

The first time the command runs on a machine, it opens the browser and signs the user in to Willow. After that, it prints the credential saved in ~/.willow/token.json, the same file the Willow stdio MCP server uses, so a user who already has the stdio server set up is already signed in. The command needs Node.js on the machine.

Clients that can run a command to get their key, such as Claude Code's apiKeyHelper, should run this command rather than store its output, so the user never handles the credential.

If a client gets 401 Invalid or expired Willow token, the user signs in again:

npx -y @mcp-s/mcp@latest auth login --refresh --base-url <your MCP URL>

auth status shows whether the user is signed in, and auth logout removes the saved credential.

Which URL each API uses​

The gateway serves the APIs natively and does not translate between them:

Client APIBase URL to setServes
Anthropic MessagesThe Gateway URL as isModels on an Anthropic provider
OpenAI Chat Completions or ResponsesThe Gateway URL followed by /v1Models on an OpenAI or OpenAI-compatible provider

Calling a model through the other API is refused with Model … is served by …; call it through ….

Claude Code​

Add the gateway to Claude Code's settings.json, either the user's own or a managed one:

{
  "apiKeyHelper": "npx -y @mcp-s/mcp@latest auth token --base-url https://acme.withwillow.ai/mcp",
  "env": {
    "ANTHROPIC_BASE_URL": "https://acme.withwillow.ai/llm"
  }
}

The Connect tab shows this snippet with your organization's values filled in. Remove ANTHROPIC_API_KEY and ANTHROPIC_AUTH_TOKEN from the environment: Claude Code prefers them over apiKeyHelper, so they would bypass the user's Willow key.

Roll it out with the Scan Agent​

Instead of editing each machine, use Roll out to Claude Code with the Scan Agent at the top of the Connect tab:

  1. Choose an existing Claude Code policy, or New policy: LLM Gateway.
  2. Select Add and deploy, or Create and deploy for a new policy. Willow adds the gateway URL and the key command to the policy and removes ANTHROPIC_API_KEY and ANTHROPIC_AUTH_TOKEN from it. If the policy already has both, the button reads Deploy.
  3. Deployments opens with the policy selected. Choose whether it goes to the whole organization, to groups, or to single users.

Each user signs in once, the first time Claude Code needs a key.

The card only appears when Config Deployment is enabled for your organization and you have the Monitoring & Shadow AI > Manage permission. See also Claude Code Policy.

Codex​

In ~/.codex/config.toml:

model = "gpt-5"
model_provider = "willow"

[model_providers.willow]
name = "Willow LLM Gateway"
base_url = "https://acme.withwillow.ai/llm/v1"
env_key = "WILLOW_TOKEN"
wire_api = "responses"

Then set the key before starting Codex:

export WILLOW_TOKEN="$(npx -y @mcp-s/mcp@latest auth token --base-url https://acme.withwillow.ai/mcp)"

Anthropic and OpenAI SDKs​

Pass the gateway as the base URL and the Willow credential as the API key. In Python:

import subprocess
from anthropic import Anthropic

token = subprocess.check_output(
    ["npx", "-y", "@mcp-s/mcp@latest", "auth", "token", "--base-url", "https://acme.withwillow.ai/mcp"],
    text=True,
).strip()
client = Anthropic(base_url="https://acme.withwillow.ai/llm", api_key=token)

reply = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}],
)

For the OpenAI SDK, use OpenAI(base_url="https://acme.withwillow.ai/llm/v1", api_key=token) the same way.

The model is an alias or a provider model id from your Models tab.

Any HTTP client​

The gateway accepts the key as Authorization: Bearer … or x-api-key. List the models the caller may use:

curl https://acme.withwillow.ai/llm/v1/models \
  -H "Authorization: Bearer $(npx -y @mcp-s/mcp@latest auth token --base-url https://acme.withwillow.ai/mcp)"

The list contains only the models the caller's groups allow. Send an anthropic-version header to get it in Anthropic's shape, with Anthropic models only.

Endpoints​

All paths are under the Gateway URL:

EndpointAPI
POST /v1/messagesAnthropic Messages, streamed or not
POST /v1/messages/count_tokensAnthropic token counting. Forwarded with your key; no budgets or guards
POST /v1/chat/completionsOpenAI Chat Completions, streamed or not
POST /v1/responsesOpenAI Responses, used by Codex, streamed or not
GET /v1/modelsThe models the caller may use

Name your client in the logs​

The gateway labels each call with the client that made it: claude-code, codex, or cursor from the user agent, and api for anything else. The label appears in logs and is what a guard's AI agent filter matches. Your own apps can set their label with an x-willow-client header of up to 40 letters, digits, ., _, or -.

On this page