Willow Tunnels

A Willow Tunnel turns the hybrid deployment inside out. Instead of you publishing run on a hostname that Willow and your MCP clients connect to, your run opens one long-lived, outbound-only WebSocket to Willow and keeps it open. Willow gives the gateway a URL on your existing gateway domain — https://<org>.withwillow.ai/mcp/tunnel/<id> — and every request that arrives there, whether from an MCP client or from Willow itself, is relayed down that connection to your run and answered from inside your network.

There is no third-party service, agent or account involved. The connector is built into run, the tunnel terminates on Willow's own gateway, and the only credential is the same gateway AUTH_SECRET a hybrid install already has.

Use it when any of these is true:

• You cannot, or would rather not, publish a hostname for run, allow-list Willow's NAT IPs on an inbound firewall, or manage a TLS certificate.
• Your cluster has no ingress controller or load balancer (air-gapped-ish environments, dev clusters, kind/k3s on a laptop).
• You want the whole of Phase 1 of the hybrid guide to go away.

Who this is for

Tunnels are a SaaS feature managed from Settings > Gateway Settings > Add Gateway > Willow Tunnel. They are not relevant to a full on-prem deployment, where Willow already runs inside your network.

Components

Component	Where it runs	Owned by	What it does
Tunnel gateway	Willow	Willow	An External Run Service gateway with a Tunnel badge. Willow mints its id, its URL (https://<org host>/mcp/tunnel/<id>) and its AUTH_SECRET; you never type a URL. It appears in gateway pickers, OAuth discovery and MCP client configs exactly like any other gateway.
Tunnel endpoint	Willow's hosted run	Willow	Terminates the WebSocket at wss://<org host>/mcp/tunnel/_connect, authenticates the connector by AUTH_SECRET, and proxies HTTP requests for /mcp/tunnel/<id>/* over it. Scales horizontally; a request landing on a replica that does not hold your connection is forwarded to the one that does.
Connector	Inside your network, inside the run process	You run it; Willow issues its secret	Starts automatically when run boots with a tunnel-shaped BASE_URL. Dials out over HTTPS (port 443), authenticates with AUTH_SECRET, and serves each relayed request against its own local HTTP listener. Reconnects with backoff if the connection drops. Never listens on an external port.
Credential	Your secret store	Shared	The gateway AUTH_SECRET. It is what the connector presents to Willow, what Willow presents to your run for service-to-service calls, and what run uses to call the SaaS db-service. One secret, shown once.

Connection model

┌──────────────────────────────────────────────┐
│  Willow SaaS                                 │
│                                              │
│  MCP clients ──▶ <org>.withwillow.ai         │
│  Willow app  ──▶   /mcp/tunnel/<id>/…  ──┐   │
│  db-service  ──▶                         │   │
│                                          ▼   │
│                              hosted run: tunnel endpoint
│                              (holds the WebSocket)
└──────────────────────────────┬───────────────┘
                               │  wss://<org>.withwillow.ai/mcp/tunnel/_connect
                               │  opened BY your run, outbound only,
                               │  authenticated with AUTH_SECRET
                               │
┌──────────────────────────────┴───────────────┐
│  Your network                                │
│                                              │
│  run (hybrid) ── serves relayed requests     │
│                  against 127.0.0.1:PORT      │
│       │                                      │
│       └──▶ db-service via connect            │
│            (ordinary outbound HTTPS,         │
│             unchanged from hybrid)           │
└──────────────────────────────────────────────┘

Three things to notice:

1. Every connection is outbound from your side. The tunnel socket, and run's own calls to the SaaS db-service, both leave your network over 443. Nothing is accepted from the internet.
2. MCP clients use the same URL Willow does. There is no separate "internal" hostname. Your gateway URL is https://<org>.withwillow.ai/mcp/tunnel/<id>, and that is what Cursor, Claude, and the Willow dashboard all talk to. Willow terminates TLS on its own certificate; the hop into your network is the authenticated WebSocket.
3. Your run still does its own authentication. Willow relays requests as-is (with X-Forwarded-* headers set). MCP OAuth, service-to-service AUTH_SECRET checks, guards and policies all run in your cluster exactly as they do behind an ingress. The tunnel is transport, not trust.

What happens on a request

1. A client calls https://<org>.withwillow.ai/mcp/tunnel/<id>/<path>.
2. Willow's gateway host routes /mcp/* to the hosted run, which recognises the /tunnel/<id> prefix and looks up a live connection for that gateway id — locally first, then across replicas.
3. The request head and body are streamed over the WebSocket as frames. If no connector is attached, the client gets 503 tunnel_not_connected immediately; nothing is queued.
4. Your run reconstructs the request against its own listener (127.0.0.1:PORT), with the /mcp/tunnel/<id> prefix stripped so it sees the same paths it would behind an ingress at BASE_URL.
5. The response streams back the same way, including long-lived SSE responses. If the socket drops mid-response, the client receives 502 tunnel_disconnected rather than a hang.

Setup

The flow mirrors hybrid Phases 2–4 with Phase 1 removed. Budget about fifteen minutes.

Step 1 — Create the tunnel gateway

Owner: Willow org admin (needs org:edit) · Produces: the gateway URL and the AUTH_SECRET

1. Open the admin app, Settings > Gateway Settings.
2. Click Add Gateway and choose Willow Tunnel. There is no URL field; Willow generates one on your organization's gateway domain.
3. Click Add Gateway. A Gateway Auth Secret dialog appears with a one-time secret and a Next steps box listing the two values your run needs: BASE_URL (the generated tunnel URL) and AUTH_SECRET. Copy both now — the secret is not shown again.
4. The new card shows a Tunnel badge and a health badge reading Connector not attached. That is expected until Step 3.

If the dialog is gone and you lost the secret, use Rotate Secret on the card to mint a new one. The URL is always visible on the card's Run: line.

Step 2 — Create values.yaml and the secret

Owner: Platform engineer · Needs: Step 1 · Produces: the config Helm installs

Start from the hybrid guide's Step 6 values and make three changes: turn the run ingress off, set BASE_URL to the tunnel URL, and drop the TLS block.

Replace the placeholders:

• <ORG_SLUG> — your organization slug
• <TUNNEL_URL> — the BASE_URL value from the Step 1 dialog, e.g. https://<ORG_SLUG>.withwillow.ai/mcp/tunnel/gw_tun_…

global:
  domain:
    host: "unused.invalid" # nothing is published; the chart still wants a value
  org: "<ORG_SLUG>"
  OPENAI_API_KEY: ""

deployments:
  app:
    enabled: false
  connect:
    enabled: false
  db-service:
    enabled: false
  run:
    enabled: true
    ingress:
      enabled: false # no inbound path; run dials out instead
    # The gateway auth secret from Step 1. See the hybrid guide for why it
    # must not live in values.yaml.
    secretName: "willow-secrets"
    env:
      PORT: "3000"
      LOG_LEVEL: "info"
      # The tunnel URL from Step 1. A tunnel-shaped BASE_URL is what tells run
      # to open the tunnel on start — no other flag is needed.
      BASE_URL: "<TUNNEL_URL>"
      DB_SERVICE_URL: "https://<ORG_SLUG>.withwillow.ai/api/on-prem-db-service"
      CONNECT_URL: "https://<ORG_SLUG>.withwillow.ai"

Copy your organization's Connect host

As in the hybrid guide, <ORG_SLUG>.withwillow.ai is an example. Use the host shown on your gateway card's Connect: line (it may be eu.withwillow.ai or a custom domain) for DB_SERVICE_URL, CONNECT_URL and the host inside <TUNNEL_URL> — they are the same host.

Create the namespace and the secret. Keep the single quotes:

kubectl create namespace <namespace> --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret generic willow-secrets \
  --namespace <namespace> \
  --from-literal=AUTH_SECRET='<gateway-secret-from-step-1>'

Verify the rendered BASE_URL is your tunnel URL and no Ingress is produced, before installing anything:

helm template willow willow/webrix-helm -f values.yaml \
  | grep -E "BASE_URL|CONNECT_URL|DB_SERVICE_URL|kind: Ingress"

The chart normally derives BASE_URL from the ingress host. If the output still shows https://willow.unused.invalid, your chart version does not let env.BASE_URL override the derived value; upgrade the chart, or set the value through extraEnv/a secret, whichever your version supports — the goal is simply that the run container starts with BASE_URL=<TUNNEL_URL>.

Optional variables

Variable	Purpose
TUNNEL_ENABLED	true forces the connector on, false forces it off. Unset, it is inferred from BASE_URL: a URL whose path is /mcp/tunnel/<id> turns it on.
TUNNEL_URL	Override the WebSocket endpoint. Default: wss://<BASE_URL host>/mcp/tunnel/_connect. Only needed if you front Willow's domain with something of your own.
HTTPS_PROXY / NO_PROXY	Honoured for the tunnel connection as for every other outbound call run makes.

Step 3 — Install the chart

Identical to the hybrid guide's Step 7:

helm repo add willow https://charts.withwillow.ai && helm repo update
helm upgrade --install willow willow/webrix-helm \
  --namespace <namespace> \
  -f values.yaml \
  --wait --timeout 5m

Within a few seconds of the pod becoming ready the log shows the connector attach:

[tunnel] connector starting; dialing Willow
[tunnel] connected to Willow; this run is now reachable at the tunnel URL   gatewayId=gw_tun_… publicUrl=https://<ORG_SLUG>.withwillow.ai/mcp/tunnel/gw_tun_…

Step 4 — Verify end to end

1. Gateway card. Back in Gateway Settings, click the re-check arrow on the tunnel card. It should read Healthy. The check asks Willow's tunnel endpoint whether a connector is attached, then calls /healthz and /healthz/validate through the tunnel, so a green badge proves the full path including AUTH_SECRET.

2. OAuth discovery from outside. From any machine:

   curl -sS https://<ORG_SLUG>.withwillow.ai/mcp/tunnel/<id>/.well-known/oauth-protected-resource

   The resource field should be your tunnel URL (possibly with a trailing slash — run normalises it) and authorization_servers should name your Connect host.

3. An MCP client. Add https://<ORG_SLUG>.withwillow.ai/mcp/tunnel/<id> to Cursor or Claude, or pick the gateway in the Willow dashboard's connection instructions, and sign in. Tool calls now execute inside your cluster.

If the gateway is your only one, Willow made it the Default Gateway when you created it, so dashboard features that call run (tool tests, guard evaluation, discovery) already go through the tunnel.

Security model

• Zero inbound. Nothing in your network listens for Willow or for MCP clients. Remove the tunnel gateway in Willow, or stop run, and the path is gone.
• One secret, mutual. The connector proves itself to Willow with AUTH_SECRET; Willow proves itself to run on service calls with the same secret; MCP clients prove themselves to run with OAuth. Rotate it from the card like any gateway secret and roll the Kubernetes secret; the connector reconnects with the new value on restart.
• Scoped per gateway. A secret only attaches to its own gateway id. Willow also checks that the host a connector dials matches the organization that owns the gateway, so a secret cannot be replayed against another tenant's hostname.
• Transport, not trust. Willow does not decrypt, cache or inspect request bodies beyond framing them; it forwards headers with hop-by-hop ones removed and X-Forwarded-For/Proto/Host set. Policies, guards, audit and OAuth all execute on your run.
• No tunnel-specific allow-listing. Because clients and Willow both arrive at your gateway domain, the hybrid guide's NAT IP table and firewall steps do not apply. The only egress your run needs is HTTPS to your Connect host.
• Backpressure and limits. The relay streams with per-connection flow control; a slow client cannot buffer unbounded data on either side. Response heads must arrive within the same window Willow applies to any gateway (≈5 min), and bodies are chunked at 64 KB.

Switching an existing hybrid install to a tunnel

You can run both side by side. Add a Willow Tunnel gateway, deploy a second run release (or point the existing one at the tunnel BASE_URL with its ingress disabled), confirm the tunnel card is Healthy, then Set as Default on the tunnel card and remove the old External Run Service gateway and its ingress/DNS/certificate at your leisure. MCP clients pinned to the old URL will need their config updated; the dashboard's connection instructions follow the default gateway automatically.

Troubleshooting

Symptom	Likely cause	Fix
Card reads Connector not attached; run log shows nothing about the tunnel	BASE_URL is not the tunnel URL, or TUNNEL_ENABLED=false is set	kubectl exec into the pod and env | grep -E 'BASE_URL|TUNNEL'. Fix values.yaml, re-run Helm.
run log: Willow rejected the connector's AUTH_SECRET (HTTP 401)	AUTH_SECRET does not match any tunnel gateway — wrong value, rotated since, or the gateway was deleted and recreated	Compare the secret with the Step 1 value; Rotate Secret if unsure and roll willow-secrets.
run log: Willow rejected the connector's AUTH_SECRET (HTTP 403)	The secret belongs to a plain External Run Service gateway, or the host in BASE_URL is another organization's	Use the secret from a Willow Tunnel card, and make sure the tunnel URL host is your own Connect host.
run log: connection errors with ECONNREFUSED/ENOTFOUND/proxy messages	Egress to the Connect host on 443 is blocked, or a corporate proxy is required	Allow HTTPS egress to the Connect host; set HTTPS_PROXY/NO_PROXY on the run deployment.
Connector attaches, card says reachable but Auth issue	The pod has a different AUTH_SECRET than Willow stored (typically a stale Kubernetes secret after a rotate)	Update willow-secrets, restart the deployment.
Clients get 503 tunnel_not_connected intermittently	Connector flapping — pod restarts, or egress idle-timeouts killing the socket	Check pod restarts and events; the connector pings every 30 s, so raise any egress idle timeout below ~60 s.
Clients get 502 tunnel_disconnected mid-stream	The pod restarted or was rescheduled while a response was open	Expected during rollouts; run more than one run replica, each attaches its own connection and Willow uses whichever is up.
OAuth resource mismatch in an MCP client	BASE_URL differs from the URL the client used (trailing path, wrong host)	BASE_URL must be exactly the card's Run: value, with no suffix.
Works from the dashboard, fails from a client using a custom domain	The client is using a different gateway host than the tunnel was minted on	Tunnel URLs are bound to the host they were created with. Create the tunnel after setting up the custom-domain gateway, or have clients use the host shown on the card.

Limits

• One tunnel gateway serves one hybrid run deployment (any number of replicas). To tunnel a second, independent deployment, create a second tunnel gateway.
• The tunnel carries HTTP (including SSE). It does not carry raw TCP, so it cannot expose a database or an arbitrary internal MCP server on its own — point those at your run as you would in any hybrid install, and run reaches them from inside your network.
• Request bodies and responses are streamed, but a single request still has to fit the same timeouts as any gateway call. Very long-running operations should use the MCP streaming patterns they already use behind an ingress.