Skip to main content

Kubernetes

Kubernetes (K8s) is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. It provides powerful abstractions for managing compute, networking, and storage resources across clusters of machines.

Authentication Types​

Kubernetes supports 1 authentication method:

  • Bearer Token (API Key) - Primary authentication method
    • Pros: Simple to set up, works with all Kubernetes clusters, uses service account tokens
    • Cons: Requires manual token management, tokens may need rotation
    • Best for: Most Kubernetes clusters, on-premises deployments, self-managed clusters

Configuration​

Before using the connector, you need to configure:

  • Cluster API URL - The HTTPS endpoint for your Kubernetes API server (e.g., https://my-cluster.example.com:6443 or https://kubernetes.default.svc)
  • CA Certificate - The cluster's Certificate Authority certificate for secure TLS communication. This ensures that your client can verify the identity of the Kubernetes API server and establish a trusted connection, preventing man-in-the-middle attacks.

Setting up Bearer Token Authentication​

The most common and straightforward authentication method is using a Kubernetes service account token. Follow these steps:

Create a Service Account​

Create a dedicated service account for the MCP connector:

kubectl create serviceaccount willow-mcp -n default

Create a ClusterRole Binding​

Creating a Custom Role (read only version)​

This role deliberately leaves out secrets. Granting get on secrets lets an agent read every secret value in the cluster through the Get Secret tool, including other service accounts' tokens.

# willow-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: willow-mcp-role
rules:
# Core API resources
- apiGroups: [""]
resources:
- namespaces
- pods
- pods/log
- services
- configmaps
- events
- nodes
verbs: ["get", "list", "watch"]

# Apps API resources
- apiGroups: ["apps"]
resources:
- deployments
- replicasets
- statefulsets
- daemonsets
verbs: ["get", "list", "watch"]

# Batch API resources
- apiGroups: ["batch"]
resources:
- jobs
- cronjobs
verbs: ["get", "list", "watch"]

# Networking API resources
- apiGroups: ["networking.k8s.io"]
resources:
- ingresses
verbs: ["get", "list", "watch"]

Creating a Custom Binding​

# willow-binding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: willow-mcp-binding
subjects:
- kind: ServiceAccount
name: willow-mcp
namespace: default
roleRef:
kind: ClusterRole
name: willow-mcp-role
apiGroup: rbac.authorization.k8s.io

Creating a Token Secret​

This creates a long-lived token for the service account. It does not expire, so revoke it by deleting the Secret.

# willow-token.yaml
apiVersion: v1
kind: Secret
metadata:
name: willow-mcp-token
namespace: default
annotations:
kubernetes.io/service-account.name: willow-mcp
type: kubernetes.io/service-account-token

Apply:

kubectl apply -f willow-role.yaml
kubectl apply -f willow-binding.yaml
kubectl apply -f willow-token.yaml

Retrieve the Token​

kubectl get secret willow-mcp-token -n default -o jsonpath='{.data.token}' | base64 -d

Find Your Cluster API URL​

If you have kubectl configured, find your cluster URL with:

kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}'

Common patterns:

  • Minikube: https://127.0.0.1:xxxxx (port varies)
  • Kind: https://127.0.0.1:xxxxx
  • GKE: https://xxx.xxx.xxx.xxx
  • EKS: https://xxxxx.yyy.region.eks.amazonaws.com
  • AKS: https://xxx-yyy.hcp.region.azmk8s.io:443
  • On-premises: https://kubernetes.example.com:6443

Get the CA Certificate​

kubectl get secret willow-mcp-token -n default -o jsonpath='{.data.ca\.crt}' | base64 -d

The Cluster API URL must be reachable from your Willow gateway. On Willow SaaS that means a public API endpoint; for a private-only endpoint, use a hybrid or on-premises gateway inside the cluster's network.

Verify the Token​

Before adding it to Willow, check that the token works and is read-only:

TOKEN=$(kubectl get secret willow-mcp-token -n default -o jsonpath='{.data.token}' | base64 -d)
CLUSTER_URL=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
kubectl get secret willow-mcp-token -n default -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt

# Expect 200
curl -s -o /dev/null -w '%{http_code}\n' --cacert ca.crt -H "Authorization: Bearer $TOKEN" "$CLUSTER_URL/api/v1/namespaces"
# Expect 403 with the read-only role
curl -s -o /dev/null -w '%{http_code}\n' --cacert ca.crt -H "Authorization: Bearer $TOKEN" "$CLUSTER_URL/api/v1/namespaces/default/secrets"

Configure the Connector​

  1. In Willow, add a new Kubernetes connector
  2. Select API key as the authentication method
  3. Enter your Cluster API URL (from the step above)
  4. Paste the CA Certificate (from the step above), including the BEGIN CERTIFICATE and END CERTIFICATE lines
  5. Paste the token as the API key
  6. Test the connection

With the read-only role, the write tools (Create Namespace, Create Pod, Delete Pod, Create, Update, Scale and Delete Deployment, Create Service) and Get Secret return 403 Forbidden. Disable them on the integration so agents don't call them.

Available Tools​

The Kubernetes connector provides 29 comprehensive tools organized into categories:

Namespace Management (3 tools)​

  • List Namespaces
  • Get Namespace
  • Create Namespace

Pod Operations (5 tools)​

  • List Pods
  • Get Pod
  • Create Pod
  • Delete Pod
  • Get Pod Logs

Workload Resources (7 tools)​

  • List Deployments
  • Get Deployment
  • Describe Deployment
  • Create Deployment
  • Update Deployment
  • Scale Deployment
  • Delete Deployment

Services & Networking (4 tools)​

  • List Services
  • Get Service
  • Create Service
  • List Ingresses

Configuration & Secrets (4 tools)​

  • List ConfigMaps
  • Get ConfigMap
  • List Secrets
  • Get Secret

Jobs & Batch Processing (3 tools)​

  • List Jobs
  • Get Job
  • List CronJobs

Cluster Insights (3 tools)​

  • List Nodes
  • Get Node
  • List Events

Example AI Prompts​

Here are some example prompts you can use with the Kubernetes connector:

Monitoring & Troubleshooting:

  • "Show me all pods in the production namespace that are not running"
  • "Get the logs for the api-server pod in the backend namespace"
  • "What events have occurred in the last hour in the default namespace?"
  • "List all deployments and their replica counts"

Deployment Management:

  • "Scale the frontend deployment to 5 replicas"
  • "Update the api deployment to use the image myapp:v2.0"
  • "Show me the status of the backend deployment"
  • "Describe the nginx-deployment in the default namespace: which images is it running and how many replicas are ready?"

Configuration & Inspection:

  • "List all services in the production namespace"
  • "What ConfigMaps exist in the default namespace?"
  • "Show me the details of the database-config ConfigMap"
  • "List all nodes in the cluster and their status"

Batch Processing:

  • "Show me all CronJobs and their schedules"
  • "What jobs have completed in the last 24 hours?"
  • "Get the status of the data-processing job"

Troubleshooting​

401 Unauthorized Error​

Cause: The service account token is invalid, expired, or not being sent correctly.

Solution:

  1. Verify the token is correct and that the willow-mcp-token Secret still exists
  2. To issue a new token, delete and re-apply the Secret (kubectl delete secret willow-mcp-token -n default && kubectl apply -f willow-token.yaml), then paste the new token into Willow
  3. Check that the token was copied completely without extra spaces
  4. Ensure the Authorization header is being set correctly

403 Forbidden Error​

Cause: The service account doesn't have sufficient RBAC permissions for the requested operation.

Solution:

  1. Check the ClusterRole or Role bindings for your service account:
    kubectl describe clusterrolebinding willow-mcp-binding
  2. Review the permissions granted to the ClusterRole
  3. If needed, update the ClusterRole to include missing permissions
  4. For debugging, temporarily grant cluster-admin to verify it's a permissions issue

Connection Refused or Timeout​

Cause: The Cluster API URL is incorrect or not accessible from your network.

Solution:

  1. Verify the Cluster API URL is correct
  2. Check network connectivity: curl -k https://your-cluster-url:6443
  3. Ensure firewall rules allow access to the API server
  4. For cloud providers, verify the API server endpoint is publicly accessible or you're on the right VPN/network
  5. Check if the cluster is using a private endpoint that requires VPN access

Resource Not Found (404)​

Cause: The requested resource (pod, deployment, etc.) doesn't exist or is in a different namespace.

Solution:

  1. Verify the resource name is correct
  2. Check if the resource is in the expected namespace
  3. List resources to confirm they exist: kubectl get pods -A
  4. For namespace-scoped resources, ensure you're specifying the correct namespace

Empty or Unexpected Results​

Cause: Label selectors or field selectors are too restrictive, or resources don't exist.

Solution:

  1. Try listing without filters first
  2. Verify label selector syntax: key=value,key2=value2
  3. Check if resources have the expected labels: kubectl describe pod <name>
  4. For events, they may have been garbage collected (events have a short TTL)

Security Best Practices​

  1. Use Custom Roles: Avoid using cluster-admin in production. Create custom ClusterRoles with minimal permissions.

  2. Token Rotation: Regularly rotate service account tokens, especially in production environments. Deleting the token Secret revokes it immediately.

  3. Namespace Isolation: Consider creating namespace-specific service accounts with RoleBindings instead of ClusterRoleBindings.

  4. Audit Logging: Enable Kubernetes audit logging to track API calls made by the service account.

  5. Network Policies: Use NetworkPolicies to restrict pod-to-pod communication.

  6. TLS Verification: Always use TLS and verify certificates in production environments.

  7. Read-Only Access: For monitoring use cases, create service accounts with read-only permissions.

Additional Resources​