Skip to main content

Set up and check agent telemetry

To collect coding-agent activity in Runtime, enable collection in Willow and configure the agent's exporter or hooks on each device. You need admin access to the organization's collection settings and a device running the coding agent.

1. Open the collection controls​

  1. Open Runtime in Willow Admin. For another deployment, use /monitor/runtime on your admin app's hostname.
  2. Select Telemetry settings. On an empty page, Set up agent telemetry opens the same destination.
  3. In AI Discovery Settings, confirm that Agent telemetry is selected.
  4. Turn on the coding agents you want to collect: Claude Code, Codex, Cursor, or GitHub Copilot.
  5. Set Anonymize users according to whether the collected records should identify the reporting person.

The Agent telemetry section of AI Discovery Settings with controls for Claude Code, Codex, Cursor, GitHub Copilot, and Anonymize users

You can also open Admin > Settings, expand Log Settings, and select Edit on Collect Agent Telemetry (OpenTelemetry). Choose the agents in that dialog and select Save. Then configure the reporting device as described below.

2. Configure the reporting device​

Choose the path that applies to the device:

Device setupAction
Willow Scan Agent is installed and connected to this organizationLet it collect the updated settings on its next scan. It configures the selected installed coding agents. Check the device's recent scan in Discover > Shadow AI > Devices.
You want the Scan Agent to manage the configurationInstall it on a test device or deploy it through MDM, using this organization's generated connection details.
You manage a native exporter yourselfFollow the agent's instructions in Analytics Data Sources, using the endpoint and authorization values shown for your organization.

For Cursor, use the Scan Agent's hook-based collection. If Cursor collection is unavailable for your organization, contact Willow Support. If an agent already exports to another collector, use your existing collector or OTLP proxy to forward its events to Willow.

Restart the coding agent after its exporter or hook configuration changes.

3. Generate a recognizable event​

  1. Start a new session in the configured coding agent.
  2. Ask it: Reply with Runtime check complete.
  3. Note the time, the coding agent, and the reporting user or device.
  4. If you are checking MCP coverage too, make one read-only call through an MCP server already approved for that agent and record its tool name.

Use the timestamp and source to find the event. Log Content controls prompt text, and Anonymize users controls user attribution.

4. Find the activity in Runtime​

  1. Return to Runtime and select Last 24 hours.
  2. Leave All Users selected for the first check, then select Refresh.
  3. Open Sessions, Prompts, or Agent events, according to what the coding agent reports. In Prompts, use Telemetry for the exporter or Guard hooks for a hook evaluation.
  4. Match the time and source to the session you just created. Use Open in Logs where available to inspect the detailed record.
  5. If you made an MCP call, check Tool calls and MCP servers for that tool and its route classification.

Confirm that a new record appears for the session you just created. For an MCP call, check whether the route is Via gateway, Direct, or Unknown.

If the activity is missing​

SymptomCheck
No agent activity in this windowWiden the window to Last 7 days or Last 30 days, remove the user filter, and refresh. If old records appear, repeat the check against the new session's timestamp.
Gateway calls appear, but Overview reports no agent telemetryConfirm the coding agent's collection switch, exporter or hook configuration, and organization connection values. Check that the Scan Agent has completed a recent scan, then restart the coding agent.
Device is absent or its scan is oldFollow the service status and log checks in Scan Agent reference.
Exporter is configured but Willow refuses its eventsConfirm collection is on for that agent. See Analytics Data Sources troubleshooting.
Records appear only with All UsersCheck the reporting identity and Anonymize users. Do not filter to a named user for anonymous records.
Metadata appears without the test promptCheck Log Content, which controls whether records include prompt and response text.
Web AI activity is absentDeploy and configure the Willow Guard browser extension.
On this page