Skip to main content

Configure AI risk scoring

AI Risk Scoring is Willow's scoring system for AI assessments of MCP tools, skills, and discovered MCP servers. It combines grades for risk dimensions using configurable weights, peaks, and score rules.

Open Admin > Settings > AI Risk Scoring to change this configuration. You can give different priorities to MCP tools, skills, and discovered MCP servers.

For the distinction between an AI assessment and an MCP server's authentication, tools, and guards posture, see How Willow Assesses Risk.

Choose what to assess​

Select MCP tools, Skills, or Discovered MCP servers. Each profile has its own dimension weights, peaks, and guidance. Select the profile for the items you want to change; for example, use Skills when reviewing the risk of instructions people install in their coding agents.

Adjust a dimension​

Each profile covers data exfiltration, code execution, credential access, privilege and scope, destructive actions, supply chain, prompt injection and safety bypass, deception and mismatch, and provenance.

For a dimension you want to change:

  1. Adjust Weight to change its contribution to the weighted mean. Set it to 0 to drop the dimension: the AI stops grading it, and it no longer contributes to the score.
  2. Adjust Peak to change how strongly that dimension contributes to the worst-dimension part of the score.
  3. Add Extra guidance for the AI to explain what matters for that dimension in your organization.

Use Organization guidance below the dimensions for instructions that apply across the selected profile. For example, describe which destinations your organization treats as internal rather than repeating that information in every dimension.

Adjust the score rules​

The controls above the profiles affect how grades become a final score:

  • Increase Worst dimension share (W) to give the most severe dimension more influence; decrease it to give the weighted mean more influence. The default is 0.75.
  • Set Stability band (±) to control when a reassessment keeps the previous score. A new result inside the band preserves that score. The default is 0.75.
  • Set Severe dimension floor for the minimum score when an attack dimension with a peak of at least 0.9 is severe. The default is 8.
  • Set Two severe floor for the minimum score when at least two dimensions are severe. The default is 9.
  • Set Benign cap for the maximum score when no dimension is above low. The default is 4.

The AI grades each dimension from 0 to 3. Willow calculates the number from those grades, applies static checks and score rules, and then applies the stability band. See How AI grades become a score for the calculation.

Save and check an assessment​

  1. Select Save to apply your changes. Discard drops unsaved changes; Reset to defaults returns the controls to their defaults.
  2. Open an item of the type you changed and run its risk assessment again. For example, open a skill and select Retry AI analysis or Run AI analysis on Risk & guards. See Skills.
  3. Review the score and explanation. If the score stays the same, compare the new result with the Stability band (±) before changing the weights again.

On skills, AI analysis failed means the AI step failed even though guards ran. AI risk not assessed means there is no AI score yet.