Monitoring & Incidents
Whether your AI activity reaches your SIEM (security information and event management system), how long you keep the logs, and how quickly your team reviews alerts.
Alerts not acknowledged in time
Alerts that Willow raised more than 72 hours ago that nobody has acknowledged and that are still open. Target: 0.
Why it matters. If nobody acknowledges an alert, you cannot tell it apart from an alert that never arrived. A growing number of old alerts is also the clearest sign that your team is receiving more alerts than it can handle.
How to improve it.
- Work the backlog, then route new alerts to a channel someone owns.
- If the volume is too high, reduce the alerts from the noisiest event types, instead of letting old alerts build up.
See Alerts and Realtime notifications.
Framework mapping: NIST CSF DE.AE, RS.MA · Practitioner
AI activity not exported to a SIEM
Counts 1 if no log export destination is configured, and 0 if at least one is. Target: 0.
Why it matters. Your SOC (security operations centre) already has detection rules and investigation procedures built around your SIEM. None of them apply to AI tool calls that never reach it.
How to improve it.
- Add a log export destination for your SIEM and send a test event.
- Check that your SIEM can read the fields in the events, not just that it received them.
See Log Settings and Log delivery.
Framework mapping: ISO 42001 A.6.2.8 Event logging · NIST AI RMF MEASURE 2.4 · SANS Monitoring
Prompt-injection detection coverage
Percentage of active integrations covered by a guard that includes a prompt-injection check. Target: 100%.
Why it matters. Data-pattern guards catch secrets leaving. They do not catch instructions arriving. To catch indirect prompt injection, you need a check that looks at what the content is trying to make the model do. Attach that check to every integration that pulls in content from outside your organization.
How to improve it.
- Add a guard with a prompt-injection check and attach it to all integrations.
- Prioritize integrations that pull third-party content — ticketing, email, web fetch, code review.
See Prompt guard.
Framework mapping: OWASP LLM LLM01 Prompt Injection · MITRE ATLAS AML.T0054 LLM Prompt Injection
Audit log retention
Configured audit log retention in days. Target: 180 days.
Why it matters. The EU AI Act requires organizations that deploy AI systems to keep logs for at least six months. Most AI incidents are discovered a long time after they happen. If you keep logs for less time than it takes you to detect an incident, the evidence is already deleted when you start to investigate.
How to improve it.
- Set retention to at least 180 days.
- If cost is the constraint, keep full logs in your SIEM and a shorter window here.
See Log Settings.
Framework mapping: EU AI Act Art. 26(6) Log keeping · ISO 42001 A.6.2.8
Admin action logging disabled
Counts 1 if Willow is not recording administrative changes, and 0 if it is. Target: 0.
Why it matters. Without admin logs you cannot answer the first question of any investigation: who changed the policy, and when.
How to improve it.
- Enable admin action logging in Settings > Log Settings.
Framework mapping: ISO 42001 A.6.2.8 Event logging · NIST CSF PR.PS-04
Integrations with no recent activity logged
Active integrations with no tool call recorded in the last 30 days. Target: 0.
Why it matters. There are two possible explanations, and both are problems. Either nobody uses the integration, in which case it is permanent access that nobody needs. Or people use it outside the gateway, in which case Willow cannot see what they are doing.
How to improve it.
- Disable the integrations nobody uses.
- For the ones people insist they use, check whether their traffic is bypassing the gateway.
Framework mapping: NIST AI RMF MEASURE 2.4 · Practitioner
Critical AI incidents, last 30 days
Alerts raised at critical severity in the last 30 days. Target: 0.
Why it matters. Your incident rate shows whether everything else on this page is working. Do not aim for zero. If your incident count is zero, detection is usually switched off or misconfigured. Track the trend over time, and how quickly your team acknowledges each incident.
How to improve it.
- Classify each incident against OWASP LLM and MITRE ATLAS so repeat patterns become visible.
- Check whether any incident counts as a serious incident under the EU AI Act. If it does, check the deadline for reporting it.
See Alerts.
Framework mapping: NIST CSF RS.MA · NIST AI RMF MANAGE 4.1 · EU AI Act Art. 73 Serious incident reporting