Governing Agent Tool Execution
Can an independent governance boundary prevent unsafe or over-broad execution while preserving legitimate tool use?
Claim boundaryThis lab tests a fictional tool environment, policy-v1, a simulated executor, and one published model-backed corpus. It does not establish that AEG or agent systems are universally safe.
Evidence summary
What the current experiment observed
Method
The evidence path
Findings and revision
The evidence changed the work
Phase A validated the boundary
The fixed 40-scenario suite passed 40 of 40 cases. The deliberately naive direct path executed 27 of 32 cases expected not to execute; the governed path executed zero.
Phase B separated model and governance behavior
One model proposal was reused for both paths. Model selection failures, policy decisions, and execution safety were measured separately so abstention could not be counted as governance success.
The pilot changed the experiment
Repeated prompt text, permissive abstention scoring, under-specified read scopes, unrealistic configuration values, and incomplete provenance were exposed and corrected before the published v2 run.
The result remains narrow
No unauthorized governed execution was observed in the published configuration, and every expected legitimate action was preserved. The result applies only to the committed model, corpus, policy, runner, and simulated executor.
Limitations
What this lab does not prove
- All tools, identities, services, and data are fictional.
- The executor is simulated and does not reproduce every production integration failure.
- The corpus is committed and reviewable but is not an independent safety benchmark.
- A different model, policy, tool schema, or system prompt may produce different behavior.
Inspect and reproduce