Vulnerabilities
Evaluate captured agent events without hooking the agent
Black Hat Arsenal runtime security for AI coding agents, evaluated offline over recorded action events.
overview
What agentsleak does
agentsleak treats an AI coding agent as something that leaves an audit trail. Every file read, command run, and network call is an action event, and the tool evaluates a folder of those events against pattern rules and sequence rules. A single read of a credentials file may be ordinary; the same read followed by an outbound request is the sequence that matters.
Evaluation happens offline. mode=evaluate reads the event folder you mount and needs no connection to the agent or to a provider, which means you can run it over events captured last week as easily as events captured this morning. mode=policies emits the default policy corpus on its own, and mode=fixture replays a packaged sample for wiring checks.
Because the input is a folder of JSON or JSONL and the output is findings.jsonl, this drops into a scheduled workflow rather than a live hook. The Coding Agent Policy Gate runs it beside golemhalt so a recorded event stream and a deterministic policy corpus produce one correlated report.
use cases
Where agentsleak fits
Evaluate captured agent events
Mount a folder of agent-action JSON or JSONL and evaluate it against pattern and sequence rules without touching the running agent.
Catch the sequence, not just the action
Surface chains such as reading a secret and then making an outbound call, which look harmless one event at a time.
Publish the policy corpus
Run policies mode to emit the default rule set as data, so the policies your reports depend on are inventoried and reviewable.
Gate a coding agent on a schedule
Pair it with golemhalt in the Coding Agent Policy Gate so event evidence and policy coverage correlate into one artifact.
reference
agentsleak inputs and flags
| Name | Type | Flag | Description |
|---|---|---|---|
| mode | STRING | --mode | evaluate (BYO events, preferred) | fixture (demo) | policies (corpus only) |
| event | FOLDER | --event | Agent-action event JSON or JSONL folder, mounted for mode=evaluate |
Showing key inputs. agentsleak exposes 2 inputs in total.
example
Run agentsleak
# evaluate a folder of captured agent eventsagentsleak --mode evaluate --event ./events # emit the default policy corpus on its ownagentsleak --mode policies{ "tool": "agentsleak", "mode": "evaluate", "rule": "secret-read-then-egress", "rule_type": "sequence", "events": ["read .env", "POST https://example.com/collect"], "severity": "high", "remediation": "Deny outbound egress from agent sessions that touched credential files"}guidance
Choosing agentsleak
Reach for agentsleak when you already capture agent-action events and want them judged after the fact. Use golemhalt when you want a deterministic policy monitor at the hook point, and use gitleaks or trufflehog when the question is which secrets are sitting in the repository.
golemhalt
The reference-monitor side: Cedar and YARA policy at the agent hook. agentsleak judges the recorded stream instead of the live call.
gitleaks
Finds secrets committed to a repository. Says nothing about what an agent did with them at runtime.
trufflehog
Secret discovery with verification across sources. Complementary evidence, different question from agent behaviour.
workflows
Workflows using agentsleak
faq
agentsleak questions
related
More Vulnerabilities tools
commix
Automates OS command injection detection and exploitation.
dalfox
Parameter mining and XSS testing with headless verification.
dnsreaper
Subdomain takeover scanner with cloud-zone intake.
find-gh-poc
Locate public CVE proof-of-concept repositories on GitHub.
golemhalt
Black Hat Arsenal reference monitor for coding agents, inventoried as a policy and provider corpus.
honeymcp
Black Hat Arsenal deception layer for MCP servers, exported as a schedulable ghost-tool catalog.
Run agentsleak yourself
A folder of recorded agent-action events feeds agentsleak, which evaluates pattern and sequence rules and writes findings.
Facts on this page come from the live Trickest tool library.