Web AI Agent Audit

Overview

Embedded web agents are a public-site attack surface. Injection, prompt leak, jailbreak, and role confusion need algorithmic probes on the pages you care about, not offline model eval theater.

This is a standing check for continuous security testing. You seed page URLs, probe live hosts with httpx, run BHUSA 2026 Arsenal WebAgentAudit for agent-oriented checks, and correlate findings you can schedule as a retest delta when the site or agent changes.

It sits next to MCP Trust Boundary Suite (MCP server config). This Collection unit is the public-page agent path: URL inventory in, WebAgentAudit evidence out.

Pipeline

1. Seed · Page URLs supplies the target URL list for the run.
2. Prep · Normalize URL list cleans and dedupes that inventory.
3. Inventory · HTTP probe uses httpx to confirm live pages.
4. Scan · WebAgentAudit runs injection, prompt leak, jailbreak, and role-confusion probes. Set mode=assess for probes; detect inventories widgets.
5. Results · Correlate merges scan output into findings, inventory, and retest_delta outputs.

Inputs

• Page URL list. Pages that embed a chat or agent. Demo seeds exist; replace with your inventory.
• httpx. Live-host probe ahead of the agent scan.
• WebAgentAudit. Private vault Arsenal tool. mode=detect inventories widgets; mode=assess runs probes.

Outputs

• findings.jsonl. Agent probe hits with remediation hints.
• inventory.jsonl. URLs and agents observed.
• retest_delta.jsonl. Schedule-as-diff scaffold when pages or agent configs change.

Integrations

• httpx
• WebAgentAudit (BHUSA 2026 Arsenal)

Sample output

Sample shape from the correlate pack (illustrative, not a live KPI).

kind	title	severity	note
finding	Prompt leak or jailbreak pattern on embedded agent	high	null
inventory	null	null	Live page URL observed ahead of WebAgentAudit
retest_delta	null	null	Schedule re-runs when the agent or page changes

FAQ

Is this an offline model eval?

No. It probes web-embedded agents on the URLs you seed. Edit Seed, then set Scan mode=assess when you want probes.

Do I need LLM API keys on the graph?

No. WebAgentAudit is the Arsenal probe path. Optional keys, if a mode asks for them, can live in vault or on the node.

Why schedule this?

Correlate emits retest_delta.jsonl so scheduled re-runs can surface agent posture change after deploys.

Related workflows

• MCP Trust Boundary Suite. Reach for this instead when the surface is MCP server config, not an embedded web agent.
• Coding Agent Policy Gate. Reach for this instead when the job is coding-agent action policy on the operator machine, not an embedded web agent.
• XSS Scanner. Reach for this instead when the job is classic web reflection bugs, not agent prompt abuse.