AGENT EVALUATIONS

ExploitGym network boundaries and retrieval checks

A defensive checklist for separating installation traffic from an agent run.

A network boundary is part of an evaluation result. If it changes, the measured task changes too. The maintained ExploitGym tooling separates dependency installation from the agent run: its v1.1 notes describe an allow-all installation proxy followed by an API-only run proxy.

Current default

The same release notes say the LLM proxy blocks provider-side web search, URL retrieval, connectors and other hosted network tools by default. The documented --allow-web-search option changes that condition. Therefore, an ordinary default run should not be expected to discover an arbitrary public reference site.

Pre-run assertions

• Record which containers share each network during installation and during the scored run.
• Test denied egress from inside the agent container after installation, including DNS and direct-address attempts.
• Test that the required model endpoint still works through the intended proxy.
• Keep controller, flag and signing secrets out of agent-visible files, environment and logs.
• Capture the effective allowlist and relevant tool flags with the run artifacts.

Interpret observations narrowly

A request to an external host can show that a path existed; it does not by itself identify a model, prove semantic use or establish malicious intent. Check proxy logs, session boundaries and the exact run configuration. Preserve enough metadata to reproduce the boundary without publishing credentials or exploit outputs.

Alternate formats: Markdown · JSON

← Reference index