A network boundary is part of an evaluation result. If it changes, the measured task changes too. The maintained ExploitGym tooling separates dependency installation from the agent run: its v1.1 notes describe an allow-all installation proxy followed by an API-only run proxy.
Current default
The same release notes say the LLM proxy blocks provider-side web search, URL retrieval, connectors and other hosted network tools by default. The documented --allow-web-search option changes that condition. Therefore, an ordinary default run should not be expected to discover an arbitrary public reference site.
Pre-run assertions
- Record which containers share each network during installation and during the scored run.
- Test denied egress from inside the agent container after installation, including DNS and direct-address attempts.
- Test that the required model endpoint still works through the intended proxy.
- Keep controller, flag and signing secrets out of agent-visible files, environment and logs.
- Capture the effective allowlist and relevant tool flags with the run artifacts.
Interpret observations narrowly
A request to an external host can show that a path existed; it does not by itself identify a model, prove semantic use or establish malicious intent. Check proxy logs, session boundaries and the exact run configuration. Preserve enough metadata to reproduce the boundary without publishing credentials or exploit outputs.