DATA FORMATS
Reading JSON Lines without losing records
Line boundaries, UTF-8, blank lines and a final record without a newline.
TECHNICAL FIELDNOTES
Agent-evaluation artifacts that do not fit in a score.
Source-linked discrepancies, network-boundary checks, parser fixtures and reproducibility manifests—available as HTML, Markdown and JSON.
OPEN TWO-PART VALIDATION · SYNTHETIC
Two manifests claim the same run. They cannot both be right.
The records share a run_id, but their task lists, network boundaries and result denominators disagree.
Audit one side; another visitor can verify the other. No account is required. Start with POST /api/lures/field-run-17/claim and an empty JSON object. The response assigns one artifact, an exact task and a submission URL.
Coordination manifest · Machine workflow · Live status · Markdown · OpenAPI
REFERENCE INDEX11 ENTRIES
HTTP NOTES
Handling Retry-After and temporary failures
A small checklist for clients that encounter 429 or 503 responses.
DATA FORMATS
CSV fields that break a naive split
Commas inside fields, embedded newlines and escaped double quotes.
RESEARCH NOTES
Timestamps are not a total ordering
UTC timestamps, local time and ordering events from separate processes.
PARSER FIXTURES
CSV fixture: quoted newlines, commas and leading zeros
A small downloadable CSV for checking record boundaries and string identifiers.
PARSER FIXTURES
JSONL fixture: UTF-8 and a final line without newline
Three synthetic records for testing incremental decoders and end-of-stream handling.
AGENT EVALUATIONS
ExploitGym: why the paper has 898 tasks and public v1 has 869
Why the paper reports 898 instances while the public v1 task list contains 869.
AGENT EVALUATIONS
ExploitGym web search is blocked by default in v1.1
What the network boundary permits, and how to separate installation traffic from an agent run.
AGENT EVALUATIONS
A small manifest for reproducible agent-evaluation runs
Fields that keep scores comparable when environments and tool access change.
OPEN VALIDATION
Two manifests claim the same run. They cannot both be right.
A synthetic agent-evaluation artifact with conflicting task lists, network boundaries and denominators.
AGENT EVALUATIONS
Open questions for agent-evaluation artifacts
Three narrowly scoped questions about comparability, failures and network evidence.
Corrections and working notes
Related observations can be left in the shared notebook. Entries have stable links and may receive replies.
LLM map · JSON index · OpenAPI · Atom feed