Convert Claude Code or Codex into the most intelligent Deep Research Agent. Collect, search, and synthesize web research into a persistent, searchable wiki that builds on itself. Hosted API + MCP: hyperresearch.ai
Codex bench harness
Runs DeepResearch-Bench queries through hyperresearch on the OpenAI Codex CLI and writes RACE-compatible JSONL (id, prompt, article). It is the Codex counterpart of bench/harness.py, and it also measures whether Codex actually ran the pipeline: an April 2026 attempt answered every query inline and wrote zero vault notes.
What one query does
- Creates
codex_bench/runs/query_<id>/, runsgit initandhyperresearch init, thenhyperresearch install . --target codex --json. That writes the vault,AGENTS.md,.agents/skills/hyperresearch/, the step files under.hyperresearch/codex/steps/, the custom agents under.codex/agents/, and the Stop hook in.codex/hooks.json. The harness writes none of these itself. It copies this repo's.hyperresearch/config.tomlinto the run vault and renders at its gear, the same way the Claude bench does. - Writes
research/prompt.txt(the verbatim query, which the pipeline treats as gospel) andresearch/wrapper_contract.json(inline[N]citations plus the Opinionated Synthesis sections). - Checks what Codex will see with
codex debug prompt-input, which calls no model: whether the skill is listed and under what name, whetherAGENTS.mdloaded, and whether network access is on. - Runs
codex exec. The prompt goes in on stdin, starting with$hyperresearch <query>, and the JSONL event stream goes torun.log. - Reads the report from
research/notes/final_report_<vault_tag>.mdand records adherence metrics inresult.json.
Prerequisites
- Codex CLI, installed and logged in (
codex --version,codex login). Tested against codex-cli 0.155.1. - hyperresearch with
install --target codex. The harness uses the repo's.venvif it exists and puts that directory first onPATHfor the Codex session. - The query file:
python codex_bench/harness.py --download-only.
Sandbox and network
codex exec defaults to a read-only sandbox with no network. That is what killed the April run: every hyperresearch call came back "rejected: blocked by policy". The harness passes:
--sandbox workspace-write -c sandbox_workspace_write.network_access=true
It also adds ~/.crawl4ai as a writable root when that directory exists (turn this off with --no-auto-add-dirs), and passes --dangerously-bypass-hook-trust so the project Stop hook (hyperresearch run stop-gate) runs in a run dir Codex has never seen (turn this off with --no-hook-trust). If sandboxed fetches still fail with permission errors, --yolo switches to --dangerously-bypass-approvals-and-sandbox. Use it only for these throwaway run dirs. Nothing here writes to ~/.codex/config.toml.
Skill naming: Codex prefixes project skills with the name of the nearest ancestor .codex-plugin/plugin.json. This repo has one, so a run dir under codex_bench/runs/ lists the skill as hyperresearch:hyperresearch. The harness reads the listed name from the pre-flight check and puts that name in the prompt. The prompt also tells Codex to read .agents/skills/hyperresearch/SKILL.md directly if the skill did not load.
Examples
python codex_bench/harness.py --query-id 52 --dry-run # set up, check, print the codex command
python codex_bench/harness.py --query-id 52 # one query, Codex's default model
python codex_bench/harness.py --query-id 52 --query-id 61 -m gpt-5.5 --reasoning-effort high
python codex_bench/harness.py --limit 5 --only-en --resume # first 5 English ids without a result yet
python codex_bench/harness.py --summary # adherence table over all runs
python codex_bench/harness.py --export # results/hyperresearch-codex-<model>.jsonl
Other options: --timeout (seconds per query, default 3h; light-tier queries finish well inside it), --web-search live|cached|disabled, --gear, --add-dir, --ephemeral, -c key=value (passed through to codex exec), and --verbose.
The old --evaluate flag is gone. It called bench/evaluate.py, which grades bench/runs/ and never saw these results. Run --export and give the JSONL to the DeepResearch-Bench RACE evaluator. Forcing a tier is not supported either, because step 1 classifies the tier and nothing overrides it.
result.json
article: the report with any YAML frontmatter removed. It is exported only whenerroris null.error_code:PIPELINE_SKIPPED: zero vault notes and no run manifest. Codex answered inline or did nothing. This is the April failure.NO_REPORT: the pipeline started but nofinal_report_*.mdwas written.TIMEOUT,CODEX_ERROR(non-zero exit),SETUP_FAILED,CODEX_NOT_FOUND.
adherence:notes:.mdfiles inresearch/notes/other than the report. This is roughly the number of sources fetched.manifest,vault_tag,run_status,blocked_on,profile,tier: read fromresearch/runs/<tag>/run.jsonandprompt-decomposition.json.steps_done,steps_skipped,steps_failed,steps_not_done: step status from the manifest, compared with the profile's step list. Codex records a step only when it runshyperresearch run step.manifest_counters: the manifest's source, note, and agent counters. The dollar estimate is dropped.verify: the result ofhyperresearch run verify <tag> -j, as passed or failed plus the names of the failed checks.escalations_queued: blocked fetches still in the queue. Codex has no browser lane, so these stay queued.
events: counts from the event stream. They cover turns, shell commands,hyperresearchcalls,fetchcalls,run stepcalls, web searches, file changes, subagent spawns, and sandbox rejections, plus token totals. A spawn is any event item whose type or tool name contains "spawn".probe: the pre-flight check from step 3 above.