Codex Autoresearch Loop

Codex Autoresearch

Skill by ara.so — Daily 2026 Skills collection.

Codex Autoresearch is a Codex skill that runs an autonomous modify→verify→keep/revert loop on your codebase. You describe a measurable goal in one sentence; Codex confirms the plan, then iterates unattended — every improvement stacks in git, every failure reverts automatically — until interrupted or a cap is reached. Inspired by Karpathy's autoresearch concept, generalized beyond ML training to any software metric.

Installation

Option A — manual copy into your project:

git clone https://github.com/leo-lilinxiao/codex-autoresearch.gitcp -r codex-autoresearch your-project/.agents/skills/codex-autoresearch

Option B — Codex skill installer:

$skill-installer install https://github.com/leo-lilinxiao/codex-autoresearch

The skill lives at .agents/skills/codex-autoresearch/ inside your project. No config file is required before first use.

How to Activate

Open Codex in your project directory and prefix your goal with $codex-autoresearch:

$codex-autoresearchI want to get rid of all `any` types in my TypeScript code

Codex will:

1. Scan the repo and infer scope, metric, verify command, and guard command.
2. Present a confirmation summary — reply go (or correct anything).
3. Run the loop unattended until you interrupt it or the goal is met.

You never write config. Codex infers everything.

Confirmation Flow

Before the loop starts Codex always shows what it found and asks you to confirm. Example exchange:

Codex: I found 47 `any` occurrences across src/**/*.ts.
       Confirmed:       - Target: eliminate `any` types in src/**/*.ts       - Metric: `any` count (current: 47), direction: lower       - Verify: grep + tsc --noEmit as guard
       Need to confirm:       - Run until all gone, or cap at N iterations?
       Reply "go" to start, or tell me what to change.
You:   Go, run overnight.
Codex: Starting — baseline: 47. Iterating until interrupted.

Up to five confirmation rounds are possible. After that, Codex proceeds.

The Loop (internals)

PHASE 0: Probe environment (CPU/GPU/RAM/toolchains), check for session resumePHASE 1: Read context + lessons file from prior run (if any)
LOOP (forever or N times):  1. Review current state, git history, results log, lessons  2. Pick ONE hypothesis (apply perspectives, filter by environment)     -- or N hypotheses if parallel mode is active  3. Make ONE atomic change  4. git commit (before verification)  5. Run verify command  →  did the target metric improve?     Run guard command   →  did anything else break?  6. Improved → keep (extract lesson)     Worse    → approved rollback strategy (git revert)     Crashed  → fix or skip  7. Log the result to results log  8. Health check (disk, git, verify health)  9. If 3+ discards → REFINE; 5+ → PIVOT; 2 PIVOTs → web search 10. Repeat. Never stop. Never ask.

The loop runs unbounded unless you say Iterations: N during confirmation.

Dual-Gate Verification

Two commands serve distinct purposes:

Guard files are never modified by the loop.

Example verify + guard pair for a Python coverage run:

Verify: pytest --cov=src --cov-report=term 2>&1 | grep TOTAL | awk '{print $NF}'Guard:  python -m mypy src --ignore-missing-imports

Example for TypeScript type cleanup:

Verify: grep -r "any" src --include="*.ts" | wc -lGuard:  npx tsc --noEmit

Modes

Codex maps your sentence to one of seven modes automatically — you never pick a mode explicitly.

loop — iterate toward a measurable target (default)

$codex-autoresearchImprove test coverage in src/ to at least 80%

$codex-autoresearchReduce bundle size — it's currently 2.3 MB, get it under 1 MB

plan — turn a vague goal into a validated loop config

$codex-autoresearchI want to make our API faster but I don't know where to start

Codex will interview you (p95 latency vs throughput? which endpoint?) and produce a ready-to-run loop config.

fix — repair errors until count reaches zero

$codex-autoresearchpytest is failing, 12 tests broken after the refactor — fix them all

debug — evidence-driven root-cause hunting

$codex-autoresearchOur API returns 503 randomly under load, no idea why

Each iteration tests one falsifiable hypothesis. Codex presents evidence, not guesses.

security — read-only STRIDE + OWASP audit

$codex-autoresearchIs this code secure?

ship — readiness verification and release gating

$codex-autoresearchShip it

exec — one-shot execution with no loop

$codex-autoresearchRun the benchmark suite and summarize results

Inline Configuration (optional)

You can override defaults inline during the confirmation step — no file edits needed:

Example during confirmation:

You:   Go. Iterations: 30, Guard: npm test, Scope: src/api/

Cross-Run Learning

At the end of each iteration Codex writes a structured lesson to .agents/skills/codex-autoresearch/lessons.md:

Iteration 7 — KEPTHypothesis: replace explicit `any` with inferred generic in src/utils/mapper.tsChange: added <T extends Record<string, unknown>> to mapKeys()Result: any count 31 → 29Lesson: Generic constraints on utility functions eliminate clusters of `any` downstream.

On session resume Codex reads this file first. Each new run benefits from prior runs.

To resume an interrupted run:

$codex-autoresearchResume

Codex re-reads the lessons file, checks git state, re-establishes the baseline, and continues.

Parallel Experiments

Request parallel mode during confirmation or at any time:

You:   Go, parallel 4

Codex runs four hypotheses concurrently, keeps the best result, discards the rest. Useful when hypothesis space is large.

Pivot Protocol

If the loop stalls, escalation happens automatically:

You are never asked for permission during escalation. The loop continues.

Real Code Examples

Example 1 — TypeScript any elimination (Python verify script)

If you want a custom verify script instead of a one-liner:

# scripts/count_any.pyimport subprocess, sys
result = subprocess.run(    ["grep", "-r", "--include=*.ts", r"\bany\b", "src/"],    capture_output=True, text=True)count = len(result.stdout.strip().splitlines())print(count)sys.exit(0)  # always exit 0; the number is what matters

Tell Codex during confirmation:

Verify: python scripts/count_any.pyGuard:  npx tsc --noEmit

Example 2 — pytest coverage loop (Python)

# scripts/coverage_pct.pyimport subprocess, re, sys
out = subprocess.check_output(    ["pytest", "--cov=src", "--cov-report=term", "-q"],    stderr=subprocess.STDOUT, text=True)match = re.search(r"TOTAL\s+\d+\s+\d+\s+(\d+)%", out)if match:    print(int(match.group(1)))    sys.exit(0)print(0)sys.exit(0)

$codex-autoresearchImprove test coverage — target 85%
Verify: python scripts/coverage_pct.pyGuard:  python -m mypy srcDirection: higherTarget: 85Iterations: 50

Example 3 — bundle size loop (Node.js project)

# scripts/bundle_size.sh#!/usr/bin/env bashnpm run build --silent 2>/dev/nulldu -k dist/bundle.js | awk '{print $1}'

$codex-autoresearchReduce our JS bundle size, currently ~2300 KB, target under 900 KB
Verify: bash scripts/bundle_size.shGuard:  npm testDirection: lowerTarget: 900

Example 4 — lint warning count (any language)

# scripts/lint_count.sh#!/usr/bin/env bashnpx eslint src/ --format json 2>/dev/null \  | python3 -c "import sys,json; d=json.load(sys.stdin); print(sum(len(f['messages']) for f in d))"

$codex-autoresearchGet our ESLint warning count to zero
Verify: bash scripts/lint_count.shDirection: lowerTarget: 0

Unattended Runs

For overnight or long runs, ensure Codex CLI approval settings do not interrupt git commit or git revert commands. The simplest option is to run in a disposable or sandboxed repo clone:

git clone . /tmp/autoresearch-sandboxcd /tmp/autoresearch-sandbox# launch Codex here with full permissions

Results accumulate in git history. Pull the winning commits back to your main repo when done:

# in your main repogit fetch /tmp/autoresearch-sandbox maingit cherry-pick <winning-commit-sha>

Session Artifacts

These files persist across Codex sessions. Delete them to start fresh.

Troubleshooting

Loop reverts every change:

• Verify command may be returning a non-numeric value. Test it manually: bash -c "<your verify command>" should print a single number.
• Metric direction may be wrong. Confirm Direction: lower or Direction: higher during setup.

Guard fires on unrelated files:

• Narrow scope: Scope: src/specific-module/
• Or tell Codex explicitly: Do not touch tests/ during confirmation.

Session resume picks up wrong baseline:

• Delete session.json to force a fresh baseline: rm .agents/skills/codex-autoresearch/session.json

Parallel mode produces merge conflicts:

• Codex handles this internally via the pivot protocol, but if it gets stuck, reduce parallelism: Parallel: 2

Codex asks questions mid-loop:

• This means a guard crash produced ambiguous output. Pre-empt it by specifying Guard: <command> || true if guard failures should be non-fatal, or by giving Codex fuller sandbox permissions so it can run git commands freely.

Loop hits PIVOT but makes no progress:

• Supply a seed hypothesis during confirmation: Hint: try tree-shaking unused imports first
• Or run plan mode first to produce a richer hypothesis list before switching to loop.

Quick Reference

# Start a loop$codex-autoresearch<your goal in one sentence>
# Resume interrupted run$codex-autoresearchResume
# Bounded run$codex-autoresearch<goal> — Iterations: 25
# Parallel hypotheses$codex-autoresearch<goal> — Parallel: 4
# Force a mode$codex-autoresearch fixpytest has 8 failures, repair them
# Read-only audit$codex-autoresearch securityAudit src/api/ for injection vulnerabilities