HarnessScope

HarnessScope

[PyPI] [npm] [CI] [License] [Python]

Turn-level governance observability for agent harnesses. HarnessScope reads the transcripts your agent harness already writes, judges every turn against your workspace's own operating rules (plan-before-code, verify-before-done, context budget, path boundaries, …), and serves a local viewer that shows how each session traversed that governance structure — passes included, not just violations.

[HarnessScope — replay, circuit, rule health, token loss]

Every frame above is hscope demo: a synthetic corpus, generated locally. Real transcripts carry full prompt text and are never demo material.

What it found

A session is not one model talking to itself — it spawns subagents, they run on models the parent does not, and a streamed response writes the same request several times. Read that structure naively and the numbers are wrong. These are the corrections, each measured on this project's own transcripts:

The last row is the one that matters most, and it is the youngest: it was found after v0.1.3 shipped, by pointing the tool at its own workspace and reading the answer. The same class of mistake as row one, one layer further down, in code that had already passed its own tests. A tool that claims to catch a governance failure had better survive being aimed at itself — so that is now the standing use of it, and what it said next was worse.

It cost three rules their default. R1 (index-first), R2 (inline-first) and R6 (approval-gate) ask semantic questions — is this a domain question? was this fan-out warranted? is this code under that plan? — and answered them with syntactic tests: a keyword in the prompt, the prompt's character count, the project folder. Measured against real sessions, 84% of R2's violations were turns invoking a command whose entire job is to fan out. That is not a threshold in need of tuning; a proxy that does not track the concept cannot be tuned into one. They now ship disabled, each carrying the false-positive rate it was measured at (precision, a first-class field on the rule, surfaced in the model, in rule_health, and as a hoverable badge). Off is not the point — off, and here is why, with the number is the point. A rule silently shipped on is how a governance screen becomes noise nobody trusts, which is the failure this project exists to name.

R8 stopped reporting a compliance rate for the same reason. Its opportunities are its violations by construction — a re-read counter never observes a compliant re-read — so "0% compliance" was an identity, not a finding: a number that reads like an indictment and says nothing. It reports its quantity now, and no ratio.

These are observations of one corpus, not a benchmark — your numbers will differ. The point is that they are readable at all: metrics know quantities, a bill knows money, and neither knows the word "subagent". Full reasoning: docs/MULTI-AGENT.md.

How HarnessScope compares

HarnessScope watches a different layer than most "LLM observability" tools: not the app you instrumented, but the agent harness itself, read from its own transcripts.

The row nobody else covers: did the session follow the workspace's operating rules — plan-before-code, verify-before-done, path boundaries, context budget. Cost tools tell you what a session spent; tracing tools tell you what your app's LLM calls did; HarnessScope tells you whether the agent worked the way your workspace says agents must work.

Try it — 30 seconds, none of your data

pipx install hscope        # or: pip install hscope · npx -y hscope  (Python 3.11+ either way)hscope demo --serve        # synthetic corpus → ingest → judge → viewer

Three invented sessions: one compliant, one wasteful (re-reads, dead context, a boundary read, a completion declared with nothing verifying it), and one fan-out (two subagents — one cites what it read, one does not; one spawned in the background — plus an off-host MCP round trip).

The corpus is held to the same standard as real data: it must satisfy hscope doctor, so a generator writing a shape the parser does not expect gets caught by the invariants. It already earned that twice — a requestId reused across sessions (which hscope correctly read as one API call replayed by a fork), and a padded agentId: that the parser's regex swallowed whole.

Then point it at your own sessions

hscope                     # scan ~/.claude/projects → ingest → judge → check → open the viewer

No flags, no config file. Two things it will not get wrong:

• The port is found, not assumed. 8000 is a reasonable default and on some Windows machines it is a reserved range (WinError 10013). A first impression should not be a stack trace.
• It says that transcripts expire. The harness deletes them after about 30 days, and ingest is what makes them permanent. Ingest less often than that and you lose sessions silently — a session never ingested is indistinguishable from one that never happened. The first run prints how old your oldest transcript is and prescribes a cadence.

It also announces which rules it is about to judge with — neutral presets, or the workspace model it found on disk. Claiming "presets" while quietly applying a model would be the same lie as a silent fallback.

Everything is local-first: no transcript data leaves your machine (the one exception — the opt-in Tier B LLM judge — is off by default and fires only on an explicit per-session click). The tool never writes to your projects, your .mcp.json or your harness settings: a tool that observes governance must not reshape the surface it observes.

Watch it live — and only watch

hscope --watch             # the circuit stays live: sessions are re-ingested and re-judged as they run

The harness appends to the transcript while the session is still going, so the data for a live view was always there — what was missing was the loop. --watch scans mtimes every 2s, hashes only the files that moved (mtime is the trigger; the hash is still the decision), re-judges only those sessions, and the viewer refreshes itself. A tick where nothing moved parses nothing, hashes nothing and prints nothing.

It observes; it does not intervene. No hook is installed in your session, nothing is blocked, and a violation on screen is one a human has to act on. Blocking would mean a PreToolUse hook — which would make the "no hooks into the session" row above false, bind the tool to one harness, and let a wrong verdict stop your work (post-hoc, a bad call is a bad badge; live, it is a stopped hand). Seeing a re-read loop or a fan-out at 5k tokens instead of 40k is worth having; it does not require the tool to hold the brake.

The turn that is still running is shown, not judged. Judging it would break hscope's own rules: a verification-gate rule would fire on the very turn that is about to run the check — a verdict overturned three seconds later — and a streamed response's token usage is not final until its last record lands. So the live turn wears an in flight · not judged badge and its verdict appears when the turn ends. A verdict deferred must not become a verdict skipped: if you interrupt that turn the file simply goes quiet, and hscope judges it once its liveness window lapses rather than leaving it withheld forever.

What you get

Harness Circuit

[Harness Circuit]

One turn as a fixed-topology schematic: context sources → agent → action surfaces → governance outputs. Subagents are nodes with a spawn edge and a return edge — labelled with what came back, which is not what the subagent spent (one real turn: three agents burned 16.5k / 6.3k / 5.3k output tokens and handed the parent 1.5k / 2.2k / 2.0k). Off-host MCP hops and write-capable servers are flagged on the wire.

Session Replay

[Session Replay]

The turn timeline: what was loaded into context, which tools ran, and the rule verdict on each turn — passes included. A governance view that only lists violations cannot tell "compliant" from "never checked".

Rule Health

[Rule Health]

Compliance × coverage, per rule. A rule at 100% compliance over 3 evaluable turns is not the same as one at 100% over 300, and a rule nothing can evaluate is reported as such rather than scored.

Token / Loop

[Token and loss]

Per-turn token burn with loss pinpoint markers — re-reads, dead context (loaded, never cited), error churn, cache-window waste — attributed to the turn and the context window that actually paid.

Setup

Every hook, MCP server, skill and subagent you registered, next to what your transcripts show was actually called: active / unused / unregistered-call. Registration is a claim; the transcript is the evidence. (No screenshot here on purpose — this screen reads your machine's configuration, so any capture of it would be a picture of someone's real workspace.)

Rules you did not have to write

Nobody should hand-write rules to try a tool. When the first run finds no workspace model, it seeds one from your repo's structure and leaves a page for the LLM you already have:

hscope                   # ...with no rules of your own, writes ~/.hscope/<id>/HSCOPE-SETUP.mdhscope mcp --install     # prints an MCP snippet; paste it into .mcp.jsonhscope setup             # regenerate the page whenever your canon moves

HSCOPE-SETUP.md has two readers. For you: the canon, hooks, MCP servers, skills and subagents hscope found, and the canon clauses no rule cites yet. For your LLM: instructions to call get_setup_guide → suggest_rules → set_rule and write those rules. You approve; it drafts.

The whole journey is three manual actions — run hscope, paste the snippet, hand over the page. The snippet carries HSCOPE_MCP_WRITE=1, so pasting it is the write consent; there is no second step. And hscope never edits .mcp.json itself.

Did the onboarding actually work?

hscope onboard      # read-only, idempotent; exit 1 while incomplete

Six observed stages and a verdict. hscope prints it at the end of every run, so you meet it without knowing the command exists.

1  transcripts ingested             PASS   297 session(s), oldest 35d       -> oldest is 35d and the harness deletes transcripts at ~30d -- ingest more often2  hscope MCP registered            FAIL   not in your harness config       -> run `hscope mcp --install` and paste the snippet it prints3  workspace model exists           PASS   ~/.hscope/warp/workspace-model.json4  the model governs your sessions  PASS   289 of 297 session(s)5  the model is not empty           PASS   14 rule(s), 25 role(s)6  your rules actually fire         PASS   10 of 10 enabled rule(s) had an opportunity
disabled, so not judged at all: R1, R2, R6, B1
INCOMPLETE -- next: step 2, hscope MCP registered

Four of those are a checklist. Two of them fail silently, and they are why the command exists:

• Step 4 — the model governs your sessions. A model can sit on disk while its workspace.roots claim none of the paths you actually work in. hscope then judges with the neutral presets, and until this gate it said nothing about it — so you would read your own rule names on a screen that never applied one. It now prints the cwds your sessions really ran in.
• Step 6 — your rules actually fire. Twelve rules an LLM wrote for you that never get an opportunity on your corpus are not an onboarding; they are a green check over nothing. Rules with zero opportunities are named. A rule that is merely disabled is reported as off, not dead — those are different facts and conflating them is the kind of lie this tool exists to catch.

A dead rule is a finding, not a failure: coverage of zero can honestly mean "no chance yet", and failing the gate on it would reward writing fewer rules. But if nothing you wrote ever fired, the gate says INCOMPLETE — that is a vacuous pass, and hscope doctor already refuses to report one of those about your transcripts. This is the same refusal, pointed at your setup.

The workspace model (optional but recommended)

Without a model, HarnessScope runs on neutral presets: generic rules with empty domain values, path labels fall back to file names, sessions render as a flat list. Crash-free in any repo — that's the de-domaining contract.

With a model (~/.hscope/<workspace>/workspace-model.json) you declare, in one JSON document, how your workspace is governed:

• rules[] — archetype instances with your params (keywords, thresholds, markers)
• roles[] — what each path / MCP server / subagent means (label, circuit node, host boundary)
• gates[] — your governance pipeline (e.g. context-load → plan-approval → implement → verify → ledger)
• unmeasurable[] — canon clauses no detector can check (surfaced honestly instead of faked)
• work_items — optional grouping above sessions (this workspace: plans/** with a status frontmatter; other repos can map issues/PRs into the same slot). Absent ⇒ flat sessions.

Schema and a neutral example live in hscope/schema/. Validate yours:

python -m hscope.schema.validate_model ~/.hscope/<workspace>/workspace-model.json

MCP surface

Any MCP client can read what HarnessScope observed — and the loop closes: an agent opens a session by reading how the last one scored, instead of re-earning the finding.

// .mcp.json{ "mcpServers": { "harness-scope": {    "command": "python", "args": ["-m", "hscope.cli", "mcp", "--db", "/path/to/harness_scope.db"] } } }

No model runs inside this server. The client is the model: get_setup_guide hands it the closed archetype library and each archetype's param schema, suggest_rules hands it the canon clauses no rule cites, and the judgment comes back through set_rule — where the params are validated against that archetype's schema before anything is written. So the MCP path needs no provider, no credential and no off-host consent. hscope compile does the same job in one batch LLM call; this is its conversational, incremental counterpart.

Writes are off by default (HSCOPE_MCP_WRITE=1) and can only ever touch ~/.hscope/<workspace>/workspace-model.json. There is no tool that writes a project file or steers another agent — HarnessScope observes and judges; it does not drive.

Docs

FAQ

What is HarnessScope? A local-first observability tool for Claude Code: it parses your transcript JSONL files into SQLite, judges every turn against your workspace's own operating rules, and serves a web viewer showing how each session traversed that governance structure. Install with pip install hscope (or npx -y hscope).

Does my transcript data leave my machine? No. Parsing, rule evaluation and the viewer are all local; the server binds 127.0.0.1. The one exception is the opt-in Tier B LLM judge, which is off by default and sends a single per-session digest only when you click that session's judge button.

Do I need an API key or an LLM to run it? No. There is no model inside HarnessScope — the rule engine is deterministic. An LLM only enters the picture where you bring your own: the optional Tier B judge, or your coding agent authoring rules over the MCP surface.

How is this different from ccusage, LangSmith, or Phoenix? Different layer (see the comparison table above). Cost CLIs total what a session spent; tracing platforms instrument your application's LLM calls. HarnessScope reads the harness's own transcripts and judges governance: whether sessions planned before coding, verified before claiming done, stayed inside path boundaries, and where their tokens actually died.

Can my agent read the results? Yes — that is the point of the MCP surface. Any MCP client gets session_report, token_hotspots, rule_health and the workspace model, so the next session opens by reading how the last one scored instead of re-earning the finding.

What happens when Claude Code changes its transcript format? hscope doctor fails loudly, naming the harness version that broke an invariant, instead of quietly emitting a wrong number. This corpus alone declares 29 harness versions in one month; drift is detected, not tuned away.

Which platforms does it run on? Windows, macOS and Linux — the CI matrix runs the verification harnesses on all three. Python 3.10+ (3.11+ recommended).

Data & privacy

• The SQLite DB contains full prompt text from your transcripts. It is gitignored (*.db) and must stay local — never commit or upload it.
• The server binds 127.0.0.1 by default and is read-only over the DB.
• Tier B (LLM judge) sends a per-session digest off-host only when you click the judge button for that session; the rule (B1) ships disabled.

License & open-core boundary

HarnessScope is Apache-2.0 open core. Everything an individual user touches is free forever, with no feature limits — paid tiers (if/when they exist) only ever add things an organization needs, never remove things from individuals:

See LICENSE (Apache-2.0), ee/README.md (boundary), and CONTRIBUTING.md (DCO).