digline

dev.diglinePublished by digline.devv0.8.1Updated Nov 7, 2026

Read, measure and explain LLM eval runs of a digline suite. Cannot promote a baseline.

Overview

AI-generated overview

Local MCP server that lets a coding agent read, compare and explain digline LLM eval runs, and measure a new run, without promoting baselines.

What you can do
  • List, read and explain stored runs, baselines and model logs for a suite
  • Compare a run against the baseline, or diff two runs against each other
  • Execute a suite after acknowledging the planned number of target calls
  • Inspect recording status and recorded answers by case and setup
  • Suggest a first suite as facts, with the command that writes it
  • Report where the project is and the command for the next step
When to use it
Use it when a repository already holds digline eval suites and you want an agent to inspect results, judge whether a change made things worse, or run the suite with an explicit call budget. It is not a general eval authoring or baseline-promotion tool.
Requirements
Python package installed with uv pip install digline-mcp, run as a local stdio server. One server per repository, started with --root pointing at the repo and a required --names option (clear or project, the latter needing --name-table). --tenant and --env are accepted only to verify. No accounts, keys or environment variables are declared.
Before you install
Every tool, including read-only ones, loads the named suite, and a suite.py executes the project's code in the server's environment. run spends model calls and requires acknowledge_calls matching the suite's planned calls. Responses are verdict-only projections, but they still enter model context and transcripts. Baseline promotion is deliberately absent.

Installation

In SourceWeft

  1. Open digline in the dashboard and add it to a workspace.
  2. Enable the server for the chats that should use its tools.

Desktop only via STDIO. STDIO servers start a local process, so they need the SourceWeft desktop host.

Other MCP clients

Follow the launch instructions in the repository.

README

digline-mcp

The MCP server for digline: it lets a coding agent read a digline result and measure a new one, and it does not let the agent promote a baseline.

Not because promotion is refused. Because there is no such tool.

The thirteen tools

toolwhat it does
list_runsevery stored run of a suite, newest first, with the baseline marked
get_runone stored run — the verdicts, never the payload
get_baselinethe approved reference, same shape
logwhich model answered, read down the stored runs; never a gate
comparea run against the baseline: did it get worse?
difftwo runs, neither of them a reference: should I switch?
explainone run read back as typed facts, with the exit code
runexecute the suite, with the call count acknowledged first
wherewhere the project is, and the command for the next step
record_statuswhether each tenant records, and how much
record_listinga suite's recorded answers, counted by the shape of the output; never a yes or a no
record_comparea suite's recorded answers by case and by setup, each by its number in the answer
suggesta first suite, proposed as facts, with the command that writes it; writes nothing, asks no model

promote is absent by construction, and so are migrate, view, report, a record import that writes a yes or a no, and record on and off. A baseline is an approved reference, and promote writes into .digline/<tenant>/baselines/, which is committed — anything landing there arrives in somebody's diff and must arrive because they put it there. A refusal would be a conversation an agent could argue with; an absence is not.

Every tool, the reading ones included, loads the suite it is named. For a suite.py that executes the project's code, in the server's environment.

The reasoning is ADR 0011, and the judgement layer it turns into surface is AGENTS.md.

Install and configure

bash
uv pip install digline-mcp

One server serves one repository: the perimeter is the repo, and multi-project is N named servers rather than a registry.

json
{  "mcpServers": {    "digline-northwind": {      "command": "digline-mcp",      "args": ["--root", "/Users/you/src/northwind", "--names", "clear"]    }  }}

--tenant and --env are accepted and verify; they never override. The suite decides, as it does everywhere else in digline.

--names is required, with no default: clear lets the store's names leave as they are, and project projects every answer, so the names leave as tokens from the name table that --name-table module:attribute names; a refusal is not projected yet, and is a sentence (ADR 0054 §8). A configuration with --root alone no longer starts; --names clear keeps what it did. ADR 0054 has the reasons.

What run costs, before it costs it

run takes a mandatory acknowledge_calls that must equal the suite's planned calls to the target. Called without it, the tool refuses and hands back the number:

run(suite="suite.py")  → refused: this suite plans 100 calls to the target.    20 cases × 5 samples = 100 calls to the target; each answer is judged    3 times by llm_rubric.    Call again with acknowledge_calls=100.
run(suite="suite.py", acknowledge_calls=100)   → executes

The first call is the probe. An agent cannot spend a hundred model calls without having stated the number — AGENTS.md §7 as a contract rather than as advice.

What never crosses

An MCP response goes into a model's context, and from there into transcripts and caches nobody controls. So the run tools return a verdict-only projection: the name, the identity, the status, the score, the threshold, the tolerance, the measured interval, and the metadata a suite disclosed.

Never the judge's reason — the judge quotes the output, so the reason is the output. Never the sentence explaining a suspension, never undisclosed metadata, never a case's inputs, and never a prompt or its digest unless the suite declares Disclosure(artifact_body=True). A declared file outside the perimeter is refused when the run is made; one under .digline or .git is refused by the command that would let it cross. Neither is dropped in silence.

Source: packages/digline-mcp/README.md at commit 1b5cf08

Tools

0
SourceWeft started this package once in an isolated sandbox, without credentials, to list its tools. That failed: the package did not install.

Version history

7
  1. v0.8.1LatestOct 31, 2026
  2. v0.8.0Oct 29, 2026
  3. v0.6.3Oct 26, 2026
  4. v0.6.2Oct 21, 2026
  5. v0.6.1Oct 19, 2026
  6. v0.6.0Oct 18, 2026
  7. v0.5.0Oct 12, 2026

Score

50% of the applicable checks met

How complete this listing is, computed from the registry entry, the endpoint and public sources. Not a rating of the server and not a statement that it is safe or working today: reachability is a dated check, and no installs or reviews go into the score.

  • Reached in the last 30 days25 pts

    Keep the endpoint answering the MCP handshake; SourceWeft probes it every 30 days.

  • Auth declared10 pts

    Declare the credential headers in the registry entry, or keep the endpoint answering the probe so its auth can be classified.

  • Tools listed15 pts

    Answer tools/list without a credential, or ship a package that starts in a sandbox without one.

  • DescriptionRequired15 pts
  • Logo10 pts
  • Documentation15 pts
  • License10 pts