Palimpsest — censorship, China economy and model-eval observatory

Palimpsest

[License: MIT] [Python] [tests] [verify] [data] [safety] [Sponsor Palimpsest]

[DDTI refresh] [Generative Firewall] [GDELT cross-signal] [GitHub-refuge] [Wayback reconstruction]

A public, tamper-evident record of what powerful actors quietly erase, and a way for anyone to detect, offline, whether the served record still reproduces its published commitments.

Palimpsest is one primitive, a sealed append-only ledger you can verify without trusting us, pointed at two places where the record gets rewritten in the dark:

• The Verifiable Eval Registry. AI evaluation results, sealed at publication. The questions are frozen and hash-committed before any model is queried, every result is chained to the one before it, and a single edited number fails verification. Chinese state-aligned models and Western frontier models are held to the same tamper-evident, pre-registered machinery, each on its own frozen suite, watched over time for what they quietly stop answering. Not a lab, not a government, not us: if we edited a published number, our own verifier would report the break.
• The Censorship Observatory. Authoritarian deletion, measured as data. It reads the public record of what has been scrubbed, rewritten or blocked — across the network, the encyclopedia and the model — and turns what a state is burying into a live, openly licensed early-warning signal for journalists, researchers, and human rights defenders. Twenty-six signals refresh on their own, unattended, every number tracing back to public evidence.
• The AI Eval Journal. Evidence-bound essays about why the evals exist, what changed in their methods, what failed, and what the record can claim today. Every article keeps its limitations, falsifier, verification commands and exact artifact receipts; the linked readings remain authoritative.
• Live Eval Findings. Deterministic analysis rebuilt from the newest verified refusal-drift panel. Every analytical sentence binds to an exact evidence selector, while controls, denominators, intervals, counterreadings and immutable revisions remain available beside the finding.

Built entirely from open sources. It watches the censor, never the censored.

Use Palimpsest for source-bound China evidence

For a connected institutional workflow, see institutional evidence routing. LiquiLens is the front door for institution stress and risk-review questions; Palimpsest supplies relevant economic, information-control and verifiable evaluation evidence under its own coverage and rights contracts.

Palimpsest is the China-evidence layer of a broader financial-research fleet. Its money-market guide exposes official CFETS repo fixings, SHIBOR and USD/CNY parity with economic-period, release and collection clocks. Its capital-market guide is deliberately narrower: official HKEX Stock Connect aggregate observations, not general equity, bond, derivatives or investor-level coverage. The China economy API guide shows keyless REST and release-bound MCP 1.9.3 examples while keeping revisions, rights, coverage and abstentions attached.

At the 24 August 2026 publication snapshot, the broad China read and named-series forecast targets were warming_up; this is an abstention, not a validated forecast. Re-read the linked current artifacts before citing status. For system funding and money/capital transmission use Seiche, for institution risk use LiquiLens, and for exit liquidity use Undertow. The canonical Financial Evidence Agent Skill performs that routing without merging the products’ evidence boundaries. Agents can discover Palimpsest itself through its AI catalog.

For companion funding, bank and exit-liquidity research, the free agent starter kit configures Seiche, LiquiLens and Undertow, with Hermes and OpenClaw setup guides. Palimpsest is added separately through its China economy API guide above; the kit does not include its MCP connection or expand its publication rights.

Follow Belt and Road evidence without flattening it

The Belt and Road Observatory publishes a global source, rights, project-lifecycle, finance and local-impact contract with priority depth for CPEC and Gwadar in Pakistan, CMEC and Kyaukpyu in Myanmar, and seven deliberately separate Balochistan political, civic, armed, legal, state-action, rights and political-economy lanes. Its NarcoScope overlay joins only aggregate country/time context; it cannot identify an actor or establish a political, criminal, corridor or causal link.

The separately licensed World Bank WDI bundle adds 3,564 annual country-indicator rows for China, Myanmar and Pakistan. Observed, forecast and unavailable values remain distinct, and national context is never treated as project evidence. Exact Release A publication is bound by the immutable Pages receipt and receipt schema: commit 14b06772dfed6cdc736279c9ab61b444e5846598, workflow run 32984946320, and cache-busted served-byte verification at 2026-08-26T15:55:34Z. The receipt’s fresh_until clock is a 24-hour point-in-time availability statement, not continuous monitoring and not a new WDI observation date.

Fund independent validation

Palimpsest has already frozen and published a 145-row validation sample, its codebook and the agreement script. The immediate campaign has a planned $1,800 target for two independent Mandarin-speaking coders. Their completed coding sheets, agreement result and limitations will be published even if the result rejects the labelling scheme.

Fund the study on GitHub monthly or once, starting at $1. You can also inspect the campaign and direct crypto routes, review the public funding ledger, or use the Giveth project page. The Giveth verification application is submitted but not yet approved, and the project is not yet GIVbacks eligible.

Ongoing support pays for vantage points, archives, model credits and independent human review. It never buys a say in the research questions, methods, findings or publication schedule. Card checkout is handled on GitHub; Palimpsest does not collect or store donor card or bank details.

Palimpsest evaluates how Chinese and state-aligned language models respond to documented events and criticism of the Chinese Communist Party. The Generative Firewall and evaluation registry bind each result to declared questions, retained response evidence, uncertainty and detectable revisions. A screenshot alone cannot establish a repeatable pattern or a model maker's motive; every claim stays scoped to its named panel, suite and timestamp.

Prove it yourself, in one command

git clone https://github.com/beepboop2025/palimpsest && cd palimpsestpython3 scripts/reproduce_all.py

That wrapper runs the sealed-chain verifiers and rebuilds the weekly situation, gazetteer phylogeny and collector-health scorecards from committed files. Exit 0 means they still match. The individual commands remain:

python3 scripts/verify_eval_registry.py   # the eval chain + the pre-registration rulepython3 scripts/verify_refusal_transcripts.py  # current frontier text -> seal -> labelspython3 -m scripts.build_eval_assurance --check # the claim ceiling matches the evidencepython3 -m scripts.build_eval_findings --check # live findings match the sealed panelpython3 scripts/verify_ledger.py          # the erasure / censorship ledgerpython3 scripts/evidence_capsule.py verify protocol/test-vectors/palimpsest-erasure-v1.json

No install, no key, no server, standard library only. Change one sealed byte and the verifier names the break. That is the entire idea: you do not have to trust the operator, you check.

Need to carry one claim and its exact supporting bytes into another newsroom, research notebook or agent? Evidence Capsules package the evidence, typed claims and explicit limitations into one inert JSON file with the same offline verification model.

The reviewed ScamShield intelligence pack applies that boundary to scam research: 18 public sources and 8 typologies across three distinct dimensions, with support levels that describe evidence relationships rather than guilt or risk. ScamShield can create a private local Evidence Capsule while raw Telegram text stays hashed and is not sent by default; public candidates omit messages, exact indicators and owner-only liquidity values, require human review, and are never auto-published. The complete boundary is documented in integrations/scamshield/README.md.

The newsroom also exposes Whispers from the Dragon Den: human-reviewed, source-free analytical context derived from eligible public-channel capsules, with its own RSS and JSON Feed. Raw public-channel forwards remain in separate, warned Telegram channels: the catch-all feed, cyber and technology, and regional and borderlands. They never enter the website artifact without the separate human-review and sanitization path.

The China Situation desk now places publisher reports, exact-link institutional Instagram/Telegram observations, reviewed source-free Dragon Whispers, and Palimpsest Observatory measurements in one auditable view. It is not a generic aggregator: each layer keeps its own source relationship, coverage receipt, revision identity, and limitation. The social registry is closed and public; Instagram uses Meta's official API only, and the authenticated Telegram handoff can append sanitized versions without gaining authority over Instagram or corroboration. Its structured index and dedicated RSS/JSON feeds are linked from the public feed directory.

Or watch it run live: the observatory (the live censorship signals), the Verifiable Eval Registry, the AI Eval Journal, Live Eval Findings, and the Generative Firewall Index. A ten-second, zero-dependency taste: python3 demo/palimpsest_demo.py pulls the live China Digital Times feed and ranks what the censor is focused on right now (--source sample runs offline).

The full operational view is OSINT China: the public roll-up over every China-facing reading, including each source's cadence, freshness deadline, coverage state and complete machine-readable payload. A failed or stale collector stays on the board as a visible gap; it is never replaced by a plausible-looking zero.

An independent hourly recovery watchdog evaluates the published evidence deadlines, not just the last saved health labels. It re-dispatches an idle, reviewed producer for stale/missing/corrupt active sources, refreshes an old command bundle before trusting its embedded states, and caps each recovery wave to prevent a retry storm. Operational health is kept separate from analytic readiness: for example, a complete monthly Believability collection is live while its eight-month divergence baseline is still warming up. Intentionally disabled optional methods remain visible but are not reported as dead required schedulers.

The same page now exposes those retained payloads as a searchable structured ledger and adds an evidence spine across four deliberately separate lanes: information controls, monetary plumbing, aggregate illicit-market observables, and reviewed scam/laundering signals. The versioned intelligence-commons manifest connects Palimpsest, ScamShield, NarcoScope, Seiche and LiquiLens through typed, directional contracts. A connection is context, not causation: typology matches do not establish predicate offences or source of funds; private models, raw messages, exact IOCs and operational leads never cross into the public commons.

Evidence newsroom and reporting gate

The Palimpsest Wire now separates fast evidence briefs from publication-ready reporting. Its closed primary-document archive preserves exact official-source vintages and separate publication/retrieval clocks; its corroboration ledger requires an explicit human decision before a second source group counts; and its frozen network rounds retain protocol, ASN/region, same-round controls and outage scope without manufacturing a national censorship percentage.

The public Reporting Standards surface shows why every explainer or investigation is publishable or blocked. The deeper profiles require primary evidence, independent groups, history, counterevidence, relevant expert and affected voices, an explanatory visual, sentence-level citations, limitations and human editing. Investigations add a skeptical expert, independent fact-check, right-to-reply, visible updates, falsification and source safety. Passing a gate never publishes automatically. Protected interview notes are accepted only as externally encrypted bytes and never enter the public repository. Architecture and exact contracts: docs/REPORTING-NEWSROOM-V2.md.

The optional private-runtime layer is an explicit deployment bridge. An operator sets the repository variable NEMESIS_SNAPSHOT_URL to one static public-snapshot HTTPS endpoint and supplies a separate NEMESIS_SNAPSHOT_HMAC_KEY secret. The hourly roll-up authenticates the exact response bytes against the endpoint's .hmac-sha256 sidecar, then reconstructs only the closed, versioned palimpsest-nemesis.public-snapshot contract. It refuses redirects, private-network targets, oversized responses, invalid UTF-8, duplicate keys, non-finite numbers, unknown fields at every depth, contradictory health, and future-dated evidence. The accepted document is written atomically as readings/nemesis-latest.json, after which the roll-up is rebuilt, sealed and tested. If the URL is unset, the bridge remains visibly absent. If it is configured but fails authentication or validation, publication fails loudly.

[The Verifiable Eval Registry, live: sealed runs of Chinese state-aligned and Western frontier models, each on its own frozen suite, chain intact, with the frontier refusal-drift panel]

The registry, live. Chinese state-aligned and Western frontier models under one tamper-evident, pre-registered machinery, each family on its own frozen suite, every run sealed, chain intact. The drift panel catches real events: as of the 11 July 2026 panel it had recorded one Western model refusing a benign legal question its three peers answered, on a probe set frozen before any of them was queried. Later readings are their own sealed entries; the panel is a moving record, not a fixed claim.

[Palimpsest DDTI Observatory — the Censorship Fear Index and the selectivity / novelty signals]

The observatory headline: the Censorship Fear Index (one auditable 0–100 number), the top censor target, and the reachable selectivity and novelty signals. Velocity is shown suppressed, never faked. Representative data.

Why a record that cannot be quietly rewritten

Two different kinds of evidence are becoming load-bearing, and both live in files the publishing side can edit after the fact.

AI evaluations. Every serious safety claim about a frontier model now routes through evals. Labs decide whether to ship on eval results, responsible-scaling policies trigger on them, and regulators are starting to cite them. Yet the results sit in ordinary web pages, PDFs, and git repos the publisher controls. If a capability number later becomes inconvenient, the cheapest response is a quiet revision. Nobody has to lie; the page just changes, and no outsider can prove it ever said anything different.

Authoritarian censorship. Before roughly 2013 a deletion often left a mark you could see and count. Today it usually does not: a post simply stops existing, with no notice and nothing left behind. For the people it hurts most, that silence is the point. What a state rushes to delete is also one of the clearest readings of what it actually fears. Every deletion is a kind of confession.

Both problems have the same shape: without a commitment, the before state is unprovable. Palimpsest freezes that state. Hashes make an edit, deletion, reorder or cherry-pick detectable within the served record; public history, external anchors and witnesses make whole-record rewrites observable outside the operator's infrastructure. The trust boundary is explicit rather than compressed into the word “immutable.”

The integrity architecture

The central claim is that revision of the published record is detectable under the stated threat model. Here is exactly what enforces that, who each layer defends against, and, crucially, what none of it can do. A trust claim without a threat model is marketing; the full model is in docs/INTEGRITY.md.

Layers 1–2 are self-verification, and are built and tested today. Layers 3–6 exist for the one adversary self-verification cannot stop, an operator who rewrites the whole file and re-serves it, including us. The anchoring step (4–5) is wired into the refresh pipeline; the witness (6) is a single stdlib file anyone can run.

What it does not protect against, stated plainly: lying at capture time (the chain preserves a false reading faithfully, so probes are pre-registered and raw responses are hashed for re-runs); the short window between sealing and the first external anchor; suppression by omission (mitigated by an open, cron-scheduled pipeline that abstains loudly rather than skipping silently); and endpoint compromise (an attacker could append false new entries, but still cannot rewrite old ones without tripping layers 3–6). The honest limits are the point, and they live in docs/INTEGRITY.md.

Application 1 — the Verifiable Eval Registry

A public, tamper-evident record of AI-model evaluations. See docs/EVAL-REGISTRY.md.

• Pre-registration by construction. The probe set is frozen and hash-committed into the chain before a run can be accepted. Frontier v2 binds exact prompt text. GFI v2 additionally refuses the first model query until its exact protocol has been committed and pushed publicly; legacy GFI v1 committed concept identifiers and remains labelled as partial assurance.
• Sealed at publication. Each result is hash-chained to its predecessor and fingerprinted by a Merkle root. Edit a published number and scripts/verify_eval_registry.py reports the break.
• The first live audit: cross-lab refusal drift. The registry runs separate frozen suites with no model in common: cn-sensitive-generative-firewall-v1 (DeepSeek, Qwen) and frontier-overrefusal-v1/-v2 (OpenAI, Anthropic, Meta, Mistral). No model has ever been run under both. What is shared is the machinery, not the questions: the same tamper-evident, pre-registered apparatus audits a state-aligned model and a Western frontier model, each on its own frozen suite, tracked run over run for what a model quietly stops answering, an undisclosed behavioral change no changelog admits.
• Measured so it survives review. Every rate carries a Wilson interval and every suite publishes the smallest change a single look could detect. The standing alarm is a mixture supermartingale, so its false-alarm rate is bounded over the lifetime of a watch that re-reads every model every six hours, rather than per look. Each question is asked in three meaning-preserving wordings, because refusal behaviour is sensitive to phrasing and one wording cannot tell a policy from a tripwire. A frozen anchor set re-scores the classifier on every run, so an instrument change can never be published as a model change. Design and honest limits: docs/FRONTIER-DRIFT.md.
• The labels are recomputable, not just tamper-evident. The v2 suite seals a hash over the raw response digests and publishes the responses, so a reader can check that the text served is the text sealed, re-derive every label, and disagree with ours on the record. That closes most of the gap docs/INTEGRITY.md previously had to concede, where a mislabelled response would be sealed perfectly and verify clean forever.
• No vanity assurance score. The generated eval-assurance-latest.json reports integrity, prompt precommitment, raw-response recomputation, pipeline reproducibility, statistical design, human construct validation and independent replication separately. Its current ceiling is provisional-measurement; human coding and unaffiliated replication remain unfinished.

python3 scripts/verify_eval_registry.py        # chain integrity + the pre-registration rulepython3 scripts/verify_refusal_transcripts.py  # published text -> sealed hash -> labelspython3 -m scripts.build_eval_assurance --check # evidence -> public claim ceilingpython3 scripts/prove_inclusion.py 5           # inclusion proof for a single sealed result

Live: palimpsest.info/readings/eval-registry.html.

Application 2 — the Censorship Observatory

Continuous, quantified measurement of content-layer censorship: what gets deleted, how selectively, how fast, and what is newly sensitive. It fills the gap between network-layer measurement (OONI, GreatFire, Citizen Lab) and hand-documented deletion lists (China Digital Times); it ingests their public data and shares its own back.

The method: treat the censor as a sensor. Palimpsest reads the public record of what has already been removed — China Digital Times' curated deletion and directive coverage, alongside the network and model layers — and computes the Deletion-Differential Threat Index (DDTI) from what the censor chose to touch:

Velocity is listed because it is the third thing you would want and the second thing we would publish, not because it is available. A CDT item carries an editorial date, not a deletion timestamp, so no latency is derivable from this source and the reading omits it rather than estimating it. Measuring it needs the archive-and-recheck instrument below.

The index is therefore attention allocation, not a deletion rate — the reading says so in its own scope field: censor_attention_allocation (numerator-only; not a true deletion rate). It ranks what drew the censor's attention; it does not claim to know what share of posts on a topic were removed, because that denominator is not observable from outside.

What is built but not running. The censorwatch package implements the archive-and-recheck method properly: capture a public post as it appears, re-check it on an age-tiered schedule, and confirm a deletion only after repeated consistent observations. It is feature-flagged (CENSORWATCH_ENABLED), has never been enabled in production, and has archived zero posts. It is not an in-country China sensor. Collection methods that close visibility gaps from outside the wall — archives, public ledgers, opt-in observers, donation hashes, synthetic calibration — are specified in docs/GREYBALL-METHODS.md. Until CensorWatch is deliberately enabled it remains a description of an instrument, not of a running signal, and nothing on the board is derived from it.

The DDTI distils into a single, auditable 0–100 Censorship Fear Index, how hard is the state working to bury things right now, reported component by component, never a black box.

Validated by retrodiction. Run against six documented events (Li Wenliang, Peng Shuai, the Sitong Bridge protest, the White Paper protests, and more), the scorer ranks the correct term first every time and flags event-born euphemisms as novel from only a handful of deletions. Reproduce it: PYTHONPATH=. python3 scripts/validate_ddti.py. See docs/VALIDATION.md and the method in docs/METHODOLOGY.md.

Published signals and operational status

palimpsest.info publishes static artifacts from independent pipelines. Most refresh on declared schedules through GitHub Actions, while disabled, optional, stale, and abstaining methods remain visible with their own status. A current file timestamp never overrides an upstream limitation. Runs, code, and outputs are public, and no hidden server publishes them.

The economic registry records each source's implementation and access state; registered sources are distinct from collected observations. See config/china_econ_sources.json, the coverage report (PYTHONPATH=. python -m scripts.china_econ_plan), and docs/CHINA-ECONOMIC-OBSERVATORY.md. Current records do not include proprietary respondent networks or microdata.

Every value is provenance-tracked to its source document, and a signal abstains rather than fabricates when its source returns nothing. Nothing is published without evidence. Researcher docs, schemas, and citation (BibTeX) are at palimpsest.info/for-researchers.

It generalises beyond China. The method is country-agnostic; what changes per information space is the lexicon. China ships today; Iran loads from config alone (the Woman-Life-Freedom-era starter lexicon). Adding a country is a gazetteer plus a registry entry, not a rewrite. See config/regions/.

What is built, and what is not

Velocity was the honest blocker on the censorship side: from outside the wall, the moment a post dies is unobservable. The Wayback Reconstruction vantage now recovers it retroactively from open egress, reading the Internet Archive's capture timeline so every deletion is published as an explicit archive-witnessed bracket (last seen alive to first seen gone), never a false-precise instant. What still needs in-country or seam vantage is real-time velocity at minute resolution. The method built for that, UNDERTEXT many-vantage differential observation (disagreement between vantage points is the signal), is built and tested here; what scaling adds is the vantage backends. See docs/UNDERTEXT.md.

Safety is the architecture

See SAFETY.md and docs/ETHICS.md. In short: public data only; nobody inside China is ever asked to act; a deletion is never claimed lightly (the detector probes a known-live control post each cycle and suppresses all deletion writes when the network is unreliable); the sensitive-terms gazetteer is human-authored and never delegated to a Beijing-aligned model; and no state-aligned model is ever the analyst. Every figure ships with its uncertainty and known biases stated openly. Those rules are enforced in code (core/governance.py), not just documented.

Running it

# Zero-dependency demo (recommended first run), no venv needed:python3 demo/palimpsest_demo.py                 # live CDT pull + rankingpython3 demo/palimpsest_demo.py --source sample # offline deletion demo
# Verify the sealed records (stdlib only, no install):python3 scripts/verify_eval_registry.py         # eval chain + pre-registration rulepython3 scripts/verify_ledger.py                # the erasure ledgerpython3 scripts/prove_inclusion.py 5            # inclusion proof for one attestationpython3 ops/witness/palimpsest_witness.py       # become an independent witness
# Pure, offline cores (no database):PYTHONPATH=. python3 scripts/validate_ddti.py       # retrodiction backtest (6/6 events)PYTHONPATH=. python3 scripts/fear_index_demo.py     # Fear Index across documented eventsPYTHONPATH=. python3 scripts/forecaster_demo.py     # the censorship forecaster (a "called shot")
# Tests, core (stdlib only, no install, no venv):PYTHONPATH=. python3 -m pytest tests/ -q                      # stdlib only, no install needed
# Tests, full (adds the censorwatch layer, which needs the dependencies):python3 -m venv .venv && source .venv/bin/activate && pip install -r requirements.txtPYTHONPATH=. python3 -m pytest tests/ censorwatch/tests/ -q   # adds the velocity-leg suite

The live velocity leg needs PostgreSQL, Redis, and in-country / seam egress; see censorwatch/DEPLOY.md. It stays inert unless CENSORWATCH_ENABLED is set.

Documentation

Status and license

Developed in the open as a public good. This is past the prototype stage and short of a finished product: twenty-eight feeds refresh unattended on public CI, the sealed chains verify offline from a clean clone, and the whole thing runs without anyone touching it. What it is not yet is independently replicated, externally witnessed at scale, or in-country. Those are named openly in docs/INTEGRITY.md and in the built-versus-not table above rather than smoothed over. Free and open source; it is not a commercial product and never monetizes the people or topics it observes. Licensed under the MIT License so other tools can freely build on the feeds and reuse the measurement and sealing layers.

Acknowledgements and prior art

Palimpsest is built to complement, not repeat, the work of China Digital Times, GreatFire, Citizen Lab, and OONI. It ingests CDT deletion data as one input and is designed to share its data back. It draws on the academic measurement tradition of WeiboScope and the deletion-speed studies of Zhu et al. (2013) and Bamman et al. (2012), whose decade-old figures it re-measures rather than assumes. The sealing layer draws on trusted-archive integrity work (ARCHANGEL) and standard transparency-log constructions.