palimpsest

# Palimpsest

![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)
![Python](https://img.shields.io/badge/python-3.9%2B-blue.svg)
[![tests](https://github.com/beepboop2025/palimpsest/actions/workflows/tests.yml/badge.svg)](https://github.com/beepboop2025/palimpsest/actions/workflows/tests.yml)
![verify](https://img.shields.io/badge/verify-offline%2C%20one%20command-06d6e0.svg)
![data](https://img.shields.io/badge/data-public%20OSINT%20only-success.svg)
![safety](https://img.shields.io/badge/watches-the%20censor%2C%20never%20the%20censored-informational.svg)

[![DDTI refresh](https://github.com/beepboop2025/palimpsest/actions/workflows/ddti-refresh.yml/badge.svg)](https://github.com/beepboop2025/palimpsest/actions/workflows/ddti-refresh.yml)
[![Generative Firewall](https://github.com/beepboop2025/palimpsest/actions/workflows/gfi-refresh.yml/badge.svg)](https://github.com/beepboop2025/palimpsest/actions/workflows/gfi-refresh.yml)
[![GDELT cross-signal](https://github.com/beepboop2025/palimpsest/actions/workflows/gdelt-refresh.yml/badge.svg)](https://github.com/beepboop2025/palimpsest/actions/workflows/gdelt-refresh.yml)
[![GitHub-refuge](https://github.com/beepboop2025/palimpsest/actions/workflows/github-refuge-refresh.yml/badge.svg)](https://github.com/beepboop2025/palimpsest/actions/workflows/github-refuge-refresh.yml)
[![Wayback reconstruction](https://github.com/beepboop2025/palimpsest/actions/workflows/wayback-refresh.yml/badge.svg)](https://github.com/beepboop2025/palimpsest/actions/workflows/wayback-refresh.yml)

**A public, tamper-evident record of what powerful actors quietly erase, and a way for anyone
to prove, offline, that not one entry was changed after it was published.**

Palimpsest is one primitive, a sealed append-only ledger you can verify without trusting us,
pointed at two places where the record gets rewritten in the dark:

- **The [Verifiable Eval Registry](docs/EVAL-REGISTRY.md).** AI evaluation results, sealed at
  publication. The questions are frozen and hash-committed *before* any model is queried, every
  result is chained to the one before it, and a single edited number fails verification. Chinese
  state-aligned models and Western frontier models are held to the same tamper-evident,
  pre-registered machinery, each on its own frozen suite, watched over time for what they quietly
  stop answering. Not a lab, not a government, not us: if we edited a published number, our own
  verifier would report the break.
- **The Censorship Observatory.** Authoritarian deletion, measured as data. It reads the public
  record of what has been scrubbed, rewritten or blocked — across the network, the encyclopedia and
  the model — and turns what a state is burying into a live, openly licensed early-warning signal
  for journalists, researchers, and human rights defenders.
  Twenty-six signals refresh on their own, unattended, every number tracing back to public evidence.

Built entirely from open sources. **It watches the censor, never the censored.**

## Prove it yourself, in one command

```bash
git clone https://github.com/beepboop2025/palimpsest && cd palimpsest
python3 scripts/verify_eval_registry.py   # the eval chain + the pre-registration rule
python3 scripts/verify_ledger.py          # the erasure / censorship ledger
```

No install, no key, no server, standard library only. Change one sealed byte and the verifier
names the break. That is the entire idea: you do not have to trust the operator, you check.

> **Or watch it run live:** the [observatory](https://palimpsest.info/dashboards/ddti_observatory.html)
> (the live censorship signals), the [Verifiable Eval Registry](https://palimpsest.info/readings/eval-registry.html),
> and the [Generative Firewall Index](https://palimpsest.info/readings/generative-firewall-index.html).
> A ten-second, zero-dependency taste: `python3 demo/palimpsest_demo.py` pulls the live China
> Digital Times feed and ranks what the censor is focused on right now (`--source sample` runs offline).

![The Verifiable Eval Registry, live: sealed runs of Chinese state-aligned and Western frontier models, each on its own frozen suite, chain intact, with the frontier refusal-drift panel](docs/img/eval-registry.png)

> *The registry, live. Chinese state-aligned and Western frontier models under one tamper-evident,
> pre-registered machinery, each family on its own frozen suite, every run sealed, chain intact. The
> drift panel catches real events: as of the 11 July 2026 panel it had recorded one Western model
> refusing a benign legal question its three peers answered, on a probe set frozen before any of
> them was queried. Later readings are their own sealed entries; the panel is a moving record, not
> a fixed claim.*

![Palimpsest DDTI Observatory — the Censorship Fear Index and the selectivity / novelty signals](docs/img/observatory.png)

> *The observatory headline: the Censorship Fear Index (one auditable 0–100 number), the top censor
> target, and the reachable selectivity and novelty signals. Velocity is shown suppressed, never
> faked. Representative data.*

---

## Why a record that cannot be quietly rewritten

Two different kinds of evidence are becoming load-bearing, and both live in files the publishing
side can edit after the fact.

**AI evaluations.** Every serious safety claim about a frontier model now routes through evals.
Labs decide whether to ship on eval results, responsible-scaling policies trigger on them, and
regulators are starting to cite them. Yet the results sit in ordinary web pages, PDFs, and git
repos the publisher controls. If a capability number later becomes inconvenient, the cheapest
response is a quiet revision. Nobody has to lie; the page just changes, and no outsider can prove
it ever said anything different.

**Authoritarian censorship.** Before roughly 2013 a deletion often left a mark you could see and
count. Today it usually does not: a post simply stops existing, with no notice and nothing left
behind. For the people it hurts most, that silence is the point. What a state rushes to delete is
also one of the clearest readings of what it actually fears. Every deletion is a kind of confession.

Both problems have the same shape: the *before* state is unprovable. Palimpsest makes it provable.
Seal the record when it is published, and any later edit, deletion, reorder, or cherry-pick becomes
detectable by anyone, forever, without trusting the person who sealed it.

## The integrity architecture

The central claim is that the published record cannot be revised after the fact. Here is exactly
what enforces that, who each layer defends against, and, crucially, what none of it can do. A trust
claim without a threat model is marketing; the full model is in **[docs/INTEGRITY.md](docs/INTEGRITY.md)**.

| # | Layer | What it proves | Who must be defeated to fake it |
|---|-------|----------------|----------------------------------|
| 1 | Hash chain (`core/sealed_ledger.py`, `core/eval_registry.py`) | No entry was altered, reordered, or dropped within the file. The registry additionally rejects any run whose probe set was not frozen earlier in the chain. | Nobody. Anyone holding the file recomputes it offline, stdlib only. |
| 2 | Merkle root + inclusion proofs (`scripts/prove_inclusion.py`) | One 64-char value fingerprints the whole record; any single result verifies against it in log₂(N) hashes. | Same as layer 1, without needing the whole chain. |
| 3 | Public git history | Every refresh is a timestamped commit on a public repo. Rewriting it needs a force-push, visible to anyone with a clone or fork. | GitHub, plus everyone who ever cloned. |
| 4 | Internet Archive snapshots (`scripts/anchor_roots.py`) | A dated third-party copy of the exact chain bytes, outside our infrastructure and jurisdiction. | The Internet Archive. |
| 5 | OpenTimestamps / Bitcoin (`scripts/anchor_roots.py`) | The roots existed no later than a Bitcoin block time; `.ots` proofs verify against the chain, not against us. | Bitcoin's proof-of-work. |
| 6 | Independent witness (`ops/witness/`) | A from-scratch reimplementation on separate infrastructure re-verifies the served chains and checks every previously seen head is still there. Detects split views and retroactive rewrites, and alerts. | Every running witness, at once and retroactively. |

Layers 1–2 are self-verification, and are built and tested today. Layers 3–6 exist for the one
adversary self-verification cannot stop, an operator who rewrites the whole file and re-serves it,
**including us**. The anchoring step (4–5) is wired into the refresh pipeline; the witness (6) is a
single stdlib file anyone can run.

**What it does *not* protect against, stated plainly:** lying at capture time (the chain preserves
a false reading faithfully, so probes are pre-registered and raw responses are hashed for re-runs);
the short window between sealing and the first external anchor; suppression by omission (mitigated
by an open, cron-scheduled pipeline that abstains loudly rather than skipping silently); and endpoint
compromise (an attacker could append false *new* entries, but still cannot rewrite old ones without
tripping layers 3–6). The honest limits are the point, and they live in
[docs/INTEGRITY.md](docs/INTEGRITY.md).

---

## Application 1 — the Verifiable Eval Registry

A public, tamper-evident record of AI-model evaluations. See **[docs/EVAL-REGISTRY.md](docs/EVAL-REGISTRY.md)**.

- **Pre-registration by construction.** The probe set is frozen and hash-committed into the chain
  *before* any model is queried. A run whose questions were not frozen first is rejected by the
  verifier, so results cannot be cherry-picked or p-hacked after the answers exist.
- **Sealed at publication.** Each result is hash-chained to its predecessor and fingerprinted by a
  Merkle root. Edit a published number and `scripts/verify_eval_registry.py` reports the break.
- **The first live audit: cross-lab refusal drift.** The regis