semver-checks

[![npm version](https://img.shields.io/npm/v/semver-checks.svg)](https://www.npmjs.com/package/semver-checks)
[![CI](https://github.com/kyungseopk1m/semver-checks/actions/workflows/ci.yml/badge.svg)](https://github.com/kyungseopk1m/semver-checks/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Node.js Version](https://img.shields.io/badge/node-%5E20.0.0%20%7C%7C%20%3E%3D22.0.0-green.svg)](https://nodejs.org/)

# semver-checks

Catch the breaking changes your commit messages miss. semver-checks analyzes what actually changed in your TypeScript public API and recommends the correct semver bump.

```bash
npx semver-checks compare v1.0.0 HEAD
```

- [Why semver-checks?](#why-semver-checks)
- [Accuracy & Limitations](#accuracy--limitations)
- [Quick Start](#quick-start)
- [Programmatic API](#programmatic-api)
- [Change Rules](#change-rules)
- [CLI Reference](#cli-reference)
- [MCP Server](#mcp-server)
- [CI Integration](#ci-integration)
- [Comparison with Other Tools](#comparison-with-other-tools)
- [How It Works](#how-it-works)
- [FAQ](#faq)

## Why semver-checks?

Tools like `semantic-release` and `changesets` rely on developers writing correct commit messages. In practice, commit messages don't always reflect actual API impact — a "small refactor" that removes a required export gets published as a patch, and downstream consumers' builds break.

semver-checks **analyzes your TypeScript public API directly** using [ts-morph](https://github.com/dsherret/ts-morph) and recommends the correct SemVer bump based on what actually changed in the type signatures — not what the commit message says.

This is not hypothetical. Run it across real releases and it flags breaking type changes that shipped as minors or patches — for example, `p-limit` 6.1.0 added a required property to its exported `LimitFunction` type and was published as a _minor_; semver-checks flags it MAJOR. It is most dependable on **structural changes** — removed or renamed exports, narrowed signatures, added required parameters and properties — which it detects reliably. Equivalence-preserving type rewrites are a known weak spot it can over-report; see [Accuracy & Limitations](#accuracy--limitations) for exactly where to trust it and where not to.

```typescript
// v1.0.0
export interface Config {
  host: string;
  port: number;
}

// Developer writes: "fix: add missing timeout config"
// Published as patch — but this is a MAJOR change:
export interface Config {
  host: string;
  port: number;
  timeout: number;
}
//                                                    ^^^^^^^^^^^^^^^^ required-property-added
```

```typescript
// v1.0.0
export function findUser(id: string): User | null;

// Developer writes: "refactor: simplify findUser return"
// Published as minor — but consumers checking `result === null` silently break at runtime:
export function findUser(id: string): User;
//                                    ^^^^ return-type-changed (MAJOR)
```

semver-checks is complementary to your existing release workflow. Use it as a **verification step** before publishing — it tells you whether your intended bump is safe, or whether you're about to ship a breaking change by accident.

## Accuracy & Limitations

semver-checks grades every breaking change by **confidence**, so the CI gate stays trustworthy:

- **proven** — the change is its own evidence (a removed export, property, interface method, class property or class method; a newly required parameter, interface method or interface property), a _resolved_ type relation the analyzer decided is genuinely unrelated, or a rule that computed the answer itself (a dropped base that carried something; a required property added to a class anyone could have implemented by hand). `--strict` exits 1 on these, and only these — safe to leave on in CI. Proven is earned per rule, not inherited: a rule either computes its own confidence or is on that short list, and everything else is review-only. The list is short because it is empirical — see [Measured](#accuracy--limitations) below.
- **heuristic** — a conservative MAJOR the analyzer could _not_ prove (a type-text difference it couldn't resolve, or a one-directional change in an invariant position where a safe reading exists). These surface for human review but do **not** fail `--strict`; opt in with `--strict-review` if you want every MAJOR to gate.

This is the design's center of gravity: the equivalence-preserving rewrites and input-union widenings that make text-based type-semver tools cry wolf land in _heuristic_, off the default gate, while real under-bumps stay _proven_ and on it. It is neither _sound_ (zero false positives) nor _complete_ (catches everything), so a `proven` MAJOR is a strong signal, not a theorem. That isolation only covers the over-reporting surfaces in [Known limitations](#known-limitations); the under-report and structural rows in the same table are a different axis, a silent `patch` or an outright failure, not a confidence question.

It is most reliable on **conventional, single-entry packages with an explicitly-typed public surface**: added / removed / renamed exports, function and method signature changes, added required parameters and properties, and removed members are detected dependably and reported as `proven`.

**Measured, against a compiler.** The scorecard that decides which rules are `proven` uses `tsc` as its oracle, not the author's published bump - the tool exists because authors get the bump wrong, so scoring it against that bump would be circular. Each of 111 adjacent minor/patch release pairs, across 24 packages, has a consumer program compiled against both sides; the pair is a real break iff the new side produces errors the old side did not. Major-version boundaries are excluded: there the author already knows the release is breaking, so the tool's verdict carries no decision value.

On that corpus `--strict` fires on 36 of 111 pairs. 35 of those 36 are real breaks, so precision is 97.2%. **Recall is 81.4%: the corpus holds 43 real breaks and `--strict` stays quiet on 8 of them.**

Read the recall, not the precision. An earlier revision of this file reported 100% recall on a 75-pair corpus, and that was a fact about which shapes the corpus contained rather than about the tool: widening a corpus to 111 pairs, and re-examining every pair it had scored safe by reading the shipped `.d.ts` and writing a consumer that uses the changed symbol the way the package's own README does, took the same number to 37.8%. Grading recovered it to 67.4%, giving the variance probe the package's own declarations to read took it to 79.1%, and reading a class the way an interface was already read took it from there. Nothing was suppressed to get the precision figure; the one remaining false positive is described below.

`any confidence` still fires on all 43 pairs, so nothing here is a detection gap in the sense of the tool not noticing. What kept `--strict` quiet on most of the 14 that used to remain was that the variance probe had no verdict to give: a serialized type text is printed by the checker, so it names the types the package declares about itself, and the probe resolved those names in a program that held nothing but the ES libs. `ClassArray | ClassDictionary` in `clsx`, `P.Pattern<T> & UnknownProperties` in `ts-pattern` and `core.$ZodTypeDiscriminable<Disc>` in `zod` all sent it home empty. The probe now reads both snapshots' own declarations (see [What the probe will and will not answer about](#what-the-probe-will-and-will-not-answer-about)), which is what closes five of them.

Of the 8 left, three are not a probe question at all: on `hono` 4.12.18 -> 4.12.19, `hono` 4.12.19 -> 4.12.20 and `ky` 1.14.1 -> 1.14.2 the reported findings are provably inert while the change that actually breaks a consumer is reported nowhere, so even the `any confidence` figure is a per-pair coincidence on those three. Of the rest, `got` 14.6.4 -> 14.6.5 needs signature-level variance, `ts-pattern` 5.6.0 -> 5.6.1 is a `generic-constraint-changed`, which fires on a loosened constraint as readily as on a tightened one, and `bullmq` 5.80.11 -> 5.80.12 replaces a callback type with one that mentions `any`, where the probe bails on purpose. The last two are the probe declining to guess: `hono` 4.12.28 -> 4.12.29 widens a return type with a second `aws-lambda` type the old text never named, so the verdict would rest on two stand-ins being unrelated, and `valibot` 1.3.1 -> 1.4.0 turns a tuple parameter `readonly` through an alias the scope could not resolve.

Two shapes account for most of what the grading now catches, and both were invisible on the narrower corpus:

- **A widening that breaks readers, not writers.** An optional property gains `| null`, a union gains a member, a return type gains an alternative. Passing a value in still compiles; reading one back out into the old type does not. It appears in `ioredis`, `bullmq`, `ky`, `commander`, `hono`, `got` and `clsx`.
- **An interface gaining or losing members.** Losing one breaks callers, gaining a required one breaks implementers, and subclassing hides both because a subclass inherits whatever was added. The interface rules are `proven` for that reason. On a class the added-member rule is `proven` only where a hand-written implementation was possible in the first place: a class declaring a private or protected instance member is compared nominally, so no object literal satisfies it and there is no implementer to break.

The control group is [`scripts/gate/naive-baseline.mjs`](scripts/gate/naive-baseline.mjs), a 45-line exported-name-and-arity diff with no type resolution at all, kept to answer the obvious question: does the type analysis buy anything a much dumber tool does not already get? It scores 87.5% precision and 32.6% recall on the same corpus, against 97.2% and 81.4%. They overlap on 14 breaks, 21 belong to the ty