nlqdb

# nlqdb — analytical memory for AI agents

> *Memory your agent can query, not just recall — a real database it reaches over MCP.*

Connect nlqdb to Claude, Cursor, Codex, or any MCP host. Your agent writes
typed rows as it learns, then asks questions in plain English — `GROUP BY`,
`JOIN`, aggregate over what it remembered. A vector store returns the top-k
similar chunks; nlqdb runs the query a similarity index structurally can't.
The LLM never emits SQL: it returns a typed plan, our compiler emits the
parameterised statement, and you see the exact SQL every time.

**It's also a natural-language database for any app.** You write HTML; each
component asks for what it wants in plain English; nlqdb infers the schema,
writes the SQL, runs it, and renders the result. There is no backend for you
to build.

Two actions. That's the whole product:

1. **Create a database** — one word: a name (or a goal).
2. **Talk to it** in plain English.

```html
<script src="https://elements.nlqdb.com/v1.js" type="module"></script>

<nlq-data
  goal="the 5 newest orders, with customer and item"
  api-key="pk_live_xxx"
  template="table"
  refresh="10s"
></nlq-data>
```

That's the entire backend for a live order list — no API to write, no
schema to define, no JSON to parse. Engine choice (Postgres / Mongo /
Redis / DuckDB / pgvector / …), schema inference, indexing, backups, and
auto-migration between engines based on your real workload are background
concerns you never have to see.

## Status — early, open

nlqdb is **early** and built in the open, but fully public — no gate, no
invite code. The marketing site, the `/v1/ask` pipeline, the `<nlq-data>` /
`<nlq-action>` elements, the chat app, the TypeScript SDK, the hosted MCP
server, and the `nlq` CLI are all live in some form (see the surface table
below). Natural-language → SQL accuracy is still climbing toward our public
bar (BIRD ≥ 0.65, Spider 2.0 ≥ 0.75 on the free model chain), so answers can
be wrong — every response carries a confidence signal and the SQL it ran.

## Use it

Connecting an agent over MCP? One command, machine-followable end to end:
[**Give your agent memory**](https://docs.nlqdb.com/agent-memory/).

The 60-second walkthrough — plain HTML, CLI, and ten framework wrappers —
lives at [`docs.nlqdb.com`](https://docs.nlqdb.com). Start with the
[HTML tutorial](https://docs.nlqdb.com/tutorials/html/) or the
[CLI tutorial](https://docs.nlqdb.com/tutorials/cli/).

You don't generate an API key separately: describe your database at
[nlqdb.com](https://nlqdb.com/?utm_source=github), and the chat hands you a
`<nlq-data>` snippet with the key already inlined.

## Examples

[`examples/`](./examples) — minimal scaffolds in plain HTML, Next.js,
Nuxt, SvelteKit, Astro, plus a CLI-only walkthrough. Each is the smallest
valid integration around one `<nlq-data>` element or one CLI session.

## What makes it different

Four things every release has to move, none allowed to regress
([`GLOBAL-025`](./docs/decisions/GLOBAL-025-north-star.md)):

- **Engine quality** — natural-language → SQL accuracy (measured
  continuously on BIRD + Spider 2.0 + an internal eval), plus the
  multi-engine layer that moves your data to the right engine for your
  workload.
- **Onboarding** — landing to first answer in under a minute, no card,
  no config.
- **UX** — see the diff before any write, see the SQL behind every
  answer, refuse rather than guess when confidence is low.
- **Performance** — sub-400 ms cached, sub-1.5 s cold.

The bet: get this right on free, open models and it only gets better on
frontier ones — the scaffolding compounds with whatever model is
underneath.

## Models & plans

- **Free forever** on the built-in open-model chain — queries, embeds,
  and the elements, no card required.
- **Bring your own LLM key** (Anthropic / OpenAI / Gemini / Grok / OpenRouter)
  on any tier, at no markup.
- **Hosted premium models** on paid plans, when you'd rather not manage a
  key of your own.
- **Self-host the source** — the engine, CLI, MCP server, and SDKs are
  source-available under [FSL-1.1-ALv2](./LICENSE): free to self-host for
  any non-competing use, bring your own LLM key, no per-call fees. The
  license auto-converts to Apache 2.0 two years after each release.

Paid plans aren't live yet. The full model strategy is in
[`GLOBAL-026`](./docs/decisions/GLOBAL-026-llm-strategy-byollm-hosted-premium.md).

## Surfaces at a glance

| Surface | Status | Where |
|---|---|---|
| HTTP API (`POST /v1/ask`, `POST /v1/run`) | ✓ shipped | `apps/api/src/ask/**` |
| `<nlq-data>` + `<nlq-action>` elements | ✓ shipped (v0.1) | `packages/elements/**` |
| `@nlqdb/sdk` (TypeScript) | ✓ shipped (incl. `runSql`) | `packages/sdk/**` |
| Framework wrappers (React / Next / Vue / Nuxt / Svelte / SvelteKit / Astro / Solid + Swift) | ~ built + CI-tested; npm / SPM publish pending | `packages/{react,next,…}/**` |
| Chat app `nlqdb.com/app` | ✓ shipped | `apps/web/**` |
| Hosted MCP server `mcp.nlqdb.com/mcp` | ✓ shipped (host auto-detect pending) | `apps/mcp/**`, `packages/mcp/**` |
| Droppable agent-memory artifacts (AGENTS.md snippet · Claude Code skill · Cursor rules · Codex config) | ✓ shipped (4/4 hosts; external registry distribution pending) | `apps/web/public/agent-artifacts/**` |
| `nlq` CLI (Go) | ✓ shipped (core verbs; device-login pending) | `cli/**` |

Full integration matrix in [`docs/progress.md`](./docs/progress.md).

## Roadmap

The detailed plan below is a summary; the canonical phase plan and exit
gates live in [`docs/phase-plan.md`](./docs/phase-plan.md). Legend:
**✓ shipped · ~ in progress · ◯ planned**.

### Phase 0 — Foundations ✓

Worker skeleton · KV + D1 + R2 bindings · Neon adapter + OTel · LLM router
(free chain) · Better Auth (GitHub + Google + magic link) · `/v1/ask`
end-to-end · events queue + drain · Stripe webhook · CI/CD + PR preview
environments.

### Phase 1 — On-ramp

A stranger lands on `nlqdb.com`, creates a DB in plain English, embeds it,
and shares the link — in under 60 seconds, no card, no config.

- ✓ Marketing site (Astro, live at `nlqdb.com`)
- ✓ `<nlq-data>` + `<nlq-action>` elements (v0.1)
- ✓ Sign-in — magic link + GitHub + Google
- ✓ Chat surface — streaming three-part response (answer / data / trace),
  anonymous mode
- ✓ Anonymous mode — 72h token, adopted onto your account on sign-in
- ✓ Hosted db.create pipeline (table-card embeddings stubbed pending the
  pgvector slice)
- ✓ API keys dashboard (`/app/keys`)
- ◯ Hello-world tutorial polish

### Phase 1.5 — Trust + telemetry

- ✓ Diff preview on writes + visible SQL trace on every response
- ✓ Demand-signal telemetry on every "not yet" path
- ◯ Confidence floor (refuse-on-low-confidence) — lands with quality-eval

### Phase 2 — Distribution (agent + developer surfaces)

- ✓ Hosted MCP server (`mcp.nlqdb.com/mcp`) — host auto-detect pending;
  local stdio `@nlqdb/mcp` built, npm publish pending
- ✓ CLI `nlq` (Go) — core verbs + raw-SQL escape hatch; device-login +
  chat REPL pending
- ✓ `@nlqdb/sdk` — basic methods + `runSql`
- ~ Framework wrappers + native Swift package — built + CI-tested; npm /
  SPM publish pending
- ✓ Quality-eval harness (BIRD + Spider 2.0, manual on-demand) — the
  free-vs-frontier accuracy delta is the headline KPI
- ~ Bring-your-own-LLM dispatch — HTTP lane live; remaining surfaces in
  progress
- ◯ CSV upload in chat
- ~ Docs-site reference completeness — SDK + framework-wrapper guides,
  an enumerable error-code reference, and a build-time `/llms.txt` for
  agents now live; tutorial polish remains
- ◯ Custom domains for embeds

### Phase 3 — Multi-engine engine (the moat)

- ◯ Workload analyzer → migration orchestrator
- ◯ ClickHouse / DuckDB / Redis as additional engines
- ◯ Dual-read verification
- ◯ Hosted-premium model lane (demand-gated)

### Phase 4 — Beyond v1

- ~ Bring-your-own Postgres / ClickHouse — connect path live end-to-end
  (`POST /v1/db/connect` + web UI, CLI, SDK, query dispatch); prod-gated on
  the `BYO_SECRET_KEK` secret
- ◯ SSO (SAML / OIDC), audit-log export, per-org quotas
- ◯ EU data residency, VPC peering, SOC 2

## Develop locally

```bash
git clone git@github.com:nlqdb/nlqdb.git && cd nlqdb
scripts/bootstrap-dev.sh   # installs everything, pulls Ollama models, seeds .envrc
scripts/login-cloud.sh     # signs you into cloud providers that have a CLI flow
```

`bootstrap-dev.sh` stands up the whole toolchain in one shot — Bun, Node
20+, Go 1.24+, uv; Biome / gofumpt / golangci-lint / ruff; lefthook git
hooks; the cloud CLIs (wrangler, flyctl, stripe, gh); a local Ollama so the
LLM router works offline; and a `.envrc` with self-generated dev secrets.
Details in
[`docs/history/infrastructure-setup.md §8`](./docs/history/infrastructure-setup.md#8-dev-toolchain).

Day-to-day:

```bash
bun run fix          # biome format + lint --write (most issues)
bun run check:all    # biome + golangci-lint + ruff (what CI runs)
bun run hooks:run    # run pre-commit hooks against staged files
```

### End-to-end tests (manual trigger)

E2E coverage is **persona-driven** and **manually triggered** so cost stays
inside the free-tier envelope — one `workflow_dispatch` workflow per
surface:

```bash
gh workflow run e2e-opencheck.yml             # web — live LLM, Neon branch, Workers preview
gh workflow run e2e-cli.yml                   # Go testscript, hermetic
gh workflow run e2e-sdk.yml                   # vitest + cassettes, hermetic
gh workflow run e2e-mcp.yml                   # InMemoryTransport protocol tests, hermetic
gh workflow run e2e-examples.yml              # Playwright across HTML/Next/Astro/Nuxt/SvelteKit
gh workflow run e2e-examples.yml -f live=true # + staging for the curl + CLI shell smokes
```

Run the hermetic surfaces locally without GitHub:

```bash
( cd tests/e2e/cli && go test ./... )
( cd tests/e2e/sdk && bun install && bun run test )
( cd tests/e2e/mcp && bun install && bun run test )
( cd tests/e2e/examples && bun install && bun run install:browse