TensorCAD

# TensorCAD

<p align="center">
  <b>Schematic capture for neural network architectures.</b><br/>
  Draw the model, get the numbers, generate the PyTorch.
</p>

<p align="center">
  <a href="https://github.com/Filip-Pajalic/TensorCAD/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/Filip-Pajalic/TensorCAD/ci.yml?branch=main&style=flat-square&label=CI" alt="CI status" /></a>
  <a href="https://tensorcad.dev"><img src="https://img.shields.io/badge/try%20it-tensorcad.dev-2f6fb0?style=flat-square" alt="Open the editor in a browser" /></a>
  <img src="https://img.shields.io/badge/license-MIT-111111?style=flat-square" alt="MIT licensed" />
  <img src="https://img.shields.io/badge/engine-Go%20%E2%86%92%20WebAssembly-2f6fb0?style=flat-square" alt="Go engine compiled to WebAssembly" />
  <img src="https://img.shields.io/badge/presets-23%20verified-1b6834?style=flat-square" alt="23 verified presets" />
  <img src="https://img.shields.io/badge/tests-339%20passing-1b6834?style=flat-square" alt="339 tests passing" />
  <img src="https://img.shields.io/badge/MCP-server%20included-8a5b9c?style=flat-square" alt="MCP server included" />
</p>

<p align="center">
  <a href="https://tensorcad.dev">Open the editor</a> ·
  <a href="#quick-start">Quick start</a> ·
  <a href="#what-it-does">What it does</a> ·
  <a href="#how-it-is-kept-honest">Correctness</a> ·
  <a href="#for-agents">For agents</a> ·
  <a href="https://docs.tensorcad.dev">Docs</a> ·
  <a href="./CONTRIBUTING.md">Contributing</a> ·
  <a href="./ROADMAP.md">Roadmap</a> ·
  <a href="./LICENSE.md">License</a>
</p>

---

TensorCAD treats a neural network the way an EDA tool treats a circuit. Blocks
are symbols with typed pins. Tensors are nets. Shapes are checked by a real
algebra rather than by running the thing. A design-rule check tells you the
model will not fit on your GPUs *before* you rent them. And the drawing is not
a picture of the model — it **is** the model, and PyTorch falls out of it.

It started as a tool for language models. It now draws vision transformers and
convolutional classifiers too, because the same machinery turned out to work.

<p align="center">
  <img src="docs/images/editor.png" alt="The TensorCAD editor showing Llama-3-8B: a schematic of the model on the left, with the parameter count, memory budget and design-rule findings beside it" width="100%" />
</p>

<p align="center">
  <sub>Llama-3-8B, one level open. Every net carries its shape; the readout is
  measured under the operating point at the top of it, and the title block in
  the corner says how the count compares with the published one.</sub>
</p>

<table>
<tr>
<td width="50%" valign="top">
<img src="docs/images/cluster.png" alt="The cluster panel listing ways to split the training across eight H100s, each with the memory it needs per device" width="100%" />
<br/><sub><b>Fit it on the cluster.</b> Every split the cluster admits, priced
and ordered by how little it asks of you. Pressing one applies it, and the whole
readout follows.</sub>
</td>
<td width="50%" valign="top">
<img src="docs/images/ladder.png" alt="The ladder panel showing the same design at four widths, from 512 to 4096, with the parameter count at each" width="100%" />
<br/><sub><b>Sweep small, run big.</b> The same design at several widths, with
what to multiply the initialization and the learning rate by at each. Press a
rung to open it.</sub>
</td>
</tr>
</table>

<p align="center">
  <img src="docs/images/volume.png" alt="The volume view: the model drawn as stacked three-dimensional blocks of weights and activations, with flow ribbons between them" width="62%" />
</p>

<p align="center">
  <sub>The same model as volumes, a port of
  <a href="https://bbycroft.net/llm">Brendan Bycroft's LLM visualisation</a> (MIT).
  Blue is a weight, green an activation, and a ribbon is a tensor on its way
  somewhere.</sub>
</p>

Every picture here is regenerated by `bun run scripts/screenshots.ts` from a
running dev server, so it is what the tool looks like rather than what it
looked like once. `bun run scripts/export-svg.ts` writes the sheet itself as a
vector — [one is in `docs/images`](docs/images/llama-3-8b-sheet.svg) — which is
also what `File > Export the sheet as SVG` does from the editor.

**It runs in a browser: [tensorcad.dev](https://tensorcad.dev).** There is no
server behind it — the engine is the same WebAssembly module the command line
and the MCP server load, so every number on the screen is computed in the tab,
and this build has nowhere to send a design even if it wanted to: it registers
no storage provider, so `File > Save a copy` writes to your disk and that is
the only copy there is.

A separate deployment at [app.tensorcad.dev](https://app.tensorcad.dev) is this
same editor with an account attached, where designs are saved and can be
shared. The first claim holds there too — the analysis still runs in the tab —
but the second does not, which is why they are two sentences and two addresses
rather than one of each.

The documentation is at [docs.tensorcad.dev](https://docs.tensorcad.dev).

## Quick start

Needs [Bun](https://bun.sh) and [Go](https://go.dev) 1.25 or later. The analysis
engine is Go compiled to WebAssembly, which is the one build step:

```bash
git clone https://github.com/Filip-Pajalic/TensorCAD
cd TensorCAD
bun install
bun run build:wasm
```

Every number in the project comes out of one command:

```bash
bun run scripts/report.ts
```

```
preset              calculated     published   delta
------------------------------------------------------------
gpt2-small              124.4M        124.4M   exact
llama-3-8b               8.03B         8.03B   exact
deepseek-v3            671.03B       671.00B   0.0039%
ijepa-vit-h14            1.28B         1.28B   exact
alexnet                  61.1M         61.1M   exact
```

Generate a model and check it against real PyTorch:

```bash
bun run scripts/codegen-demo.ts llama-3-8b      # writes out/llama-3-8b/model.py
python -m tensorcad_runtime verify out/llama-3-8b/model.py
```

Open the editor:

```bash
bun run --cwd packages/ui dev      # browser
cd desktop && wails3 task build    # desktop app (Wails v3 + Go)
```

Both load the same engine. So do the command line and the MCP server, which is
the point: the numbers cannot depend on where you asked for them.

## What it does

**Schematic capture.** Blocks, typed pins, orthogonal wire routing, four-sided
pin anchors, junction dots on branching nets, hollow circles on unconnected
pins. Containers unfold in place so a 32-layer stack reads as one frame with a
`32×` bracket, the way published architecture figures draw it.

**A feature timeline.** Every edit is kept as an operation with its arguments,
not as a snapshot, so a step can be taken out of the middle and everything
after it replays on top of what is left. Suppress the step that widened the
model and the rename you did afterwards survives. A step that cannot replay —
because you suppressed the one that added the block it wired — says what it
could not find rather than being dropped in silence.

**Tensors you can point at.** A wire is a tensor, and clicking one says what it
carries: its shape, its dtype, the block that made it, every block that reads
it, and its share of the activation memory. Every segment of the same net
lights with it. A block that fans out — Nemotron-H's `split` holds 290 MiB
across three output pins, 128, 160 and 2 — is where that matters: its own
number answers neither which of them is the big one nor what dropping one would
save.

**A real shape algebra.** Every tensor dimension is a multivariate polynomial
with exact rational coefficients over named symbols. `B` and `T` stay
indeterminate all the way through, so a mismatch is a genuine polynomial
difference rather than two numbers that happened not to match. Splits carry
divisibility obligations instead of silently rounding.

**Design-rule checks.** Eighteen rules: head divisibility, vocabulary padding,
RoPE dimension parity, interface breakage, whether the design fits the GPUs you
selected under the sharding plan you chose. The DRC panel correctly refuses
Llama-3-8B at 90.16 GiB/GPU against an H100's 80.

**Quantitative analysis.** Parameters, FLOPs, activation memory (Megatron
formulas), KV cache, ZeRO/FSDP/TP/PP sharding, roofline throughput, Chinchilla
budgets. Nothing in the UI computes its own numbers; one `validate()` call per
document and operating point feeds every panel.

**PyTorch generation.** `generateTorch(doc)` emits a runnable model with an
`init_weights()` method — because `nn.Embedding` defaults to a unit normal, and
that is the difference between a next-token loss of 466 and 10.94 against the
`ln(50257) = 10.82` baseline.

**A 3D volume view.** Every tensor as a plate, sized by its real dimensions,
with flow ribbons between them. Ported from Brendan Bycroft's
[LLM visualisation](https://github.com/bbycroft/llm-viz).

**An MCP server.** So an agent can design, validate, analyse and generate
without a human in the loop.

## How it is kept honest

This is the part worth reading, and the reason to trust the numbers.

**Twenty presets are the regression suite.** Each one carries the parameter
count its authors published, and the tests assert the analysis reproduces it.
Seventeen match *to the parameter*; the other three are checked against rounded
vendor figures with an explicit tolerance.

**Every preset is instantiated in real PyTorch.** `python -m tensorcad_runtime
verify` builds the generated model on the meta device and reports its true
parameter count, module by module, up to DeepSeek-V3 at 671,026,419,200.

**FLOPs are checked against a profiler.** For AlexNet the agreement is exact —
1,428,376,960 per image, ratio 1.000000 against `torch.utils.flop_counter`. For
GPT-2 small the profiler says 251.78 MFLOP/token and the analysis says 249.42,
and *the whole difference is the causal mask*: a profiler counts the attenti